@HaraldHanche-Olsen Doesn't \expanded obsolete that?

@FaheemMitha not really, it affects the uses just as it affects the uses of toks registers for controlling ## but sometimes you still want to expand a single token not a whole list.

I thought $2^n-1$ was for a complete expansion, but never having done it, I don't really know.

@FaheemMitha no, it expands one token, n times.

@DavidCarlisle Oh.

Can one use the url package in plain TeX, or does one have to switch to LaTeX?

@FaheemMitha You can always re-use the code with appropriate adjustment

@JosephWright But there is no plain TeX equivalent?

@FaheemMitha It's not really a plain concept, is it?

@FaheemMitha Seems it does work:

\input miniltx %
\input url.sty %
\url{foo.br}
\bye

@JosephWright Oh, is that how you do it?

@JosephWright I've no idea. I was just trying to keep things "simple". I could use LaTeX, I guess. Doesn't really matter.

@FaheemMitha Well, url uses various LaTeX things like \@namedef, but they are all 'core' ones covered by miniltx

@JosephWright I see. Just by accident?

@FaheemMitha The thing with plain is it's good for basic tests, but for most users it's not idea for 'real world' use. Of course, there are people who like the idea of 'doing everything themselves', but unless one is a real expert, that's problematic. And one needs repeatedly to set the same things up.

@JosephWright So, LaTeX for testing, then?

@FaheemMitha Not really: miniltx is there to provide 'enough LaTeX' to allow various pretty generic packages to load (see for example graphics)

I've no intention of using plain for real world use.

@JosephWright Is that the same as /usr/share/texlive/texmf-dist/tex/plain/graphics-pln/miniltx.tex?

@FaheemMitha No, for testing TeX core concepts. For example, most engine bugs are shown in plain TeX demos as this allows one to know exactly what is going on.

@JosephWright I mean, should one use LaTeX for basic testing? Most people seem to use plain TeX, presumably because there is then less "going on".

@FaheemMitha If you look at the tex-core questiones, they are all demod with plain

It probably doesn't matter most of the time.

@JosephWright Ok.

@FaheemMitha It depends what you are doing. Like I say, for something like showing an engine bug I'd use plain

@JosephWright Ok.

Have yet to see an engine bug.

So if one wants to expand out:

\def\testa{\testb}
\def\testb{\testc}
\def\testc{abc}

One can use:

\expandafter\url\expandafter{\expanded{\testa}}

But if one is not using \expanded, more \expandafters would be required. Is this correct usage?

Though I'm not sure one can't just change \url to fully expand its argument before proceeding. Is there some reason for that?

@FaheemMitha yes, but the correct usage would be quite a few \expandafters:

\def\testa{\testb}
\def\testb{\testc}
\def\testc{abc}
\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter\detokenize\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter{\testa}

@FaheemMitha what if the URL is using special characters? I guess there are corner cases where this would break things.

@FaheemMitha what if the URL contains a ~, that would certainly break in full expansion.

@Skillmon What I meant to ask is, am I using \expanded correctly? Given that I have yet to read any actual documentation on it.

@Skillmon Does \url detokenize or similar by default?

Oh, and I meant to ask. \expanded and \expandafter seem different in usage. \expanded seems to expand everything between the following braces. I.e. \expanded{xxx} expands everything in xxx (balanced text?). But \expandafter just operates directly on the token following the next token.

Question: where should user defined lua modules go in the texmf tree? Perhaps that's documented somewhere...

This has probably been asked before, but how can I propagate an error from \write18 to TeX? assuming this is even possible.

@FaheemMitha depends what type of error?

@yo' Standard error, usually.

Basically, the question is how to send standard error from inside a \write18 call to standard error for the TeX file.

Historical question: when and by who was the term "balanced text" coined?

TeX by Topic first uses the term on pg 262, but does not define it.

@FaheemMitha There was one discussed here recently (for luatex, not etex or Knuthian tex). Things got a bit confused because my examples were in plain tex, whereas others were thinking latex. Anyway: ↓↓

@JosephWright and @FaheemMitha -- When you think you've found an engine bug and want to report it, it won't be accepted unless it's demonstrated with plain. Instructions are found on Knuth's TeX web page.

@FaheemMitha It's a pretty standard term in computer science.

@FaheemMitha There's Perl module Text::Balanced which has been around since 1997 for example.

@FaheemMitha And it's an instance of the language a^nb^n which is the classic example of a context free language (i.e., not a regular language).

@FaheemMitha Yes. But also, \expandafter performs just a single expansion, while \expanded expands recursively – with some exceptions: Tokens preceded by \noexpand are not expanded, neither are macros defined with \protected\def, and the contents of \the\sometokenregister are not expanded.

@HaraldHanche-Olsen Yes, I remember seeing that.

@barbarabeeton Noted, but it's rather unlikely to happen. And I think you're talking about the original (Knuth) TeX, right?

Do such rules apply to the other engines?

@FaheemMitha Maybe not formalised, but in practice, almost certainly.

@HaraldHanche-Olsen Do those exceptions apply to both \expandafter as well as \expanded?

@AlanMunn Ok. I'm not familiar with that terminology. Can anyone point me to a TeX-specific definition?

@HaraldHanche-Olsen As I understand it, \expanded does a complete expansion, like \edef.

But again, no documentation.

@FaheemMitha Not really, as \expandafter does just a single expansion. But hang on, let me run an experiment or two.

@FaheemMitha It's very similar, yes.

@HaraldHanche-Olsen If those exceptions are to \expanded, then where did you get your information from? LuaTeX?

There's an informal discussion on pg 32 of the LuaTeX manual.

With a fair number of grammatical errors.

@FaheemMitha \expandafter\foo\noexpand\bar behaves just like \foo\bar, as you might expect.

@FaheemMitha Basically, I took what I know about \edef (as documented in the TeXbook) and verified by experiment that \expanded behaves similarly. But \protected is a later addition (etex).

And that appears to be it for the LuaTeX manual. Which appears to have a fairly relaxed approach to documentation, actually. I guess they're busy writing the code. And no actual example usage of \expanded.

@HaraldHanche-Olsen I see. That sounds sensible. Maybe you should write the documentation. :-)

@HaraldHanche-Olsen I got the impression from the discussion that the LuaTeX bug you discovered was a regression. Is that accurate?

@FaheemMitha According to @UlrikeFischer, it is. I have no first hand knowledge of this.

@FaheemMitha As far as I know, \expanded was studied for inclusion in pdftex, but it didn't find its way as development basically stopped. The code was used in luatex, and then very recently the primitive was added to all other engines. It is basically like \begingroup\edef\x{\endgroup<tokens>}\x, but without the assignment, so it can be used in an expansion context and it just requires one expansion step.

@egreg It just requires on expansion step, meaning if it is expanded once it then expands the "balanced text" following it completely?

@HaraldHanche-Olsen I can't be completly sure. Hans wrote in 2015 that it was resolved, but I don't remember if I tested at that time and I didn't check with older luatex now.

(Please excuse if I'm using the "expansion" terminology incorrectly.)

I just looked for the term "balanced text" in the TeXBook as well. It's there, but no definition. So it's a list of tokens surrounded by braces? Anything else?

I know that Joseph Wright has used that term often, but I don't recall if he gave a definition.

Actually, I think he's the first person I recall using the term, but I'm sure others have too. I probably wasn't paying attention.

@FaheemMitha There is no formal definition of balanced text; it means tokens where explicit braces (more precisely, tokens with category code 1 and 2) are properly balanced in the common sense of the term.

@egreg So a starting and ending brace?

@FaheemMitha Also {a{b}c} and a{b}c

@egreg Oh.

@FaheemMitha TeX's syntax has “general text” that specifies braces around a “balanced text”.

@FaheemMitha -- Yes, the original. But never say never -- there are several reports in the queue for the 2020 review.

@egreg I don't quite parse that sentence. Is "general text" then just a list of tokens?

@barbarabeeton How many of them look like real bugs?

@FaheemMitha A “general text” is {<balanced text><right brace>

@egreg Not following. So balanced text with braces around it?

@FaheemMitha Yes: {abc} is an instance of “general text”, with abc as “balanced text”.

@FaheemMitha -- At least one, if I remember correctly. But depending on the nature of the problem, and the extent of the likely consequences if changed, it could be declared a "feature" and handled by documentation.

@egreg So "general text" is a subset of "balanced text"? Doesn't "balanced" require braces, then?

@barbarabeeton I see. Have they been posted online somewhere?

@FaheemMitha I don't follow: “general text” is used in some syntax rules. It doesn't make much sense in other contexts.

@egreg Well, I'm not sure what it meant by the term. And I'm also still fuzzy about "balanced text". I thought it meant beginning and terminating in braces, but apparently that is not the case.

This is why definitions are useful.

@FaheemMitha No, it's not the case: a\foo b is an instance of a balanced text.

@egreg Ok.

@FaheemMitha Each token of category code 1 increments an internal counter, each token of category code 2 decrements it; at each point the counter must be nonnegative and at the end it should be zero.

@egreg Oh. Ok, that's easier to understand. That's the definition of "balanced text"?

Is that actually how it is handled internally?

@FaheemMitha Yes. When the syntax rule requires a general text, TeX expands tokens removing space tokens and \relax tokens, until finding an explicit or implicit token of category code 1; at this point it absorbs a balanced text by setting the internal counter to zero; an explicit token of category code 2 found when the internal counter is zero ends the general text.

@egreg Ok. So something like ab{cde}?

@FaheemMitha This is a balanced text

And where are you getting this information from? Is it in the TeXBook? Or in the source code?

@egreg Ok, so a}b{cde}?

@FaheemMitha Of course not.

I don't know why I have this question up, but it has a lot of comments, and no answer.

Judging from the comments (which I didn't really try to follow), an answer is possible.

@UlrikeFischer Never mind. It's not important.

@FaheemMitha Mostly texbook, top of page 276. The implementation using counters is not specified there.

@HaraldHanche-Olsen Oh. The implementation of what? edef?

@FaheemMitha my guess would be "balanced" in a context-free sense.

@PauloCereda I don't know what "in a context-free sense" means.

@FaheemMitha syntactic nesting, like in your example... (aa(aa))(a)(aa(((a)))a...

@FaheemMitha No the implementation of the parsing of a balanced text.

@HaraldHanche-Olsen Oh, as egreg was describing?

@FaheemMitha “context free” is a technical concept in parsing theory. More precisely, one can define a context free grammar, and a context free language (which is a language that can be specified by a context free grammar).

@FaheemMitha Right.

He didn't say where he got it from.

@HaraldHanche-Olsen Ok.

it's difficult to imagine what such a font would look like but I'm curious

although I'm not sure if METAFONT the software is actually able to do it

@texdr.aft Is that the same thing as a scalable font?

Also I've been reading more about it and the METAFONT language is something of a hot mess. It's like algol with TeX-style (i.e. syntax-modifying) macros

@FaheemMitha I suppose. I'm not entirely sure what you're asking

@texdr.aft Is a meta-font the same thing as a scalable font?

@FaheemMitha Oh. Yes and no. It's a "font that describes fonts": a program that creates shapes parametetrically; these parameters can be varied to create new fonts from the meta-font

@texdr.aft Oh.

@texdr.aft I see. So are they meta-fonts?

Weird reason, but ok.

@FaheemMitha That's actually a really good question. I think that "Computer Modern" is the metafont, which is defined by files like romanu.mf, romanl.mf, mathit.mf, etc., and then the fonts are cmr10 etc.

s3-us-west-2.amazonaws.com/visiblelanguage/pdf/16.1/…

Page 11 has a good example of how one can vary parameters

I'm debating whether it's worth buying a paper copy of "TeX by Topic".

@texdr.aft I see.

@FaheemMitha Can you still get paper copies? I have one, but after I got the pdf, I hardly ever consult the paper version anymore.

@HaraldHanche-Olsen There are used and new copies on Amazon (US).

Paper copies are good for browsing, though usually overpriced.

@FaheemMitha True. I guess I am done browsing TbT. Nowadays, I only use it for looking up stuff that I thought I knew but have forgotten (or remembered wrong).

I suppose an ebook reader would be an alternative. But paper books are much harder to destroy. And nobody would steal "TeX by Topic".

@HaraldHanche-Olsen I don't actually know TeX. But you've probably realised that.

There's a used copy on Amazon for around USD 12.50. Which is actually not that much.

Assuming it gets here, of course. But Indian customs, in common with the rest of the world, don't care about TeX books. Or any books at all, really.

I suppose they might steal a cookbook or something.

@HaraldHanche-Olsen Can you comment on how you think "TeX by Topic" compares to other TeX books, and the TeXBook in particular?

@FaheemMitha Well, it's more concise than the texbook, which makes it easier to look things up in. But the texbook may be easier to learn from, as it has more examples. It is essentially written as a tutorial, albeit not an easy one.

@HaraldHanche-Olsen Which one did you use to learn from?

@FaheemMitha The texbook. It was the only book at the time. In fact, I read it long before I even had access to a tex system.

@HaraldHanche-Olsen Oh. That's impressive.

@FaheemMitha It was a long time ago. I am sometimes impressed with my younger self. These days, not so much.

My short attention span makes it almost impossible for me to any textbook through completely.

I even have a hard time with novels, unless they really grip me.

@FaheemMitha Not unlike @PauloCereda, then:

@HaraldHanche-Olsen ooh I had cookies back then

@HaraldHanche-Olsen I don't like ducks as much as he does, but otherwise probably yes.

@FaheemMitha Ducks are delicious. More so than cookies.

@HaraldHanche-Olsen I meant in the non-edible sense.

@FaheemMitha This is the tex chat. Here, we twist each others' meanings beyond recognition.

@HaraldHanche-Olsen oh no

@PauloCereda Am I being mean again?

@HaraldHanche-Olsen Do we have to?

@HaraldHanche-Olsen ^^ ooh? :)

@FaheemMitha No, it's voluntary. Just be aware of the phenomenon.

Ok.

@HaraldHanche-Olsen I take it you do actually use TeX, unlike some people here? Not naming names...

@PauloCereda So you could apply a convex function and use Jensen's inequality to make me less than average.

@FaheemMitha I do. Like most mathematicians, my life would soon be unbearable without it.

@HaraldHanche-Olsen hm I sense some catamorphism

@HaraldHanche-Olsen That's what I thought.

ooh banana algebra

Though I think that typing math papers in LaTeX is TeXnically not very demanding.

@FaheemMitha No, it's quite easy most of the time. Though the pitfalls are numerous, especially with big multiline equations.

@HaraldHanche-Olsen Pitfalls? Layout issues?

Floats are a pain. LaTeX has strange ideas about that.

@FaheemMitha Mostly, yes.

@FaheemMitha It is a very hard problem to deal with algorithmically.

@FaheemMitha A year or two ago, I did the typesetting of a conference proceedings. I had to deal with bad latex written by multiple authors, trying to get a consistent layout from the mess. It did help that they all used the same class file provided by me, but it was still a lot of work. Sometimes TeXnically difficult, too.

@FaheemMitha -- Bug reports in original TeX should be submitted as specified on Knuth's TeX web page, i.e., sent to Karl Berry. (I was the designee through 2014.) Then they are examined by experienced "vetters" approved by Knuth, a determination made whether the report is new (if not, an explanatory statement is sent to the submitter), tests performed and detailed analysis documented. This discussion is set aside to be sent to Knuth when he asks for it. Not posted publicly until Knuth reports.

@barbarabeeton I see. Are the older bug reports available publicly? I don't mean the errata.

@HaraldHanche-Olsen Sounds like hard work. And probably unpaid.

Couldn't you have got the authors to do some of the cleanup?

@FaheemMitha Yes, and yes. (Though I do get paid, just not in proportion to the amount of work I do.)

@FaheemMitha Getting people to do things is hard to begin with. Getting people who have written bad source code to fix it is likely even harder, since if they knew how to write good source code they probably would have...

@AlanMunn Well, one could try to educate them. After all, that all that reviewers do, isn't it?

@FaheemMitha Not realistically. Just explaining to them why their latex is bad, and what they would need to do to fix it, would be more work than doing it myself. And most likely, multiple iterations would be needed, at the end of which the authors would be quite resentful. And me too.

Well, some of what reviewers do, in theory.

@HaraldHanche-Olsen I see.

I take it the LaTeX really was bad?

@FaheemMitha They will review the mathematics, not the latex.

@HaraldHanche-Olsen Fair point. :-)

@FaheemMitha It was all over the map. Some was quite good, some merely acceptable, and some totally rubbish.

They probably consider TeX an annoying irrelevance.

@HaraldHanche-Olsen Ok.

@HaraldHanche-Olsen Negatively correlated with author age?

@FaheemMitha But the mathematics was good, in all cases.

@AlanMunn Somewhat, yes.

I've never had anyone correct my LaTeX ever. Or any code, period. Well, aside from the occasional person on SE. And that's on a pretty small scale.

@FaheemMitha -- Knuth has written up a summary report of changes for TUGboat after every update. These should be pretty easy to pick out in the cumulative TUGboat index by author: tug.org/TUGboat/Contents/listauthor.html

@barbarabeeton There's a cumulative TUGboat index by author?!!

@AlanMunn When we first started getting computers in our department, with tex on them, the one who started using it first and with the most enthusiasm and skill was the oldest of our professors. (But he had actually worked at Bell Labs previously.)

@HaraldHanche-Olsen My experience is that people stick with what they learned, often in grad school, and then propagate it, so bad practices persist over long periods of time.

@FaheemMitha -- Also one by category/keyword, and one by title. Not perfect, but usually workable.

@barbarabeeton I wish I had a guardian angel who would tell me these things.

@AlanMunn That is the more common scenario, yes.

@HaraldHanche-Olsen Of course then there's the people who deal with your LaTeX who are themselves clueless. I recently just got an email requesting my .bib file (which had been included in the zip of my submission). But it was loaded using \jobname and I suspect when they renamed the source, they could no longer find the file.. :)

@FaheemMitha That's not what guardian angels do. ;-)

@HaraldHanche-Olsen Well, they should.

@FaheemMitha -- Someday, when you have nothing better to do, poke around on the tug.org website. There are some really interesting things there.

@FaheemMitha It might distract them from their primary task (protecting you from harm).

@FaheemMitha You need a reference angel.

@barbarabeeton I've done that before. Though I have other things to do. Whether they are better is debatable. Less pleasant, almost certainly.

@AlanMunn :-o

@HaraldHanche-Olsen Well, we're talking about a hypothetical guardian angel. And I believe the best guardian angels are multi-tasking.

There is a glowing and very comprehensive review of "TeX by Topic" in the 1992 TUG.

tug.org/TUGboat/tb13-2/tb35resources.pdf

Starting pg 185.

@FaheemMitha -- Something I remembered about "TeX by Topic" that may or may not be relevant. The original was formatted using Victor's "lollipop" (tug.org/TUGboat/tb13-3/eijkhout.pdf) meta-format. This was converted to LaTeX to prepare the pdf file now available online. The content is the same (there may be a few minor updates), but the appearance is somewhat different. (I think that every glow in the cited review is well deserved.) It's too bad that lollipop never caught on.

@barbarabeeton You have a good memory. Is Lollipop not part of TeX Live then?

@barbarabeeton But in true TeX fashion :) I find it very easy to use, but if something goes wrong the errors can be mystifying in the extreme.

@FaheemMitha No, it still exists. Documentation was last updated in 2016.

I honestly really like the way TeX handles errors. It's one of the few programs that manages to have a "personality"

@AlanMunn A search in Debian doesn't bring up anything.

@texdr.aft I think you're in the minority there.

Across the world, even as we speak, college students sigh and curse over their mysteriously mis-compiling LaTeX documents.

Of course, they are mostly grad students, so are inured to suffering.

@AlanMunn That part is a little redundant.

@FaheemMitha Well, compare it to ed, which prints "?". Meanwhile TeX gives you an interactive prompt where you can actually correct the error before it goes on

LaTeX errors, on the other hand...

@texdr.aft Well, if ed is your gold standard for error reporting, you're setting a very low bar.

@FaheemMitha -- lollipop is in TeX Live, as @AlanMunn has already noted. His comment on error messages is on target too, but one gets used to that after a while. (I read "TeX by Topic" in proof, and, like the reviewer, enjoyed it. I don't remember finding any egregious errors, but Victor has always been quite good about such things.)

And if you like ed, I know some people you should meet...

@FaheemMitha You should tell them about ex

Oh, it's in a package I don't have installed. texlive-formats-extra.

@FaheemMitha -- CTAN says it's in TeX Live. I believe CTAN. Maybe the installation you're querying didn't load the full thing.

@texdr.aft They probably already know. I was referring to the folks in the U&L chat room.

@barbarabeeton I did a poor search, sorry. I just searched in installed packages.

It's in a package I don't have installed.

@FaheemMitha A big part of the suffering these days is an unfortunate side effect of Overleaf, which while having done great things to make TeX more accessible to everyone, tends to hide errors or at least make them less obvious unless you know what you're looking for. And some IDEs do an even worse job. So the problem isn't entirely the fault of TeX.

I remember trying to get LaTeX to work in the mid-1990s. It was terrifying. And nobody helped me.

@AlanMunn I was imagining people using a traditional Emacs setup or similar. But I don't know what people use these days, of course.

The bottom line is, if you aren't used to it, it can look quite daunting. Of course, that is true of a wide variety of things.

@FaheemMitha I would bet that a majority of new users start off with Overleaf.

@AlanMunn I expect you know more about it than I do.

@FaheemMitha Certainly that's the pattern among our students.

I'm looking at an article by Edward Barrett from 2008, about porting TeX Live to OpenBSD. I don't recognize the name. Is he around here?

@JosephWright -- Looking at the meta question Are duplicate questions with no answers deleted?, I conclude that it would probably be a good idea to give a "token" answer even if a (new) question is closed as a duplicate, especially if the question is well phrased and might be easy to find by a newcomer; otherwise this value might be lost. Comments?

@FaheemMitha Edd? He was in UK-TUG for a bit

@JosephWright So not active now?

@FaheemMitha No, I think not: he's a nice chap, though (came to some meetings)

@JosephWright Ok.

@JosephWright ooh

@JosephWright you are a nice chap too :)

@DavidCarlisle Should I tell Frank 'fewer' ;)

@JosephWright and s/d/t/

@DavidCarlisle ?

@DavidCarlisle :)

@egreg Hi, I'm trying this code...

\documentclass[a4paper,12pt]{article}

\usepackage{amssymb}
\usepackage{mathtools}
\usepackage{graphicx}

\begin{document}
$\mathop{\subset}_{\rotatebox[origin=c]{-90}{$\curlywedge$}}^{\curlywedge\curlywedge}$
\end{document}

but I have leaved because I should move the two symbols one to the left and one to the right, rotating them and making them smaller. Surely 100% to this question you will make a very good code

@JosephWright are you online?

@StefanKottwitz I'm about

@JosephWright I sent you a mail.

@StefanKottwitz Sure: I'll look at it (a bit late here for me to give it 100% attention: will be the morning)

@JosephWright 90% is enough ;-)

@StefanKottwitz Reading now