LaTeX2e <2021-11-15> patch level 1

@JosephWright oh I was expecting siunitx

@JosephWright do we get also an accompanying dev release?

Reading unravel source code...

(from the look of it) It really simulates TeX?

... wouldn't it be better to just modify TeX source code then...

@PhelypeOleinik I'll try to wrap my head around that.

@PhelypeOleinik For the record, some or all of that explanation should be in the answer itself. Typically answers on TeX SE contain too much of "here is how you do it", and not enough of "here is what this means", and "here is how this works".

Magical recipes are fine in Harry Potter, but aren't that useful in real life.

@PauloCereda :)

@UlrikeFischer At some stage I guess: I was not sure it was super-urgent

@user202729 Yes, that's the plan

@user202729 Needs a different skillset, means getting engines built (and named), ...

@FaheemMitha The usual tension that in the end 'read the manual' covers much of 'how does it work' (certainly 'read the sources' does, as we can go all the way to tex.web)

@JosephWright I don't think we need to go that far. :-)

@FaheemMitha :)

But many answers contain no explanation at all. And much of the details of how TeX works aren't exactly intuitive for the non-expert.

I think there is room to strike a happy balance.

@FaheemMitha Oh, definitely

@FaheemMitha I tend toward a 'Here is what to do ---- Here is how it works' approach

For example, @PhelypeOleinik's answer I just referenced isn't exactly staggeringly obvious, even if you have some idea what catcodes are, and what they do.

@FaheemMitha Ah, but the problem then becomes how much to detail

@JosephWright Indeed, hence "happy balance".

@UlrikeFischer Let me check my l3build change works

Phelype's answer contains some explanation, so I guess that was his happy balance.

But I would have been happier with more.

The problem here is...

It's really very hard to understand how these "fiddle with this internal command and this other internal command produces this result" works

The only way is to read their source code.

If you already understand catcode, that is.

For example, he wrote a comment,which says in part:

> The catcode manipulation is because the varbatim-isation of filecontents makes the backslash be an ordinary character; thus you need something else.

Which seems like an important detail to me.

@user202729 I can believe that.

OTOH, perhaps his comment is obvious.

@user202729 But I'm making a general point. @PhelypeOleinik's answer has some explanation. It's quite common for answers to have no commentary whatsoever, not even to indicate general strategy.

... yes, I complained about the same thing not-very-recently.

I suppose one can take the viewpoint of - people need to figure it all out themselves. And I suppose that's a legitimate viewpoint in some sense.

I'm newer than you are, so I even have difficulty figuring out where the documentation is. (in the end I write a new answer for it.)

@user202729 I recommend "TeX by Topic", which is free and part of TeX Live.

It's not super comprehensive, but very helpful, and organized in a sensible way.

Yes, already figured out these and others.

texdoc texbytopic

There's also that free book floating about on the net, which is occasionally helpful.

latexref.xyz

This one does have errors, though. But they fix the errors.

This manual has made a start at collecting the huge amount of LaTeX information available in one place. And which the printed books typically don't have space to include.

@user202729 Figured out what?

(That the books you mentioned exists.)

@JosephWright True that.

Named?

Although yes, on the other hand if you just have a patch to the source code it will over time get merge conflicts and stuff

@user202729 but then you'd have to distribute the modified binaries, and have modified pdftex and luatex and xetex and .. and so it would be a different thing altogether.

@user202729 As in 'different to pdftex, etc.'

@HenriMenke github.com/pgf-tikz/pgf/pull/1083#discussion_r763853381 Daring ...

@HenriMenke I'm going to work on the test suite/l3build today, I think I see what I need to do

Just post the patch and let interested readers figure out how to compile TeX themselves

Basically the "I want to share this with the world but too lazy to support it" kind

@user202729 Er, that would be a very small set of people (DEK, @DavidCarlisle and I when we have to, @PhelypeOleinik, the small set of people who provide TL binaries, Christian Schenk, that's probably it)

But then people have more motivation to figure out how to build from source

The set of people who build gcc from source is not just the gcc developers either

@user202729 Oh, the KerTeX guy

@user202729 Perhaps, but why would anyone want to?

@user202729 Somehow I doubt it ... it's not ultra-difficult, but it's not exactly a routine task either

@user202729 I guess I'd point out that nowadays even the LaTeX team don't bother building their own LaTeX formats, we just leave it to TeX Live :)

@user202729 the people who want to trace tex macro expansion and who need to debug compiling tex source are almost completely disjoint.

I didn't ever mentioned debug compiling TeX source though?

@user202729 you suggested using a modified tex source above didn't you?

@user202729 compiling tex on a new machine isn't exactly unlikely to fail without some lowish level debugging of library dependencies

... sounds bad

Then hopefully someone will make it better, with cmake or conan or something

Not that I know how to use these things

@user202729 There's a make recipe ...

But?

@user202729 and you have cmake set up on windows, and cygwin and android and rasberry pi and all the other texlive platforms?

@user202729 Fine if you are using Linux, less fine for macOS or Windows (I do any engine-compilation tests on a Linux VM as a result)

@user202729 not to mention the 7 million tex users using overleaf and no access to a commandline at all

@DavidCarlisle :)

@user202729 I guess really what I'm trying to get at is that for almost anyone debugging some TeX thing, asking to use a self-compiled engine is not going to help: they are much better off with \tracingall or similar. Using unravel is a way to lower the barrier from within a setup they already have

@JosephWright Your efforts regarding l3draw are outstanding and the backends are well-tested, so it would be stupid to not make use of them. I expect that this will also solve all the problems we currently have with XeTeX.

expl3 also provides useful abstractions for common algorithms like walking lists and expanding arguments.

But I can also say for certain that pgfmath will never be replaced by l3fp.

@HenriMenke Sure: you can't do widthof or similar by expansion

\fpeval is coming ...

By the way I made something to let 2^\sqrt 2 works without the extra braces. (why isn't it the default?)

@user202729 What do you mean 'work'? The input is ambiguous at best, and there's the whole 'active ^ with an argument' business

> What do you mean "work"?

In my case... at least that particular example outputs what you would expect (same as 2^{\sqrt 2} before the change)

@user202729 the markup is simply horrible though it is really unfortunate that 2^\mathrm x works without braces

@user202729 depending how you did it you probably broke tex4ht and most other tex-to-anything translators though.

Yes, I understand that currently they are not intentionally made to be error or not error

@user202729 The other way round would be better: make 2^2 not work so users get into the habit to always use {} and thus avoid problems in the future :)

But that's really annoying to read

@samcarter That's quite easy to do

Hm? I thought that one is impossible

Unless you make something active

But the {}s are annoying to read

@user202729 don't use tex syntax if you don't like {}

@user202729 you get used to them very quickly. In the long run, the make it actually easier to read.

After using it for a while I conclude that TeX has almost the best most flexible syntax, but not completely flexible

Compared to Python "have to put quotes almost everywhere" if you want to use it for the same purpose

@user202729 Exactly, make ^ active and do a look-ahead

@user202729 but actually I think x^2 is fine. It is unfortunate that the argument of ^ is not parsed the same way as the argument to a macro, so a single token or brace group. the fact that ^ expands as needed at the primitive level to find a brace group means that it has a syntax completely different from all other top level latex constriucts

Let's see...

The reason why there's the extra complexity in TeX the engine in the first place is to make it a little easier to read

(compare \^{2}{3})

Actually since ^ doesn't really take the left one as input (does it?)

Still, there's \over in plain TeX

Niceness and consistency should be balanced, I guess. Too much consistency leads to lisp's (+ 2 3)

@user202729 \over is absolutely the worst thing in the tex design and responsible for much of the complication with math fonts and \mathchoice weirdness.

And too much niceness leads to C++? Nearly impossible to parse, too many special cases (T::template f<int>(1), std::vector<std::vector<int>>)

Then in that respect, somehow it seems that TeX is neither nice nor consistent (let's say it's historical artifact then. You can't just throw it away and redesign the core)

@user202729 sure (well you can, start again see Don H's finl for example finl.xyz but for most people most of the time working within tex constraints meets most requirements)

Yes, you pointed that one to me once before

Last time I remember it being incomplete or idea-only

Still, with roughly one developer and about one year that's... okay I guess

@user202729 idea only (mostly) as far as I know.

So in other words "not usable at the moment"; still, idea is good (only) if I'm willing to tackle something that large

Back to the ^\sqrt thing, so the consensus is mostly "allow nothing but ^<digit>"?

Why not be more flexible? User can choose to use all-{} or not

Or an opt-in library (... it's just a library, why don't I make one myself?)

Okay looks like parsing \tracingall output programmatically is hard

After \escapechar is changed it also changes in output?

By the way do anyone know anything that compiles Python to expl3, or something similar?

@user202729 say what?!

@user202729 Why would one do that?

@user202729 Yes, \tracingall respects \escapechar

@JosephWright Because...

It took me way too long to get used to writing expl3 syntax

Look, someone did write PyLaTeX

Was trying to look for ways to parse tracingall output but looks a little... hard

@user202729 Just read it? ;)

Programmically.

@user202729 Yes, but that's at the product-a-document level, whereas expl3 is TeX programming so has rather different requirements

@user202729 See my previous answer ;) (As in, if I'm stuck I just work through on paper imagining I'm TeX)

Still, I guess the point is that someone find TeX too hard to read

@user202729 some things in life come at a cost. :)

Both writing something-else and put up with the current situation is hard.

@user202729 People don't read \tracingall because it's fun, but because there's a (usually well hidden) bug. Making a buggy parser for that would hardly improve things

@PhelypeOleinik You sure that @DavidCarlisle doesn't consider it a fun pastime while waiting for the duck to get crispy?

@samcarter oh no

@samcarter Well, maybe @DavidCarlisle does it for fun, because he doesn't know how to make bugs :)

@PhelypeOleinik :) must be such a boring life without bugs

@samcarter Indeed :)

@samcarter /entomologists in the audience just fainted

My point is...

@PauloCereda <3

I'm thinking a non-buggy parser is possible.

But because it's full of special cases it turns out to be hard...

There obviously isn't any way to tell TeX to digest the next token (\digestafter like \expandafter) right? (actually there might be with LuaTeX. Should look at it later...)

If it were that easy unravel wouldn't need to do such hard things

/Gödel intensifies

@MarcelKrüger ooh github.com/rockerBOO/awesome-neovim

@DavidCarlisle -- I queried DEK specifically about this. His response was essentially that his goal was that TeX input should sound natural to a mathematician when spoken. \frac doesn't, even though it's easier to parse. This is a key place where one recognizes his origin as a mathematician rather than as a computer scientist.

@user202729 This will break when the command uses runtoks internally, but otherwise you can use

\directlua{
  tex.runtoks(function()
    token.scan_token()
    tex.quittoks()
  end)
  print'This comes between the two \noexpand\\show commands'
}\show\abc\show\ABC
\bye

Actually it might not even break then. Probably does weird things when nested though.

@PauloCereda Don't tempt me. I don't have time to obsess about my neovim setup for the rest of the day... :)

@barbarabeeton yes I have seen his comments on over before. Even great men can be wrong sometimes:-)

@MarcelKrüger <3

@user202729 no don't special case digit. If I had a choice I would do (what tex4ht does) and make ^ active and defined to take a normal #1 argument (so a single token or brace group)

@user202729 flexibility is good in places but having to run the full tex execution mechanism in order to be able to parse the argument to a superscript command means there is no possibility for any sort of grammar or any reliable tex-to-anything translation that doesn't use tex

@PauloCereda <3

@user202729 is it reasonable to expect editors to correctly syntax highlight this for example

\def\foo{\bgroup  abc}

$x^\foo xyz} + 5$

\bye

@DavidCarlisle \@backslashchar broke xcolor?

@UlrikeFischer Oh no!

@UlrikeFischer the question that you just commented on?

@DavidCarlisle if you mean the bug report in xcolor.

@UlrikeFischer oh no haven't seen that (I'm at Mum's and only online occasionally)

@UlrikeFischer ah yes sorry I did see it your "comment" was in the the issue email, OK, that's due to the backslashchar change?

@DavidCarlisle I think yes, that's the place where the tracing starts to differ.

@UlrikeFischer I guess I should look...

@DavidCarlisle this here has changed too. Now you get relax:

(The original one in plain TeX was \over. Its \frac was different.)

So it's fair to say that plain TeX (tries to) maximize readability while keeps implementation/reading cost reasonable...

(over time it becomes unreasonable.)

I'm talking about the specific case 2^\sqrt 2 here

Yes, of course, as long as it's possible to redefine catcode in TeX it's impossible to syntax highlight correctly in general (that one doesn't even redefine catcode)

Thinking about it, how easy/hard would it be to correctly parse 2^\sqrt 2 without a TeX engine? Actually pretty hard

(compare 2^\alpha)

@user202729 Sure you could throw it away and resign the core. It would just be a tremendous amount of work.

And most people would probably do something different, anyway. They aren't necessarily enamored of TeX's design.

@user202729 BTW, you don't have all your sites linked in your profile. It just show SO.

show -> shows.

@user202729 it's the same as my \foo example above, if it works at all then you need to know \sqrt inserts a bgroup at the start and need to know what tokens it absorbs and where it inserts an egroup to end the superscript.

@UlrikeFischer that would be wrong but it's mum's birthday and we are having duck a cup of tea so color will have to wait

@DavidCarlisle phew tea

@PauloCereda it is possible I made an error in the markdown and struck out the wrong phrase

@DavidCarlisle oh no

@DavidCarlisle I think the problem is a spurious space ;-) \csname\string\color @#1\endcsname eats the space after "color", but \csname\@backslashchar color @#1\endcsname doesn't.

@samcarter twitter.com/shouldhaveaduck/status/1468259308550852610

@PauloCereda ohh

@UlrikeFischer grr I thought I'd caught those: better make an update I guess but I'll be driving home shortly so will be later tonight unless you beat me to it:-)

@DavidCarlisle xcolor seems ok, only color is wrong. I can make a pull request and you can then check, I will be away in the evening too.

@UlrikeFischer thanks

@DavidCarlisle thinking about the commit message ...

@UlrikeFischer Get @egreg to do it ;)

@UlrikeFischer I'll tell my mum that I'm being bullied

@DavidCarlisle but not on her birthday!

var=$(echo $(wget whatthecommit.com -q -O -) | tr -d '\n' | sed -e 's/.*<p>\(.*\)<\/p>.*/\1/' | awk '{split($0,a,"</p>"); print a[1]}')
git commit -e -m "${var}"

@UlrikeFischer ^^ :)

@PauloCereda Is that cmd or powershell commandline syntax?

@PauloCereda "This should work until december 2013."

Hm....

@PauloCereda -- oh, my. But some teas are absolutely lovely! And coffee doesn't suit Chinese or Japanese food at all! You need to broaden your horizons.

@PhelypeOleinik oopsie :)

@DavidCarlisle Does it speak bash? :)

@PauloCereda blame @UlrikeFischer if it doesn't

@barbarabeeton @JosephWright was a true gentleman for not beating me up during TUG 2018. I refused tea during breakfast. :)

@DavidCarlisle ooh

@PauloCereda whatthecommit.com/b9cd524375178de2b53f8cc44caca3ed

@samcarter oh no

@PauloCereda It's much easier to do git commit -em "$(curl --silent --fail http://whatthecommit.com/index.txt)"

@PauloCereda <3

@samcarter <3

@DavidCarlisle ^^ see @MarcelKrüger is more efficient :)

pnas.org/content/118/50/e2107827118

@samcarter ooh

@PauloCereda but still bashful

@samcarter the funny thing is that I have co-authored a paper about pedestrian dynamics :)

@DavidCarlisle ooh

Asking a font question for a friend. Are there any script fonts that support the extension of the swash on the capital R? The purpose is to create a logo.

@PauloCereda ohh! You should write one about the pedestrian dynamics of people running towards cute ducks :)

@samcarter ooh

@samcarter -- This calls into question application of the adjectival meaning of "pedestrian".

@barbarabeeton :P

@barbarabeeton google suggests a few vv

@barbarabeeton Zapfino or Zapf Chancery?

@barbarabeeton the latter also available as TeX Gyre Chorus =)

@JasperHabicht and @DavidCarlisle -- These fonts do already have obvious swashes, and I believe I've seen multiple lengths of swashes on the Zapfino R. but I can't find a feature that allows extension of the swash. (In other respects, Zapfino is quite attractive.)

@barbarabeeton so you are looking for fonts that have an opentype swsh feature which affects the R?

@JasperHabicht -- Yes, exactly. Actually, I love the swash on the Zapfino Z (and have a "personal" example of his "HZ" signature, which is just breathtaking). A swash of that sort on the R would work quite well, but getting a font where adjustment of the swash is "automatic" is not easily found, at least not by me.

@barbarabeeton Except it's not an adjective here. :)

@AlanMunn -- True. I'm just being difficult.

@barbarabeeton Join the club.

@AlanMunn -- Oh, I did, long ago. My older sister, were she still alive, would be delighted to confirm that.

@barbarabeeton it would indeed be great if one was able to search fonts by their opentype features!

@barbarabeeton Agmena Book Italic sold by Linotype seems to have this feature. you can check on their website: linotype.com/de/1110408/agmena-book-italic-product.html

@barbarabeeton EB Garamond also supports this, but it's only italic and not really script. same goes for Agmena Book, though. it seems that script fonts often already have such an swashy R ...

@JasperHabicht -- The examples I'm finding are only "fixed" swashes. Very attractive, many of them, but nothing about extensibility. Thanks for the pointers though.

@barbarabeeton I think, you will only find two or a few alternative glyphs in one font. I don't know about such a feature being implemented as variable font ... but who knows! event this one only has to glyphs: v-fonts.com/fonts/zinzin-vf

@JasperHabicht -- I know that the shape could be created with Metafont or Metapost; it's very like the arabic kashida.

@barbarabeeton that's true.

@JasperHabicht -- Oh, I do love that sassy Q! But the R is relatively pedestrian.

@barbarabeeton sorry for my multiple posts. i am only getting used to this chat here ... I also found this design.tutsplus.com/tutorials/… (especially the first font that they use as example) but it is still only a lot of alternative glyphs, not totally variable

@JasperHabicht -- No problem. My laptop is misbehaving, and sends off bits before I intend that it should. Nothing too embarrassing yet, but ...

@MarcelKrüger I've been thinking about the tdsdirs stuff: I plan to make an xcopy() function to copy entire directory trees - OK?

@JosephWright Sounds good.

@JasperHabicht -- Now that is definitely worth passing on, and I shall do so. It doesn't provide an exact answer to my question, but it has other useful information. Thanks!

@barbarabeeton you're welcome!

@JasperHabicht added as feature request for albatross. :)

@barbarabeeton I know I'm late to the party, but for a logo I would choose a font which overall matches the look and then use a vector editing tool (e.g. inkscape) to adjust the extend of the swash manually.

@samcarter -- The latest communication I've had is that the individual involved will talk to a retired (reformed?) calligrapher he knows to ask advice. He also suggested that maybe he'll investigate FontForge. He's even more of a fussbudget than I am, so I'm sure that what finally emerges will be attractive.

@barbarabeeton I just thought that if it is only for a logo, that is, you only need to edit or animate (?) one character, it may be easiest to grab a nice R, get the outlines and then use some vector graphics software to re-shape them? It is even possible to animate vector graphics (SVG) in web browsers with tools such as snapsvg.io

@barbarabeeton you could get someone with natural artistic flair like what I have to design a custom R

@DavidCarlisle Looks like me in the early morning.

Is the layout of the profile changed?

@JasperHabicht -- That's a possibility. I'll pass it along. This person is much more adept at using such tools than I am. My experience has been with function tables, log-log graph paper, french curves, india ink and a ruling pen. Thanks.

@CarLaTeX apparently yes

@DavidCarlisle More "verbose"

@CarLaTeX I feel it looks a bit strange, because the "Summary" part in the Summary is still quite densely stuffed, but the rest has now a very loose layout ... well, we will get used to it anyways, as we got (probably) used to all the other things ...

@JasperHabicht Yes, of course :) but the previous was more clean

@CarLaTeX lots of space allocated to list all my questions

@DavidCarlisle :D

@DavidCarlisle The answers are now more detailed it seems

@CarLaTeX Now the endless search for that one link to that one function starts again ...

@JasperHabicht Indeed! :D

@CarLaTeX that's part of the ongoing champaign to make things "responsive"

@samcarter no comment

@CarLaTeX I think they should sent every user one or more new mice. With so much additional space it now takes much more scrolling to see the same amount of information - this will dramatically increase the wear and tear of the scroll wheel :)

@samcarter Also the click button is involved, to remove a bookmark now you have to click twice

@CarLaTeX ohh, maybe a secret marketing plot of the computer mouse industry?

@samcarter Maybe :D