TeX, LaTeX and Friends

David Purton

1:00 AM

@DavidCarlisle, good effort from Boland, don't you think? 6/7!

David Carlisle

@DavidPurton I can't guess what you are referring to, sorry.

user202729

4:47 AM

We have tags for plain TeX and context but not LaTeX... because LaTeX is the default?

user202729

6:43 AM

Reading around a bit it looks like that utf8x supports many characters with diacritics by auto-generate them from Unicode definition. tex.stackexchange.com/questions/304639/… For example this character

Can utf8 package do the same thing? (even now that character is not supported in inputenc.)

Paulo Cereda

7:37 AM

@yo' Yay! <3

user202729

Also, what kinds of characters exactly are "supported" by LaTeX? i.e. I know that the LaTeX team don't want to add math characters (I don't know why however)

But what about other kinds of characters? Or only characters with diacritics are supported?

There are some mention of "breaking text mode", but then it could do some check to see if it's in math mode or not and output accordingly

Besides, it's only a problem for characters that might appear in text mode (α), but not e.g. ∈, so what's the issue?

Paulo Cereda

8:22 AM

@Skillmon ooh rabbits attacking twitter.com/rabbitdoingwork/status/1475678027996614656

Joseph Wright

8:37 AM

@user202729 Yes, broadly: as for a lot of people TeX ~ LaTeX, most questions would be tagged the same

@user202729 Font coverage

@user202729 Out-of-the-box the aim is to support the chars covered by T1

user202729

What do you mean font coverage? Characters like ∈ can be defined in terms of math font, right?

I mean for math characters like ∈ that can't be used as text why don't inputenc define it as math?

user202729

9:06 AM

Can someone change the interface3 documentation to state that \tl_replace_all only work on top level...?

Joseph Wright

9:22 AM

@user202729 You mean,m I guess, that there's no attempt to enter into TeX groups?

@user202729 Yes-but-not: inputenc is about text: using the math-mode ∈ for that character given in running text would be questionable-at-best

user202729

@JosephWright Yes that

Joseph Wright

@user202729 I think it's implicit in the fact we are dealing with token lists not detokenized strings, but I can add something

@user202729 ∈ is not in T1 (there are only 256 slots, after all)

user202729

Token list is not item list? Besides compared to regex_replace_all it does recurse in

Joseph Wright

@user202729 Yes, it does, but it's not a tl function :)

@user202729 Well, even 'strings' are token lists, but they don't have catcode 1/2 tokens ...

user202729

No I mean

I don't know. It doesn't seem that clear. Documentation says "tl can be view as list of tokens or list of items"

It doesn't say list of items is the "default" unless mentioned otherwise?

... well maybe it should be understood like that

Only concern the replace, remove and if_in functions

Joseph Wright

9:28 AM

@user202729 At, that's for something like a mapping, where we can grab balanced text (brace groups) or we can look at individual tokens

user202729

For comparison document of \tl_reverse does explicitly mention that it does not recurse in groups

Joseph Wright

@user202729 I've added something to the start of the section on replacement

user202729

Cool (may take long time for next distribution version or something)

Maybe add to \tl_if_in:Nn documentation too

Joseph Wright

@user202729 Also done

Skillmon

10:45 AM

@PauloCereda We killer rabbits exist... :)

Joseph Wright

11:37 AM

@user202729 I see your point: I'll think about some more text (@egreg, @DavidCarlisle?)

David Carlisle

@user202729 ?

user202729

@DavidCarlisle Okay never mind

So inputenc is only intended to support characters in T1 right? That's very restrictive I think

David Carlisle

@user202729 I have an answer on site somewhere that reads the unicode-math definitions backwars so gives inputenc definitions for all commands that unicode-math assigns to single unicode code points, I'll see if I can find...

user202729

@DavidCarlisle Don't need, I already have

What I ask(ed) is why the team decide to not include these by default

3 hours ago, by Joseph Wright

@user202729 Out-of-the-box the aim is to support the chars covered by T1

With this it (reasonably) makes sense

David Carlisle

@user202729 yes I didn't understand the question originally, wasclearer when I read whole thread. Actually I think we could provide a package that does something like

9

A: Unicode maths in pdflatex

I would not do this. The source is far clearer if you use the command forms (even if using unicode-math with LuaTeX). However, you can input unicode-math's unicode-math-table.tex, inverting its normal definition so that it defines the character to expand to the command rather than the other way a...

user202729

11:45 AM

Someone can, but it's only "canonical" when provided by "the LaTeX team"

~~make them responsible for maintaining the code~~

David Carlisle

@user202729 yes except we also support Greek (LGR) and Cyrillic (T2) if thiose fonts are used

user202729

Okay I can find a character table, let me see

David Carlisle

@user202729 sure but that's just a matter of smoke and mirrors. If a package is supported as part of the oberdiek ho-tex collection it's me and Ulrike, if it's etoolbox or siunitx it's Joseph, if its fontspec or unicode-math it's Will. In all these cases the support is by core team members and what badge it is distributed under is mostly history rather than any logical reason

Skillmon

Strange team fragmentation is strange :)

user202729

12:05 PM

For the combining character part I mean...

For example the character ǫ is not a single character in any font (as far as I can see? There are too many old TeX fonts encodings)

But inputenc supports it anyway

So where is the line between "can be added to inputenc if someone pull request" and "will not be added to inputenc"?

David Carlisle

@user202729 combining characters are pretty much unsupportable in pdftex but doesn't matter if it is a single glyph in the font inputenc would map ǫ to \k{o} easily enough

user202729

So "things that can be easily supported by the default text font encoding(s) can be included in inputenc"?

David Carlisle

@user202729 it's not a line it's a fuzzy grey time dependant area weighing up the pros and cons in each case. (Adding stuff costs time for everyone unless we also preload into the format)

user202729

Hm, it's true that loading takes time too

David Carlisle

@user202729 see the hit unicode-math takes when it loads unicode-math-table (we really should fix that and preload those definitions into the unicode engine formats but...)

user202729

12:14 PM

Although I don't think that's the issue

Just tested, utf8x is 0.01s slower than utf8

"completely negligible"

Maybe another 0.01s with mathletters

Ulrich Diez

@user202729 A difference between TeX and programming languages like C++ or Java is that TeX-code is carried out during compilation. TeX-code is not that much about running a program after compilation. That's why the concept of tokenization and expansion plays such a big role in TeX. With C++/Java etc tokenization and carrying out things like branching during compilation is not of much interest to the programmer, only when it comes to parsing compiler-directives and the like. The only reason

@user202729 why things look complex to you is that expandability is required. I.e., that it is required that everything is done in TeX's gullet. Most things in TeX can be done without that requirement. If you don't insist in expandability, i.e., in processing in TeX's gullet only, you can easily compare token-lists by defining scratch-macros and \ifx-comparing those.

user202729

Uh okay, I know about TeX parsing layers and stuff.

I mean

Okay never mind

Skillmon

@samcarter my wife says your your tikzlings look too perfect :)

user202729

12:31 PM

Okay I go debug it a bit

Loading whole Unicode-math takes around 0.8-0.9s

Where loading latinmodern-math takes 0.54s

More than half already

For LuaTeX

David Carlisle

@Skillmon presumably "perfect" meaning "looks better than a fur-less pink rabbit thing" ?

Skillmon

@DavidCarlisle she means that there are too many "perfect geometrical shapes" involved and stuff.

user202729

Guess precompiled header is mostly useless, then

Ulrich Diez

@user202729 Of course, when defining scratch-macros for comparing, you need to obey that token-lists might contain single hashes # and that therefore you need to do something like \edef\itchy{\unexpanded{<token list 1>}}\edef\scratchy{\unexpanded{<token list 2>}}\ifx\itchy\scratchy..\else..\fi rather than \def\itchy{<token list 1}\def\scratchy{<token list 2}\ifx...- the latter might lead to ! Illegal parameter number...-errors .

With TeX there is always a subtlety lurking around that might bite you if not aware of it. ;-)

David Carlisle

12:46 PM

@user202729 what were you timing? On this setup I get a more or less 2 second delay here (1.9x on most runs)

\documentclass{article}

\usepackage{unicode-math}

\directlua{t1=socket.gettime()}
\begin{document}
\directlua{print('\string\nTime: ' .. socket.gettime() - t1 .. "\string\n")}

\end{document}

user202729

Maybe faster computer, then

Whole compilation is roughly 1.3s with Unicode-math

David Carlisle

@user202729 I have some overhead from cygwin, file access especially

Skillmon

1:20 PM

@DavidCarlisle and you're benchmarking on Windows. As @PauloCereda once said: Benchmarks are like air conditioners, they cease to work as soon as you open Windows.

5

samcarter

@Skillmon The original idea was to build them just from simple geometrical shapes -- then I discovered Bézier curves and everything went south from there :)

@Skillmon At some point, one should make a calendar with these quotes :)

Skillmon

@samcarter she thinks they are cute, but we're designing a flyer right now and for that purpose she thinks they aren't fitting.

@samcarter I'm afraid we don't find enough, else I would've said "next year?"

user202729

Is the time to append two token list proportional to the total length?

(I think so, but who knows if TeX has some internal optimization for special cases)

samcarter

@Skillmon easy solution: change the topic of the flyer to something for which they fit :)

Skillmon

1:45 PM

@user202729 here is some proof:

user202729

Okay benchmarked some myself, looks like yes

Sadly

Maybe TeX can optimize some special cases

Macro in TeX is stored as singly linked list with left at start, right?

Can't remember, but "from the impression of reading LuaTeX code"

In that case left-append can be optimized. Whether or not it improves performance much is another issue

Skillmon

@user202729 but there is no "append" (neither left nor right) at the macro level, there is just \def, \edef, \let... And concatenation in expl3 is done as \edef\foo{\unexpanded\expandafter{\foo}\unexpanded\expandafter{\bar}}. TeX expands both lists and stores their contents into a macro, both contents are reread this way.

user202729

I know

Still, the engine can special case "common pattern" and speed them up

Maybe add to my TODO list. Had way too many things already

Marcel Krüger

@user202729 LuaTeX does this for token list registers when you prepend with \tokspre (and to a smaller degree for \toksapp).

Paulo Cereda

@CarLaTeX @UlrikeFischer @samcarter 79 views!

user202729

1:58 PM

Okay (why didn't \tl_left_put use that)

Marcel Krüger

@user202729 expl3 needs to work for all engines and it's tl variables are not token list registers.

samcarter

@PauloCereda yeah! That's fantastic!

user202729

... right

Then maybe the engine could make something similar for tl variables too

For "work for all engines" it remains possible to detect if the feature is available

Although the change could be done without any modification in expl3 code as well, "just" make special cases in the engine code

"Easier said than done"

samcarter

@Skillmon ... if there aren't enough to fill an Earth year, we could move to another planet instead

CarLaTeX

2:23 PM

@PauloCereda I sent it to a friend of mine and she was enthusiastic about it!

Paulo Cereda

@CarLaTeX ooh more people

@samcarter did you enjoy my "hat"? :)

@samcarter ^^

CarLaTeX

@PauloCereda Look at the statistics from Italy!

Paulo Cereda

@CarLaTeX ooh

@CarLaTeX I actually can't, I can only see what Vimeo allows me to see on their free tier. :)

CarLaTeX

@PauloCereda :(

Ulrike Fischer

2:49 PM

@Skillmon we found this while cleaning up for the move:

2

samcarter

@PauloCereda Nice box to sit in!

Paulo Cereda

@samcarter :)

Ulrike Fischer

@user202729 no. If you load e.g. LGR or T2A encoding, inputenc will load the corresponding utf8 definitions. Only the default is restricted to T1+TS1.

samcarter

@CarLaTeX The more interesting question is why is Stuttgart the city with most views? Is <insert name of one of the big car manufacturers> using the video as teaching material for their marketing team? :)

David Carlisle

@UlrikeFischer @user202729 more generally it will load definitions for any font encoding of a suitable zzenc.dfu file is in the path

Ulrike Fischer

3:02 PM

In Germany they have started redecorating the roundabouts ^^^^

CarLaTeX

@samcarter Maybe prof. van Duck has some friends there

Paulo Cereda

3:20 PM

@samcarter at least it's not Bielefeld

David Carlisle

@PauloCereda is that a guillotine on your profile?

Paulo Cereda

@DavidCarlisle oh no

user202729

3:51 PM

Have someone made a token list pretty printer? \tl_analysis_show is unacceptable

David Carlisle

@user202729 what do you mean by pretty here better indenting? (tl_analysis_show is distinguishing catcodes so can't easily show abc as abc so prettiness wasn't really an aim

user202729

I'm thinking of some output to HTML for syntax highlighting and "reveal more on tool tip" or similar

Not necessarily HTML, but GUI helps

samcarter

4:06 PM

@CarLaTeX Probably! After all the ancestral home of the van Duck dynasty is not far away:

Entenstein Castle

Entenstein Castle (German: Schloss Entenstein) is a medieval castle surrounded by a moat situated in the center of the town of Schliengen. Schliengen is located in the district of Lörrach, Baden-Württemberg, in the very south-west of Germany in the proximity of the Black Forrest.The origins of the castle can not be clearly dated. The first use of a building at this location can be traced to Walter of Schliengen in 821. By 1000 a tower house might have been in use. In the 13th century, Rudolf von Üsenberg (1207-1231) was the owner of the castle. The name of the castle derives from the German...

CarLaTeX

@samcarter Ooohhh, we must visit it!

samcarter

4:20 PM

@CarLaTeX yes! Prof. van Duck visiting his castle :)

Paulo Cereda

@samcarter is the plural of Einstein in German Zweistein? :)

ooh German is so easy

David Carlisle

@PauloCereda yes by the theory of relativity

Paulo Cereda

@DavidCarlisle ooh

Skillmon

5:05 PM

@UlrikeFischer love it! To be honest, I only marginally heard about your move. Where did you move to?

Ulrike Fischer

@Skillmon We are now living in Bonn (or more precisely Beuel). Do you want the poster? I could send it to you.

samcarter

@PauloCereda sure, why else would there be a television series called Siebenstein? de.wikipedia.org/wiki/Siebenstein (sorry, it seems there is no English version of the page)

Skillmon

5:26 PM

@UlrikeFischer somehow I doubt my wife would like it :P

barbara beeton

6:12 PM

@UlrikeFischer -- One of my favorite members of our stuffed menagerie is a bunny of what looks like about this size. I will not show her this picture! She's too likely to get ideas.

Paulo Cereda

7:17 PM

@samcarter ooh

Transcript for

$TeX - LaTeX$ TeX, LaTeX and Friends