Some people are brave:

@TeXnician very brave!

@TeXnician perhaps finally we will get a proper implementation:-)

@DavidCarlisle pgf broke the interfaces package (unmaintained since 2011 ...) by using \scantokens.

@UlrikeFischer Yay!

@UlrikeFischer the error message had \scantokens written all over it but I didn't bother tracing last night:-)

@JosephWright github/fix-interfaces......

@DavidCarlisle yes same with me.

@DavidCarlisle The main things that broke are imho all the redefinitions of internal pgfkeys commands so some public tikz/pgf developer is needed here ...

@UlrikeFischer or as I think I saw you comment under the question, just declare the interface package as not working, and move on....

@DavidCarlisle much better idea.

Another question answered by not using one of my packages...

Did anyone get a luaotfload update with texlive?

@UlrikeFischer I have an up to date tl, what version do you expect? by the way vvv

 %% FIXME  The date is meaningless, we need to find a way to
    %%        use the git revision instead.
    [2018/10/28 v.2.93  OpenType layout system]

@UlrikeFischer couldn't l3build fill that in now....

@DavidCarlisle I expect Lua module: luaotfload-main 2019-05-18 2.97 in a log. Where is the FIXMe from?

@UlrikeFischer luaotfload.sty

@UlrikeFischer -main.lua has

local ProvidesLuaModule = {
    name          = "luaotfload-main",
    version       = "2.96",       --TAGVERSION
    date          = "2019-02-14", --TAGDATE

@DavidCarlisle oh, looks as if the tagging is missing this. Will check.

@DavidCarlisle I will write to the texlive list. lualibs came on sunday, but luaotfload seems to be lost.

@DavidCarlisle found the error, a period too much ... But I'm a bit disappointed that Petra didn't complain.

@UlrikeFischer I have lots of complaints from Petra in my mail archive that I could forward if you feel you are missing out

@DavidCarlisle second hand complaints ;-(

Ulrike regarding the missing update in tl, what kind of hello doc do we need to trigger that information?

Sorry my bad, too used to look in the console log.

Fully up to date from Danish mirror (dotsrc.org)

@daleif every document compiled with lualatex is fine, as it shows the version of luaotfload-main. But the update is really missing, it should now say 2019-05-18 2.97.

@UlrikeFischer seems so, the Danish mirror is usually quite fast with its updates once they are running again after a new TL release.

I'm assuming that since luaotfload is an internal component in lualatex, that is why it never shows up under \filelist

@daleif lua files aren't shown anyway, but I added everywhere the module messages, so at least the log is rather clear now.

I don't know why, but I wrote a parser that translates \romaji{niku} into にく the last night. Does anybody know whether such thing already exists in LaTeX? (I know there is kanaparser for LuaTeX, but only found out by chance when I entered texdoc kana)

Oh, and \romaji{TABERU} translates to タベル (guess what that means in English)

I'm trying to partially rewrite some TeX code into LuaTeX.

So, I understand that one includes Lua code in TeX using \directlua. And called to e.g. tex.sprint are sent to TeX.

But suppose I want to replace part of a string I'm sending to TeX (using sprint) with a Lua function, say.

How would that work?

(I realise this is vague. I'm just thinking aloud.)

@FaheemMitha I am not sure what you mean, you would normally execute the lua first and then tex.print its result to return it to tex

@DavidCarlisle I didn't/don't have a clear question. Just experimenting. Trying to get a feeling for how it works. What's possible.

@FaheemMitha I recommend studying the chickenize package, despite the silly name and silly default behaviour (changing every word to "chicken") it's clear code and well documented, i learnt a lot from it.

@DavidCarlisle texdoc chickenize?

Or just the source code?

And LuaTeX, I assume.

@FaheemMitha both (but source code mostly)

@DavidCarlisle Ok.

Wow, that's a awful lot of Lua code.

@FaheemMitha but actually doc from texdoc is quite good with tutorials on lua callbacks

@DavidCarlisle Sorry, I didn't follow that.

@FaheemMitha yes but look at the doc it is a lot of different text manipulation functions and each is more or less independently coded, so quite easy to follow

@DavidCarlisle Ok. Doc of chickenize?

@FaheemMitha yes part II starting on page 10 is not doc for the package but general documentation on how to code in luatex

Is it a hard requirement for LuaTeX that all existing TeX and LaTeX packages keep on working without change?

@DavidCarlisle Oh? Ok, looking.

@FaheemMitha it is not even an aim. luatex is explicitly, by design,not compatible with tex.

@DavidCarlisle So packages can break?

Even LaTeX packages?

@FaheemMitha yes and any pdftex font related package is probably sub-optimal in luatex even if it doesn't actually give errors

The version of chickenize I have is v0.2.5. Is this up to date?

@DavidCarlisle Well, that's discouraging.

@FaheemMitha yes

@DavidCarlisle Ok, thank you.

@FaheemMitha pdftex font packages are almost all related to squeezing characters into 256-character fonts in weird tex-specific encodings. luatex (and xetex) use system fonts in Unicode encoding so almost everything that a pdftex font package does is not needed

That tutorial does indeed look interesting. Thanks for the tip.

@FaheemMitha All packages can break, except for @DavidCarlisle's

@PhelypeOleinik well that is true.

@PhelypeOleinik Good to know.

@DavidCarlisle ;-)

@DavidCarlisle I was especially amused by the amount of typewriter questions in the past few days.

@PhelypeOleinik scary that people actually use it, I thought it was a joke. But then I thought afterpage was a joke as well and people use that....

@DavidCarlisle Moral: don't upload jokes to CTAN :-)

nobody told me that the new tugboat is there ...

@DavidCarlisle Since you are apparently around for the moment, what are your thoughts on SILE, if any?

@FaheemMitha I have seen a couple of talks and read a couple of papers but not motivated to try it yet,

@FaheemMitha beware of software with a user base of 1, no matter how good it is....

@DavidCarlisle How do you know it has a user base of 1? Evidence suggests it is higher.

Issues and pull requests, for example.

Actually, one of our high rep users over in U&L is the second largest contributor behind Simon. Namely Caleb.

@FaheemMitha well perhaps. Also it was a while (year?) ago I last looked so it may be a few more now but still same applies, replace 1 by 10 or 100 or whatever guess you think reasonable.

@UlrikeFischer nobody told me, either.

@DavidCarlisle It's certainly not very large.

@FaheemMitha for me the use of a marked up document as communication is as important as the typesetting aspect so a markup language with a small user base is a fundamental problem

That chickenze tutorial recommends:

\directlua{dofile("filename")}

I was doing:

\directlua{luamacro = require "foo.lua"}

Are these equivalent or not? Should one prefer the first for some reason?

@FaheemMitha usual these days is \directlua{require("filename")}

@DavidCarlisle Oh. But these all do the same thing, right?

Sometimes require has brackets, sometimes it doesn't. It's confusing.

@FaheemMitha dofile is lower level than require (require doesn't reload already loaded packages), like \usepackage luamacro= would grab any retuned object (although name a bit odd as it would not be a macro)

@FaheemMitha that is just lua string syntax can be " delimited or [[ delimited, affects the quoting rules

@DavidCarlisle Ok.

@FaheemMitha oh you meant () brackets not [] again just Lua syntax being forgiving for functions that take a single string argument

@FaheemMitha There are places you can shortcut, but the parenthesis here are always allowed

@DavidCarlisle I see.

@JosephWright Ok.

@JosephWright did you see the article from Hans in tugboat?

I was reading Barbara's editorial comments in the new TUGBoat, and came across the Diacritics to die for: Brill (nice name :). Does the link for the slides mentioned there (www.tiro.com/John/Hudson-Brill-DECK.pdf) work for any of you? Since that question was asked I couldn't access that pdf.

@PhelypeOleinik tiro.com/John/Hudson-Brill-DECK.pdf works for me

@DavidCarlisle For me it never loads. ping www.tiro.com doesn't reach :/

@PhelypeOleinik works for me, so i don't care:-)

@DavidCarlisle Where are those “you are mean” messages when I need one?

--- tiro.com ping statistics ---
12 packets transmitted, 12 received, 0% packet loss, time 11012ms
rtt min/avg/max/mdev = 165.872/166.656/167.886/0.766 ms

@DavidCarlisle Close enough ¯\_(ツ)_/¯

@PhelypeOleinik ^^

@DavidCarlisle Not that I'm asking anything, but did you know of send.firefox.com? ;-)

@PhelypeOleinik send.firefox.com/download/071e6bf53cb2aac5/…

@DavidCarlisle Thanks :D

@PauloCereda ^^

@DavidCarlisle ooh

Do all sprint calls within one Lua file share state? In the sense that if I was to define a TeX/LaTeX macro in one call, future calls in that same file would know about it?

I'm assuming it's the same TeX "process", for lack of a different word, being fed tokens in single session.

@UlrikeFischer -- I routinely announce to the TUG board when my copy arrives. Should I do that here too?

@barbarabeeton quack <3

@PhelypeOleinik -- I've just checked, and it loaded immediately. (Sometimes links do disappear between the time an issue is sent to the printer and when it's received in one's mailbox, but this one's okay here. And I did test by copy-and-pasting what you posted, so you did enter it correctly.)

@PauloCereda -- Likewise.

@barbarabeeton Thanks for checking! I think it's some region issue, because since that question was posted I didn't get to open the link (the browser spends some minutes trying to connect and then gives up). David sent it to me though :)

@PauloCereda Does this link work for you? tiro.com/John/Hudson-Brill-DECK.pdf

@FaheemMitha no it doesn't work that way at all, Lua shares state but anything you write back to tex works just as if you \input a file with that string at that point so it is just inserted into the tex document at that point and executed by the main tex procrss and subject to any text grouping at that point.

@barbarabeeton normally we get a mail about it, perhaps it comes tomorrow.

@PhelypeOleinik oh, Katakana.

@Skillmon Katawhat?

@PhelypeOleinik that smiley uses the Katakana character "chi", it's a Japanese script.

@Skillmon Oh, nice. I knew it was something I can't access from my keyboard, but I didn't know from where :)

@PhelypeOleinik no, it does not resolve.

Probably our backbone.

@DavidCarlisle I wonder if we should/can do something about crop.sty. It has no luatex code and this has some curious side effects: it tests for pdftex by testing for \pdfoutput. And so with luatex it either doesn't use the pdftex code, or if microtype is loaded (which gives \pdfoutput a definition) it fails because of the missing \pdfpagewidth:

\documentclass[a5paper]{article}
\show\pdfoutput
\usepackage{microtype}
\show\pdfoutput
\usepackage[center,cam,a4]{crop}
\begin{document}
blub
\end{document}

@DavidCarlisle Well, ok. That's what I meant. But I guess your point was that (for example) if a macro was defined inside a group somewhere in the TeX input, then it would be invisible outside that group?

After messing around with LuaTeX a bit, I realise I don't understand how it works. Take this example.

gist.github.com/fc89c3f55dcfdf9e2dbb0f8d770d5b52

@PhelypeOleinik If you send me a mail, I can send you back the PDF. You should easily find my address looking at my profile page: follow the home page link and look at the bottom.

How does tex.sprint distinguish the variable "row.name" from the characters "row.name"? It's a Lua variable (I guess?) but how does TeX know that?

Does it have access to some context I don't know about?

@FaheemMitha Huh?

@FaheemMitha Lua carries out it's work, and passes the result back to TeX

Consider the line:

 tex.sprint(-2,row.name)

@FaheemMitha So you send back to TeX a string comprising the value of row.name: TeX sees no Lua

@JosephWright Oh, right. Sorry, I was being dumb.

@PauloCereda Hm. Apparently that page doesn't like Brazilians :-)

@PhelypeOleinik who does heuhehuehueheuheue

:)

@egreg David already sent it: chat.stackexchange.com/transcript/message/50392743#50392743. Thanks a lot anyway :)

@PauloCereda You are right mean :-)

@PhelypeOleinik oh no

@PauloCereda you are not mean :)

ooh

@PhelypeOleinik we are good people. :)

@PauloCereda LOL

Readers of TUGboat, now that I've got your attention here. I'd like to do a little survey. Are there topics that you'd like to see more of? Conversely, are there topics that you'd like to see less of? Obviously, I'd like to see more articles that are understandable by non-experts, but it's a hard sell trying to persuade people to write them. And obviously, I'm happier if I find a topic interesting. But ideas are welcome. Thanks.

@barbarabeeton My husband joined TUG mainly because of the DuckBoat column. He claims that he would develop a bipolar disorder without it. He sometimes exaggerates.

@UlrikeFischer is the author still contactable do you know?

@DavidCarlisle I found a github site: github.com/drehscheibe

@UlrikeFischer oh last commit 2017 the date in the tl version is 2003, push a PR for luatex?

@DavidCarlisle it doesn't harm to try.

@PauloCereda you are not mean :)

@PauloCereda now you can cite another person.

@Skillmon awww <3

Thanks, mr. rabbit!

@PauloCereda you're welcome, mr. duck!

@UlrikeFischer -- I think exaggeration is endemic with husbands. I noticed that he was the impetus behind "Die Walküre"; delightful! (If a bit noisy.) Gordon and I attended a week-long seminar on the Ring at the Peabody Institute; were treated to a couple of different films of all four operas, plus background, theory, analysis, etc. (But they missed out on Anna Russell's interpretation.) Since Gordon and I met in an Althochdeutsch class, this was a natural. (I hope we will see you at TUG 2019.)

@barbarabeeton I really like writing macros (more than I like their output on paper :), but I'm rather new to this whole thing, so the ones I like most are the tutorial ones. I have read Stephan v. Bechtolsheim's tutorial articles (namely “\string and \csname”, “A tutorial on \expandafter”, and “A tutorial on \futurelet”) probably far too many times. I like them because they are thorough in one single concept, so they are excellent for learning the very basics of how these primitives work.

@barbarabeeton Another very good one is Joseph's “Exploring \romannumeral and expansion”, which gives an excellent explanation on expansion. As a beginner I found (and often still find) it really difficult to understand the basics of writing TeX code, so these tutorials are a great help.

@PhelypeOleinik /Blush

@PhelypeOleinik -- Those are good examples. Can you suggest other primitives that you would like to see examined? (Then I can go searching for potential authors.)

@JosephWright ;-)

@barbarabeeton Get @DavidCarlisle to write about \outer ;)

So, if I want to pass TeX code interspersed with Lua variables, would I be correct to think I need to use string concatenation?

@barbarabeeton Right now I can't name any, but when I remember one I'll let you know :-)

I guess I could test it...

@FaheemMitha yes there is no structured connection you just generate a string and re-parse from tex, unless you manipulate nodes directly

@DavidCarlisle Ok. I read about nodes. I didn't understand it, though. Seemed TeX-specific.

Like TeX structures exposed at the Lua level.

In that case, I'm at a disadvantage, not really knowing TeX.

@DavidCarlisle So functionally tex.sprint behaves similarly to the Lua print command? That seems to be the case.

@FaheemMitha well yes except it prints to the tex input buffer not the terminal

@JosephWright -- and why \outer has been "disallowed" in latex. (My instincts were developed with plain tex, and, honestly, I miss \outer!)

@barbarabeeton because it is the most useless annoying feature in tex. If I had my way we'd \let\outer\undefined

@DavidCarlisle Right

@DavidCarlisle ^^

@DavidCarlisle and @CarLaTeX -- Here's a news flash for you (for different reasons).

@DavidCarlisle -- Well, it has saved my hide more than once. We will just have to agree to differ.

@barbarabeeton Indeed it may be a sensible reason to get extinct!

@santimirandarp it's not that the margins are large, it's that A4 paper is wide

@barbarabeeton you are allowed to be wrong sometimes:-)

@santimirandarp -- As @DavidCarlisle says, the paper usually used in a laser printer is wide. You probably would rather carry around a book that isn't meant for presentation on a coffee table. If you set the margins to be narrower, the text would get cut off when a normal sized book is printed.

@barbarabeeton

davidc@dc-bantham /usr/local/texlive/2019/texmf-dist/tex
$ grep -r 'csname new' . 2>&1 | wc -l
418

@barbarabeeton the only effect of making \newcount and friends \outer is that over 400 instances have to obfuscate their code as \csname newcount\endcsname just to avoid the error, and \outer does nothing other than make errors that people have to try to avoid

@DavidCarlisle -- you apparently never had to develop code in plain tex for highly nested stuff that had to be counted, and accidentally put \newcount within a group, hence running out of hashsize.

@barbarabeeton well naturally I never have any errors in my code, but making \outer is completely the wrong way to catch bad use within a group, making it impossible to define anything that needs to allocate a new counter like newtheorem or newfloat or newanything that should be used at the top level, but as part of a higher level structure declaration that just needs to allocate a counter as part of its internals

@santimirandarp which code?

Random thing I was wondering about. Did Knuth actually write TeX all by himself? Did he maybe have teams of collaborators working on it, including grad students? Since he was at Stanford. I know he worked on it for a while. 10 years?

@santimirandarp yes but A4 is too wide for single column text to be readable, the widths latex uses are about the maximum that good practice says should be used for a singl eline of text

I don't really know the history. But getting a working system like TeX involves an awe-inspiring level of difficulty. Seems like a lot for one person.

@FaheemMitha he writes at length about that, students helped notably Liang's phd thesis on the hyphenation algorithm

@santimirandarp well of course you can do lots of things, you can specify 1mm margins all round and have long lines, or you can use two column or you can use larger fonts or ... see:

@santimirandarp also it depends on the content, if you are writing a novel with lots of dense paragraphs you really do want to aim for a "paperback book" size and not have over long lines filling A4 paper, but if your content has lots of display math or indented lists or other things that break up the lines anyway, you can get away with making the text block wider. there are no actual rules, it's art not science:-)

@FaheemMitha -- As @DavidCarlisle says, DEK has written a lot about this. His first cut was to leave two grad students to write a prototype from his outline while he was on sabbatical. And other grad students contributed both ideas and code, but in the end he (re)wrote the entire final code himself; that's what's published in TeX: The Program.

@barbarabeeton Any particular essay or piece of writing you'd point to?

And did he not work with other researchers on this?

I mean non-grad student researchers.

@FaheemMitha -- His book Digital Typography contains just about everything he has written on the subject. (Much of that is republished from TUGboat articles, but I don't have the energy to look that up right now. But you can make guesses from the TUGboat contents list by author.) A general overview is presented in this article: "TeX: A branch of desktop publishing evolution, Parts 1 and 2". He did work with other people, but mostly about fonts.

@barbarabeeton If he worked that all out himself, it seems quite hairy.

A group approach would be... easier.

@barbarabeeton Fancy article. And relatively recent.

@FaheemMitha -- Which is one reason why TeX hasn't been superseded by something else. But he had good reasons for the solo approach -- it didn't turn out to be either a zebra or a camel. (The classic examples of a horse designed by a committee.)

@barbarabeeton Some languages survive being designed by committee. E.g. Common Lisp.

@FaheemMitha -- An uncommon example.

That looks like a really detailed article.

Page numbers? Where are the page numbers?

@FaheemMitha lisp hardly conquered the world.... compared to say python

@DavidCarlisle It isn't a competition. And CL is doing ok. Just making a point.

And yes, a single person design does bring more coherence to the table. Not disputing that.

@FaheemMitha -- What you've got there are the "accepted" drafts, not the final version. The page numbers are applied when the articles are put into an issue.

@barbarabeeton Oh.

@FaheemMitha -- Another article, from a different perspective, is "Communication of mathematics with TeX". Not as much about the development, but the evolution and the current environment and use of the tool,

@barbarabeeton Thank you.

to get a left aligned block on the right use \hspace*{\fill}\begin{tabular}{@{}l@{}}one\\two\\three\end{tabular}

How does the NewDocumentCommand option argument show up in Lua? It looks like a boolean.

I'm not even sure how it is represented in TeX.

@FaheemMitha sorry the question doesn't make sense:-)

@FaheemMitha you just get a list of characters and pass in lua source code, there is no automatic mapping of internal structures.

@DavidCarlisle What character is that optional argument then?

@FaheemMitha sorry I can not even guess what you mean

Or what characters are the true/false values?

@DavidCarlisle I think he means \BooleanTrue

@JosephWright Probably.

@FaheemMitha whatever code you are putting in \directlua{...} you can put in \edef\tmp{...}\show\tmp and what gets shown needs to be valid Lua code, that is all there is to it.

@FaheemMitha Currently they are \char0 and \char1

@JosephWright Oh. Is that 0 and 1, then?

That would also have been my initial guess.

@FaheemMitha it is to tex but to Lua it's probably a syntax error, depending what you do with it, as I say you need to construct valid Lua expression

@FaheemMitha They are not just 0 and 1, they are \chardef'd, so not expandable: you really want to pass using something like \IfBooleanTF #1 { <true marker> } { <false marker> }

@FaheemMitha why are you exposing internal implementation details of tex commands to Lua?

@FaheemMitha I have to wonder, like @DavidCarlisle, what your aim is here: xparse is for document-to-code linking, doesn't really belong at the Lua level at all

Oh, never mind. Perhaps it's not necessary to bring Lua into this after all.

@FaheemMitha You haven't shown what you are doing but it sounds like you are doing something very strange:-)

@FaheemMitha I have no idea what you are trying to do, but here is some code to query a boolean from lua: tex.stackexchange.com/a/438782/2388

@DavidCarlisle Not strange at all. Very ordinary, and very boring.

@FaheemMitha No, if you are exposing internal implementation of xparse to Lua then you are doing something very unexpected.

@DavidCarlisle Well, I don't need to. So I'm not.

@FaheemMitha Ok but you have been on the site long enough to know that asking for help by talking in riddles gets frustrating after a while, I tried to give you some pointers to luatex use but it's impossible to understand the last set of questions so i give up.

@DavidCarlisle Sorry. I didn't realise I was talking in riddles. It's not always easy to know when one is being obscure.

@FaheemMitha it's easy to know that you have asked dozens of questions without posting a single line of code

@DavidCarlisle Sometimes I post code. :-)

@FaheemMitha no, not in this round of luatex questions.

I could post my current code, but it's not pretty to look at. Probably migrane-inducing.

But I'm making progress.

@FaheemMitha you know the drill, no one is interested in the full original document but you can always make a small test file for any specific issue.

@DavidCarlisle Yes, I know. But mostly I've been managing to isolate the issues myself.

If I get stuck, I'll post a MWE.

@FaheemMitha well the last set of questions is completely impossible to answer without one.

@DavidCarlisle You mean the boolean thing?

@FaheemMitha scroll up and count the number of luatex questions you have asked and how many examples you posted:-)

Possibly 5 or 6, depending on how one counts.

I actually made good progress today. But I'm tired.

@santimirandarp You want a blank image in the place of those images?

I'm not sure what a "square box" is.

@santimirandarp What is an image-space? You mean just a border with nothing inside?

If you are trying to generate something like an image in place, TikZ seems to be the popular option.

Please upvote this post which was ignorantly downvoted.