TeX, LaTeX and Friends

Nasser

02:43

omg; I've been struggling for hrs trying to figure why some of my files do not build after I changed to latexmk using lualatex. It turned out it was using pdflatex and not lualatex, even though I did use -lualatex. I found out I had to also add -pdflatex=lualatex option otherwise it was using pdflatex. So I changed all my Makefiles now and all problems went away.

But latexmk is really nice. It figures automatically all the dependencies, so I do not have to keep updating manually my Makefile tell it which file each file depends on. I am switching all my makefiles to use it, also using it to update when I need to build HTML.

Nasser

04:15

tex4ht is so slooooow. I wish I know why. Same file 10,000 pages only compiles to PDF in about 10-20 minutes (did not measure to be sure, but it was not too long). tex4ht has been running for more than 2 hrs and it still did not finish the first pass. Normally it takes 2-3 passes to finish. I think it is dvilualatex which is so slow. OK, I am running on VBox but it has lots of RAM and on very fast PC. It is only tex4ht which is slow.

I'll post a small movie to show how slow it scrolls. it it like 5 second per one line.

The above is real time. You do not think it is slow??

With lualatex, it scrolls so fast on screen, I am not able to read the lines. With tex4ht it is like walking on egg shells.

zetyty

04:50

@JosephWright @DavidCarlisle @TeXnician Thanks for giving your point of view. Some arguments in favor of Codidact:

(i) it is a not-profit organization (so we can hope that profit will not be the main reason that will guide the evolution of the site).

(ii) the site is very similar in appearance to SE (so one can imagine doing something equivalent there.)

(iii) the devs are really attentive to the community and the features-request (so we can even imagine doing better than SE)
For example, the Q&A format is not very conducive to discussion especially on Meta, we can imagine features allowing to modify the format only for Meta (just to say that everything is open to discussion).

(iv) the code is open source (so if the devs don't have the time to develop a feature, you can do it yourself if you have the skills).

(v) sites can have different features so we can imagine features specific to LaTeX (and friends).

(vi) The decision board will be open to community members (if not already).
(I'll come back if I have more to share)

(vii) an aggressive policy towards AI generated posts.

zetyty

05:33

@FaheemMitha Yes you're right, the chat is on discord at the moment... It is not optimal but it's a chat I suppose.

Faheem Mitha

07:19

@Nasser You can add a flag on top of your LaTeX files to force it to compile with LuaLaTeX.

\RequirePackage{iftex}
\RequireLuaTeX

@Nasser Yes, latexmk does a good job of handling dependencies. Which is nice, because it's a really hard thing to handle manually.

@Nasser Try profiling tex4ht? I don't know how actively maintained it is. The original maintainer died a long time ago.

@zetyty The SE Chat is quite good. Would that be hard to emulate?

@Nasser You need to use latexmk -pdflua foo.tex to use LuaLaTeX. Possibly latexmk -pdflua -shell-escape foo.tex if you need -shell-escape. -pdflatex=lualatex is incorrect syntax, and will probably just cause you problems later on.

Ulrike Fischer

07:41

@Nasser without knowing which options you use with tex4ht and an example document one can't tell you what is slow (but my guess would be that you activated something that uses lots of regex to handle the math).

Nasser

@FaheemMitha **pdflatex=lualatex is incorrect syntax**  But according to  tex.stackexchange.com/questions/28730/… it says to do that. And once I did, then all the problems I was having are gone and it is working OK so far. But I will look into the option you gave. This is now what I do

latexmk -pdf -pdflatex=lualatex -silent -shell-escape -jobname=LUA -lualatex="lualatex --shell-escape --file-line-error index.tex" index.tex

And it is working so far.

@UlrikeFischer The command I use is

make4ht --shell-escape -ulm default -a warning -c nma_mathjax.cfg -e $TEXMFHOME/tex/latex/tex4ht_build_files/MAIN.mk4 $filename.tex "mathjax,htm,fn-in,,notoc*,p-width,charset=utf-8,cut-fullname" "-cunihtf -utf8"

Where there is .cfg and standard make4ht build file from Michal. I think what you are saying about lots of regex seems true, I remmeber now Michal saying something like this long time ago.

But tex4ht is really slow. Painfully so. it takes 3-4 hrs to build one HTML documents when lualatex takes 20-30 minutes for same latex.

Nasser

08:14

I actually thought when using make4ht in mathjax option it will be faster. Because all the math is being send/shipped to mathjax to handle in the browser. So logically it seems this should make it faster than say generating images for math as I was doing long time ago. I do not know anything about dvilualatex, but it is the one always running when I run tex4ht and always taking lots of CPU time.

Nasser

08:28

here is small movie showing CPU used while using texth4t and nothing else running on system (perl must be from running latexmk as I do not think tex4ht or make4ht uses perl, and dvilualatex from running tex4ht)

David Carlisle

why am I opening more bugs at latex2e than fixing them?

2

zetyty

08:45

@TeXnician Since Codidact is non-profit and user-oriented, it is a good candidate for a federation if this is a comunity request. Strong links (even synchronization between user accounts) between different sites and Codidact could be guessed as long as the sites agree (this will not be the case for SE unless perhaps Codidact grows a lot...) . If the devs don't have time to develop these links, everyone can do it because it's open source.

(It is just my opinion regarding what I understand now and it is not a guarantee of course...)

@FaheemMitha I don't knonw really but I think it's not big deal to implement a chat but they have not a lot of devs currently and since they already have a backup solution with Discord, maybe it's not a priority now...

Ulrike Fischer

@DavidCarlisle how did you find it?

David Carlisle

@UlrikeFischer the math alphabet one I haven't really found yet (not looked) the luadef one I ran diff on \tracingall :-)

Ulrike Fischer

@DavidCarlisle I meant where this came up, but I now just saw the tex.sx question.

David Carlisle

@UlrikeFischer yes it would be better if @zetyty would lure users away to competing sites so they don't disturb us with latex bug reports disguised as innocent questions

Faheem Mitha

@Nasser Read the documentation, not random questions and answers on the internet. texdoc latexmk.

The -pdflatex option is, as the name suggests, for the user to choose their PDFLaTeX executable. Which is pdflatex by default, but of course may be called something else on the user's system.

zetyty

09:01

@DavidCarlisle Ok I'll stop now. Sorry to bother all of you. I had no bad intentions, but I screwed up.

Faheem Mitha

@zetyty I'm not familiar with Discord. Does it have a good chat implementation/equivalent?

@zetyty I think David is just joking. It's his British sense of humour.

You noticed I typed "humour", not "humor".

David Carlisle

@zetyty as @FaheemMitha said

Faheem Mitha

@DavidCarlisle Maybe a wicked fairy who was also a Word user put a curse on all TeX users, that they would have to constantly have to file bug reports as the price of admission.

@Nasser As you would expect, there is a similar -lualatex option.

@zetyty Since it seems you are involved in Codidact, do you know why it does not have a TeX site, though Topanswers does? Just historical accident, perhaps?

zetyty

@DavidCarlisle Ok, I'm not very comfortable with this discussion so I haven't been able to determine where exactly the level of irony in your comment was... So I continue but if I don't understand on my own when it must stop, I assume that I will be told!

You don't know me and I would understand that the interventions are annoying. I allowed myself to talk about this new community project because I assume that you are all people used to "community projects" and that you do not see the harm in talking about it even if it conccurence SE. If I'm wrong please tell me!

@FaheemMitha About discord yes it is a very good chat really (but it seems that it doesn't support markdown...)

David Carlisle

@zetyty as I said, personally I don't think multiple sites are a good thing, but certainly it is not unreasonable to disagree with that or to say that one alternative is so much better it could replace this one

Faheem Mitha

09:14

@zetyty At the risk of stating the obvious, there is nothing wrong with talking about other sites here. There is no kind of rule, written or unwritten against this. And nobody cares one way or the other.

2

I suppose if someone was to keep going on and on about it, people might, hypothetically, get a bit annoyed. But that's hypothetical.

@zetyty BTW, please use the reply feature. It's better for context.

zetyty

09:31

@FaheemMitha I don't know, maybe the TeX Topanswer community just start first. Maybe better to ask @samcarter about that.

Ulrike Fischer

@zetyty I have no problem with non-profit and open source. I'm in that business too. If people think that is worth their time and work for free on something it is fine. But I have also no problems that e.g. overleaf sells premium accounts and tries to make a profit from it or that my baker sells bread or that the people working for stackexchange want money to pay their rent. I do not expect that such services run only through self-exploitation and wouldn't switch because of such an argument.

zetyty

@UlrikeFischer I totally agree with that, for me non-profit doesn't mean "don't make money by charging for a product or service"

If it could help the project...

samcarter

@zetyty \begin{history} When we were looking for an open-source alternative to start a TeX Q&A, codidact was still in a planing phase and at that time, the idea was to build a new Q&A site from scratch. This would have taken quite some time. TopAnswers on the other hand was already up and running and we could just jump in.

Since then, codidact changed course a bit and instead of building a new system from scratch, it ended up building onto an existing system which speed things up quite a bit \end{history}

Joseph Wright

09:50

@zetyty I guess my concern is your reasons are all 'philosophical': what drives most users is features/info, and I'm not seeing what we'd gain in practical terms from another site

@zetyty I really am not being critical, I'm asking as things will only work if there is a driving force (e.g. TopAnswers is much less busy than TeX-sx)

samcarter

@zetyty Just my personal opinion: I find TopAnswers betters suited for TeX questions. The well integrated chat is definitely a big plus and I like the approach to not prominently show some reputation counter.

Joseph Wright

@samcarter I suspect the issue with rep. would be that on the SE network it's used as a way to give privileges that scales reasonably well - not perfect, but overall not terrible. TopAnswers still seems to be pretty small, so that 'does it scale' issue is less of an issue

@samcarter Chat I agree works well for the type of question that comes up for TeX: again, I can see why it wouldn't work for example on the main site (SO)

samcarter

@JosephWright On SO such an approach would be a disaster. They would need whole legions of mods to contain the chaos :)

Joseph Wright

@samcarter Quite, and I think that's where some of the problem with other approaches comes - 'does it scale' is really an important question

samcarter

@JosephWright Codidact has an interesting approach to award privileges: The different privileges are awarded separately based on actions which show that the users can handle them. Making suggested edits will lead to the edit privilege, suggesting x tag descriptions will allow to edit tag descriptions etc.

David Carlisle

10:00

@samcarter for me the benefit of these sites (comp.text.tex then tex.se, ...) is seeing how people use stuff in practice, and (as in my "issue" comment above) provide an effective way for bug reports to get back from end users to the developers (even if the end user doesn't see it as a bug) so technical features of the site matter little and governance matters not at all, what matters is having a critical mass of developers and end users, and the more sites you have the harder that is to achieve

2

zetyty

@JosephWright Ok I understand your concern. I haven't been clear with my personal motivations. Indeed, the philosophical aspect is important to me but it's not really the fact that it's non-profit but the fact that the community has a say in the site itself and not just the management of their community.

It's really the fact that the features-request are really listening and discussing that I like a lot and seems to me to have enormous potential in order to create sites better adapted to different communities and try to modify what does not work optimally on SE.

2

Joseph Wright

10:37

@zetyty All reasonable, but as @DavidCarlisle says, in the end it's critical mass that's important. So what specific features are you imagining we'd gain in a move, that would encourage a shift of that critical mass?

@zetyty I'm asking because I've no issue with a change, but I do want to be sure it would work

Ulrike Fischer

@zetyty what concretely do you want to change on SE? And did you try to change it, e.g. by making a feature request and where is it?

daleif

11:12

what exactly is \mathcolor? Sopurce3 mentions it as LaTeX2es \mathcolor but Source2e that comes with my TL23 does not seem to define it. (just wondering if one should use \mathcolor{color} over \color{color} in math mode?)

Phelype Oleinik

@daleif You should. \mathcolor doesn't spoil spacing as \textcolor does in math

@daleif github.com/latex3/latex2e/blob/develop/required/graphics/…

Ulrike Fischer

@daleif texdoc mathcolor

zetyty

11:44

@UlrikeFischer I do not want to put SE on trial, I just want to know if people would be interested in a new adventure (which could not succeed without people like you) and I was asked to give some arguments in favor of Codidact, which I did.

David Carlisle

@zetyty it helps if you keep the threading (that reply is to a separate thread about mathcolor)

Faheem Mitha

@samcarter Codidact built on an existing system? What system was that?

daleif

@PhelypeOleinik I hardly ever use \textcolor only {\color{color} ... }

zetyty

@DavidCarlisle Sorry, I always forgot...

Joseph Wright

@daleif Same issue

David Carlisle

11:56

@daleif mathcolor is better. if you use {\color{color} ... } you get a mathord and the color restore comes out where it comes

daleif

@DavidCarlisle Here I only use color temporarily (is cleaning a manuscript (when I need a break), and I color the when better constructions have been used)

samcarter

@FaheemMitha github.com/codidact/qpixel

Faheem Mitha

@samcarter Is it pre-existing, or built for the site? It sounds like the latter.

samcarter

@FaheemMitha Art of Code, the codidact project lead, happened to have a basic Q&A site already lying around. This was used as a starting point for what you see now.

Faheem Mitha

@samcarter I see. Thank you for the information. I would never have guessed.

TeXnician

12:12

@zetyty Do you have any experience on implementing federation? Doing that post-hoc is hard. And codidact has features that are hard to federate (moderation, voting, …). The devs do see it as a long-term perspective but it's hard because of the massive experience required both in codidact data structures and in federation principles. This is not your usual “open-source and it will be contributed” task (and as an open-source developer I do know that these won't get tackled by random devs).

@zetyty Discord is bad because it's a silo on its own. Corporate control and all that. If you want to make a point for enabling the community then corporate-controlled chat is a no-go, especially with Discord. If anything, this shows that it still has some way to go to get ready for prime-time ;)

Nasser

16:26

There are probably Linux experts here. Any one knows why when I was trying to direct the output of make4ht command to /dev/null, it is stuck, and I can't even ^C output of it? The whole process hanged. I typed

make4ht -ulm default -a debug a.tex "htm,mathjax" 1> /dev/null

What is wrong with the above? Here is screen shot. I was trying to see if I can send stdout to /dev/null so that I only see errors on screen.

I am just following standard ways to do this, for example:

I wonder if Latex internally changes things to cause this?

David Carlisle

16:47

@Nasser (why 1> not > although it's same thing) are you sure you never get any interactive error prompts? they are hard to handle if you redirect stdout

you have debug logging on?

Paulo Cereda

@DavidCarlisle oh no, stream redirection :)

@TeXnician ^^

Nasser

@DavidCarlisle I do not have debug logging on. Can you please try it on your linux? a.tex is simple file, here it is

\documentclass[12pt]{book}
\usepackage{graphicx}
\begin{document}
My document
\includegraphics[width=\textwidth]{ZZZ}%will cause error
\end{document}

And please type the command

make4ht -ulm default -a debug a.tex "htm,mathjax" 1> /dev/null

does it hang for you?

David Carlisle

@Nasser " I do not have debug logging on. " er... -a debug

Nasser

Oh, I did not know this is what you meant, I thought you meant linux wide thing. Yes, I have -a debug but this is the whole point. I want to only see the errors, not stdout.

If I use -a warning then make4ht do not stop on errors. So `-a warning is not good for me.

David Carlisle

17:07

make4ht -ulm default -a debug a.tex "htm,mathjax" > a.log2
then you can see a.log2 getting longer, it takes a while though as you have full debug info logging

Nasser

@DavidCarlisle but this does not really do what I want. I type

make4ht -ulm default -a debug a.tex "htm,mathjax" > a.log2

and see nothing on the screen as errors also go to a.log2. So I have no idea if it is running or if stopped. Ofcourse I can do `tail -f a.log2` to see, but we are back to square one. I want to see limited output, same as `-a warning` shows, but also _stop_ when an error happens. https://tex.stackexchange.com/questions/689582/using-a-warning-in-make4ht-does-not-stop-at-first-error it seems to be hard to do. So I either live with zillions of output on screen but stop…

David Carlisle

@Nasser ???? it redirects exactly the same as your version but just to a place you can check

Ulrike Fischer

@DavidCarlisle see this question for the background: tex.stackexchange.com/q/689582/2388

Nasser

@DavidCarlisle I am not explaining things well it seems. Yes > a.log sends all output to file (stdout and stderr). But I do not see the error now unless I am looking at a.log all the time. But this is the same as just not using a.log in first place and looking at the terminal.

David Carlisle

@Nasser but if you redirect the same things to /dev/null then you don't see any of it

Nasser

17:20

@DavidCarlisle I wanted to direct ONLY stdout to /dev/nul, that is why I used 1>

I was hoping to use `-a debug` so that tex4ht stops on error, but not be bothered by all the stdout, and only see errors. I am assuming an error message such as

! LaTeX Error: File `ZZZ' not found.

will go to stderr and not stdout.

David Carlisle

@Nasser no

@Nasser 1> is same as >

Nasser

@DavidCarlisle are you saying Latex error messages such as the above also go to stdout? In this case, then redirection will not help. So I am stuck

David Carlisle

@Nasser yes, you can test your self: use > a1.log 2> a2.log and see which goes to each stream

Nasser

Ok, so only solution then is to use -a debug and live with zillion of screen output which makes it very hard to see where the build is. But at least now it will stop at error. It does not seem to be an alternative. lots of output also could slow the build a little.

David Carlisle

@Nasser I would guess it's notably slower using debug

Nasser

17:30

@DavidCarlisle you are right. That is also why I wanted to use -a warning, but this does not stop at error, it keeps going. So if I have a build that takes 10 hrs, I will not notice the error unless I am sitting looking at screen 100% of the time to notice. This is not practical solution. But with -a debug the make stops and so I can see right away the error on the screen anytime since it stopped scrolling.

zetyty

18:29

Well I think that's enough for today concerning Codidact. I just want to thank everyone who shared their views with me. It was instructive for me. Thank you for being patient and constructive!

Faheem Mitha

18:45

@Nasser tex.stackexchange.com/a/689628/3406

I don't know much about latexmk of course, so please do point out errors and deficiencies.

It's been a while since I wrote an answer here. Actually, I think it's been a while since I wrote an answer anywhere on SE.

Nasser

20:46

@FaheemMitha THanks. Will try it. It will good if you also show an example using your new method how to pass arguments directly to lualatex. This is what I do now:

latexmk -pdf -pdflatex=lualatex -silent -shell-escape -jobname=LUA -lualatex="lualatex --shell-escape --file-line-error report.tex"report.tex

So using your latexmk -pdflua how will the above change? in order to pass any arguments directly to lualatex? Having such an example in your answer will be useful.

I learn better by seeing examples, rather than reading 200 pages manuals :)

Faheem Mitha

@Nasser I think that some arguments can be passed directly to latexmk. I know -shell-escape can. Check what the documentation says.

@Nasser The manual isn't long.

Nasser

Yes, but most users just want to see a usage example(s). But will try it later, can't now.

Faheem Mitha

latexmk -pdflua -silent -shell-escape -jobname=LUA --file-line-error report.tex

might work. Your command is definitely unnecessarily long, with lots of duplication.

Are you sure you want -jobname=LUA?

Usually one wants to keep the original file name.

Oh, I see there is a man page as well. man latexmk. Not sure if it is identical to texdoc latexmk.

LaTeXereXeTaL

@barbarabeeton Yes as @TeXnician said that was me. According to the feedback, the workshop went quite well and there are already requests for more. As I described before, I deliberately took a VERY low level and slow introductory pace and that was precisely what the participants said they appreciated the most.

@barbarabeeton Your excellent suggestions will definitely make it into the next workshop in the series (yes, a series has been requested!). I was somewhat surprised that academics were so "afraid" (for lack of a better term) of LaTeX. Some had not used it since grad school, others were afraid of the learning curve.

@DavidCarlisle @UlrikeFischer Given the publication of TLC3, is there any reason to consider an update to Lamport's original LaTeX book to include new kernel features, etc.?

David Carlisle

21:01

@LaTeXereXeTaL don't think so, but you'd have to ask the author. the latex book isn't really affected much I suspect: it never documented package level code

Joseph Wright

@LaTeXereXeTaL What @DavidCarlisle said

LaTeXereXeTaL

@DavidCarlisle @JosephWright Excellent. Thank you!

Faheem Mitha

21:18

@LaTeXereXeTaL What was this workshop about?

Nasser

@FaheemMitha I got chance to try it and it works. So adding --file-line-error latexmk passes this to the compiler. No need to use -lualatex="lualatex --shell-escape --file-line-error a.tex" as I had before. So your method is simpler. Thanks will change my makefiles to use it instead of what I had before. Both work the same way, but simpler is always better.

Faheem Mitha

@Nasser Glad I could help.

Alan Munn

@LaTeXereXeTaL I think one fear is just the time-sink potential. People develop ways of working over many years, and even if they're inefficient, they're efficient for them. I see this all the time with LaTeX users as well, who aren't afraid of LaTeX, but are very resistant to change.

Faheem Mitha

@AlanMunn Well, to be fair, TeX does not, in my experience, actually make one more productive. Sometimes it feels like quite the reverse. What it does is enable you to do many things you could not do otherwise. And do better typesetting, of course.

I think TeX methods might be more productive if you are trying to do heavy automation. But I don't know how often it is used for that purpose.

Alan Munn

@FaheemMitha At least in my field, LaTeX trivially makes you more productive. Of course it has enormous procrastination potential, but that's a separate issue.

Faheem Mitha

21:33

@AlanMunn Does it make one more productive because of automation, or because it lets you do things you could not otherwise do?

Alan Munn

@FaheemMitha Neither, it let you do things you have to do orders of magnitude more easily than using Word.

Faheem Mitha

@AlanMunn Doesn't that count as automation? Perhaps not.

Alan Munn

@FaheemMitha By automation I thought you meant repeating things. But citations and cross-references alone are better handled in LaTeX than in Word. If that's automation, then yes, but I didn't think that's what you meant.

Faheem Mitha

@AlanMunn I don't have a clear picture of what automation means, but yes, I think it probably means doing basically the same task with varying inputs.

Generating a railway timetable for example. Or generating forms.

Or exams.

Alan Munn

@FaheemMitha Same for lining up word-for-word glosses of example sentences. This is 'automatic' with a package, but I wouldn't really call it automation.

Faheem Mitha

21:38

@AlanMunn OK.

Alan Munn

@FaheemMitha Yes, these were the kinds of things I thought you meant, so no, none of the things I mentioned are of that type.

Faheem Mitha

@AlanMunn OK. Thank you for the clarification.

LaTeXereXeTaL

21:59

@FaheemMitha It was part of a workshop series in the OPTYCs initiative, which is an NSF funded grant program to establish a national institution for the teaching of physics in two-year colleges. I am one of the coordinators for continuing professional development workshops. optycs.aapt.org

@AlanMunn I agree.

@FaheemMitha One participant left feedback saying my approach was "charming and informative." I'll take it.

Faheem Mitha

@LaTeXereXeTaL A workshop series about learning basic LaTeX, or something more specialised?

barbara beeton

23:04

@LaTeXereXeTaL -- I've finished my slides for the conference, but they're not posted yet. If you send me email, I'll be happy to tell you where to nab them.

LaTeXereXeTaL

23:40

@FaheemMitha The goal is to show two-year college physics faculty the good ways LaTeX can be of use in a physics classroom environment. For example, for years my students used LaTeX (both local installations and then Overleaf) for doing homework assignments, including computation assignments. But first, many expressed a desire for a VERY basic, introductory workshop or two before getting into classroom usage.

@barbarabeeton Email sent as requested.

LaTeXereXeTaL

23:58

@zetyty IMHO the single worst thing about SE is that every single written suggestion, comment, question, answer, and yes, even chat comment, is subject to upvote/downvote/favoriting/gaining rep/losing rep. Sometimes, things should be able to be expressed without worrying about literally being judged for agreeing/disagreeing/suggesting something new or different, etc. It's quite offputting. I WANT to be far more active here, but this continual rating of my thoughts keeps me from it.

Transcript for

$TeX - LaTeX$ TeX, LaTeX and Friends