How much faster is luacode's math versus pgfmath? I've never learned lua, but I'm considering it to prevent my computer from taking ages to make things.

Maybe the question shows inexperience, as I'd assume a general programming language would be much faster than tex.

I'm just wondering how much this would improve my life. By a factor of 3? 10? 20?

is there any reason that a change in spacing at the end of a line where that line is followed by a blank line would cause an (additional) overfull hbox?

@Jasper tex is much faster at arithmetic than pgfmath. this is why forest has the 'argument processor' feature. because you can use the argument processor to avoid pgfmath. that's partly a question of just expanding macros, but it is also doing whatever dimension and integer calculations, comparisons etc. you possibly can with tex.

@DavidCarlisle In my opinion it would simply be better to have the serifs at the serif slots and sans at sans slots, and then do a setup when using the font what default one wants. As it is in this font is just confusing.

@Jasper tug.org/TUGboat/tb40-1/tb124wright-fperf.pdf

@Jasper Depending on operation, up to three or four times but if you want to use the FPU, could be sixty times

I also get differences where space changed at the ends of lines within tabular cells, but I have no clue why.

!        %
  ! Missing character: There is no ^^98 ("98) in font ybdr2j8c.
  <recently read> \TS1\textpertenthousand
  l. ...}

but this looks harmless, at least.

but e.g.

*** 14169,14175 ****
  ============================================================
  TEST 5: test-T1-clm-m-sc
  ============================================================
! Overfull \hbox (4.60548pt too wide) in paragraph at lines 39--39
  []\T1/clm/m/sc/10 (clmcsc8t10) ?`But aren't Kafka's Schlo^^ff and ^^c6sop's ^^d7u-vres of-ten na^^efve vis-^^e0-vis the d^^e6-monic ph^^f7nix's
  \hbox(7.65999+2.105)x455.24408, glue set - 1.0
  .\hbox(0.0+0.0)x0.0

seems odd.

@JosephWright So doing math in tex using the FPU is significantly faster than using luacode? If so, I will need to do this for some pictures I want to make before I'm 80. Sorry for the delay, I only saw the notification now.

According to the link you posted, if you know you are using luatex, then luacose is significantly faster. (I know you wrote it)

I need to calculate somewhere between a thousand to ten thousand cross-products for an image.

@Jasper Er, no - Lua is always faster

@Jasper That sounds like you should be using an external tool for data processing - TeX is good for making nice looking plots, but real data crunching needs a specialist tool

What if I did the math in Python, and wrote and ran the tex file from the Python file?

Thank you for your time by the way

@Jasper Yes, for example, that would work

@Jasper Personally I would save some digested form of the data, but that' is how I work - I'd keep the plotting in TeX

@Jasper tex is slow. but pgfmath is much, much slower.

I would digest the data, but I want to make animations, so it needs to be remade every time. and @cfr, good to know. I guess I will use fpeval when I still want to remain in tex

@cfr See my paper on exactly this :)

@Jasper so prooftrees is still slow. but it is nowhere near as slow as it was when it did everything with pgfmath rather than doing whatever it can with tex. the argument processor in forest got extended essentially because prooftrees was unbearably slow and there really wasn't any way to make it faster without using internals. the argument processor is basically providing a public interface to faster, tex internals. but that isn't fast at all. it is just much less slow.

@Jasper take a look at matplotlib if you want to plot in Python, it can run the results through TeX to generate a PDF or save as pgf code.

@JosephWright ?? pgf is basically running its own parser, as I understand it.

@Jasper there is also gnuplot which tikz can use out-of-the-box.

Interesting. I would strongly prefer something like luacode or gnuplot which could run from a latex file, but I am also okay with doing external calculations all in python, then writing a file based on that. @Skillmon, I think I would matplotlib's functions and methods, but write the output into a tex file. Unless it can deliver the same level of control as tikz

@Jasper I don't know how fast gnuplot is. but you can use it directly in a plot specification, so the integration is definitely great.

@cfr I wonder how it would compare to luacode?

@Jasper why do you have to remake it each time for an animation?

@cfr Yes - TeX itself can't do maths beyond simple arithmetic (e-TeX a bit more) - I looked at all the packages offering maths support and how fast they are

@cfr let me grab a link real quick to show you

@Jasper I have no idea. but gnuplot is, after all, a specialist tool.

I want to make more elaborate vector fields, as in tex.stackexchange.com/a/736173/319072

the divergence/convergence ones are fast, but the curl ones are incredibly slow

I have the code to make a solenoid inducing a field through it, but it takes forever to run even on low samples

@JosephWright yes, but prooftrees doesn't need anything fancy. most of the pgfmath use was for very simple things: assignments, simple comparisons, addition/subtraction and parsing user input. very little pgfmath remains. almost everywhere, it uses the argument processor and any pgfmath is limited by using P.

@JosephWright thanks. I really was asking which paper :-).

@cfr tug.org/TUGboat/tb40-1/tb124wright-fperf.pdf

@cfr Ah, right - yes, pgf for parsing is not great, that is true

@JosephWright the logical stuff is also fine in the argument processor. only I can never get the conditional syntax right. the rest is fine, but ? never makes sense to me. (once you know the rest is just reverse Polish notation, it is fine.)

I guess I'm going to decide that an extra overfull box is an irrelevant change ...

@Jasper as I said, it creates pgf code (though not very human friendly one). But customisation is good, and you can make it use the exact same preamble as your main TeX file when you directly generate PDF. I did all my plots in university with it.

@JosephWright e-TeX just provides an expandable variant of the same basic arithmetics, no additional high level maths functions.

@Skillmon could I ask you a vim question?

@Skillmon Rounding versus truncation ;)

@Skillmon I was thinking specifically expansion TBH

@Jasper Could be a good question on the main site, showing the code and asking for ways to make it run faster.

@cfr asking's free

@JosephWright oh, yes there you're right (the rounding thing is a weirdness of e-TeX, it's correct for \dimexpr and \glueexpr, imho, but absolutely abhorrently wrong for \numexpr)

@Skillmon I thought I could use :myspell to create a spelling list from a .pws. only I have two problems. one has to do with the encoding for Welsh, but the other is English and looks as if it is working but never actually produces a compiled list. I don't get an error. I seems to start doing stuff, but nothing further. (It tells me it is doing it, but that's all.)

that isn't technically a question.

that is, I loaded the .pws and then tried :myspell %. it starts happily but nothing is every written. I wondered if you knew what I'm (not) doing.

zauguin's github action works again!

@mickep The problem is I think is it is as fast as it can get for pgfmath. I think the solution is for me to buckle down and learn the basics of lua. It seems like an open-ended question to ask, and it would request a lot of work from people to reimplement it in another format. What do you think?

@Jasper I think such a question could be useful for the next person who is struggling with a slow pgf...

@mickep Okay, I will ask a question which is framed that way

@cfr I must admit that I have no experience with custom spell lists, for me de and en work just fine.

@Jasper And then you can blame me if it is not well-received. :)

@Jasper you could also plot in luamplib from within TeX (if you're using LuaTeX)

@Skillmon You should not reveal the answers before the question is asked :P

@mickep oh, sorry :)

@Jasper Could you add an image of what that code is giving?

sure! I will replace it with more simple code then so my computer doesn't fry

@Skillmon If it needs to be 3d then mp essentially goes...

better yet, I will show a primitive picture, and explain how it will be elaborated in the code

@mickep you can fake 3D just like TikZ fakes 3D

@Skillmon I know, but it is not convenient.

@mickep see the edit

@mickep someone should write a convenient 3D lib for mp

@Skillmon That is on the todo list for a person I know. :)

@mickep I expected as much

But other projects are done before...

@Jasper one obvious performance improvement: Your \pz and \ppz are identical, why calculate it twice?

@Skillmon I wrote that part using simple syntax since I can't test the code. Usually the second one is incremented a little bit. You are definitely correct for this case though.

@Jasper Also your \LoopOverSegmentList does nothing, it just calculates a few values which are reset at the end of the implicit groups without drawing anything.

@Skillmon Did you see the \global\let? I made a recursive loop there

The idea is to superimpose values by adding them together in the loop

The values are calculated recursively, then we use the final value outside of the loop

@Jasper ahh, sorry, I misparsed the closing brace of one \pgfmathsetmacro as the closing brace of the \LoopOverSegmentList argument.

Ah, I see which one you mean. It does look like that

@Jasper the way you're using \vx, \vy, and \vz messes with TeX's safe-stack which results in a slow down.

Basically: Don't mix local and global assignments to the same macro/register in TeX.

@Skillmon Not that I'd want to do it in TeX anymore due to the speed problems, but how would I update it recursively in a \LoopOverSegmentList?

\xdef with \fpeval?

I'm thinking luacode or python will probably be the most practical answer for this

@Jasper How long is "forever" that you mention in your question? It might be good to know for someone trying...

@mickep It has a typeout which tells you where in the loop you are. I will emphasize this in the question

It will be much longer with more samples though

I make 24-frame gifs, so a gif of this would take hours

@Jasper yes, or a local temporary variable and a separate global one.

@Jasper seems like a good consensus of what an eternity is. We can conclude that an hour is longer than 4 eternities

@mickep That's sort of what the original noto sans math did, but that wasn't popular either. Some interfaces really just want the font name and it's confusing if a sans fault defaults to serifs. Then there is the issue that the Unicode math alphabets are nonsensical so if you put serifs in the serif slots you have nowhere to put sans serif Greek

@DavidCarlisle Well, we never have a place to put the sans serif Greek. They (the unicode people) should really add that alphabet. Also, where do you want to put sans bold greek? In serif bold greek or in sans bold greek? For which alphabets should one put sans in serif? I think it only becomes a mess if one does this. And I think it was the wrong decision for Noto.

@mickep they plan to, but currently most sans fonts make the "normal" slots sans serif so it's not so much an issue (except then there are no serifs at all) If the font is completely sans serif then I think that's OK just if you then want serifs you need to load the serif font as a second font

@DavidCarlisle Oh, so it sounds as if the issue will be resolved, eventually. And some kind of a mess until then. :)

@mickep it might need me to find time to push a greek alphabet proposal through the unicode process (unless you want to:-)

@DavidCarlisle I might be able to look for example usage.

(I now wonder where they found example usage of the alien crab that is stepping left. Not to mention the one stepping to the right.)

@mickep I wonder that about all the emojis

It should be sufficient to point at presentations where people are using or wants to use sans greek.

Just point them to three questions on TeX.SX

@mickep my secret plan was to refuse to play that game and assert that the math alphabets only make sense if you can switch alphabets without half the characters vanishing. So assert that if you have bold sans Greek not having sans Greek is illogical and complicates many interfaces.

Otherwise you get in to the nonsense position of having to find evidence for each letter.

@DavidCarlisle Ah, a good secret plan, indeed!

@DavidCarlisle will there also be code points for bold italic small caps Greek?

@Skillmon well one reason is every time I look at this I want to get more adventurous, the minimal request would be just the missing sans greek (I think that's sure to be accepted if someone just does all the right paperwork) then a more adventurous request would be to normalise the alphabets so that if they have greek at all they have the same set of latin and greek letters and digits, currently digits are sometimes included, sometimes not,

then there are further extensions such as different alphabets entirely, or deprecating the variant selector way of selecting mathscr or mathcal and giving them first class code points ..... then you realise it'd take years to push through unicode and iso and more years before any of the fonts caught up and you decide just living with the status quo is simpler...

@DavidCarlisle myria.math.aegean.gr/~atsol/newpage/lecturenotes/… that one could be worth to mention.

@mickep yes although that really circles back to the first point, that's a sansserif document so arguably just needs a sans math font with the sans serif greek in the usual greek slots. You really need an example of a serif document using sans serif as a distinguished math alphabet (eh for a type of Group or Category) , which also uses Greek

@DavidCarlisle Well, that is the point I do not agree with. It could very well be that you want to have a serif greek character or two in that document, and then where to get them?

I think we agree that the clean solution is to have a dedicated sans greek alphabet in unicode...

@mickep same as if you want a serif value in text, load a second, serif font. I think it's clear that the math alphabets were not intended for whole-document style changes. eg if you want to typeset in Fraktur you probably need digits and even in Roman script you may need accented of scandanavian characters (they have some weird extra ones, don't you know:-)

@mickep yes but I don't think that alphabet should be used for that document. the math alphabets are for "semantic" uses of alphabets as a set of symbols for some mathematical meaning. If you just decide to typeset a document in sans serif it's like deciding to typeset it in Times instead of latin modern, it should not involve changing the code points of the text.

Well, it is a mess, and it becomes a bit weird when there are slots for both serif and sans and they are sometimes used and sometimes not...

@Skillmon thanks. I have rather an extensive list of additions to the standard aspell dictionaries for en and a less extensive one for cy. I really do not want to add 3,300+ words for English one-by-one :-).

@mickep Unicode seems to make some weird choices given what is not included?

@cfr Indeed. Maybe the one with most money decides what goes in.

@mickep apparently logicians don't get rich, then.

@cfr Do you miss a lot of logic symbols?

@mickep some are included but with a different meaning. others are not there as far as I can tell, but it is not very easy to be sure about what is and isn't there. typesetting logic is always a pain. for standard stuff, you just need a bunch of redefinitions and the Unicode is there. but more more advanced or historical things, not so much.

@mickep yes I'm not arguing the unicode math alphabets are a masterpiece of successful design, but they are what they are. On the whole they have probably had a beneficial effect of promoting some standard (if illogical) character set for math fonts.

@DavidCarlisle Can't live with it, can't live without it.

@mickep the benefits of being extinct

@DavidCarlisle Indeed!

somebody is actually using my package?!?

@cfr Sure, why not? =)

Xour package even got its own tag!

@JasperHabicht oh, that was me.

@JasperHabicht well, I didn't put it on ctan or anything.

@cfr Hm, well ... then that's indeed remarkable

But at least it is on GitHub =)

@JasperHabicht actually, there used to be a chronos tag, I thought, but somebody must've deleted it.