As some may know, I'm implementing a huge TeX macro package which has to do a lot of internal computations, and hence will give a lot less overall performance "per page" than, for instance, your usual LaTeX document.
Looking for quick wins in enhancing performance, I'm currently investigating wh...
@egreg But especially \ratio (implemented by an approximation algorithm) would be a good candidate to replace by something like \dimexpr\p@*#1/#2\relax followed by \strip@pt.
@StephanLehmke We used to have some expl3 code that was a straight reimplementation, but the rounding/truncating business caught us out, and to be honest it was a bit of a waste of time (as calc works perfectly well)
@StephanLehmke Then you wouldn't want what we had: it was more a 'thought experiment', at least as I understood it.
@StephanLehmke The benefits of expandable approaches don't necessarily extend to performance, depending on the use case.
@StephanLehmke On the arrays business, I'm mainly going on the analysis of TeX's hashing algorithm by one of the other people on the team (I forget who: might be David Kastrup). The argument runs that TeX's algorithm isn't that good, so hash collisions are likely to happen with arrays of any significant size and number: it got raised in the context of tables of data rather than simple lists.
However, I've not tested this out carefully: I'm no CS expert so I tend to rely on others for that sort of thing.
@egreg What exactly is the significance of that on a decimal representation? I sincerely doubt the precision of the current \ratio calculation is enough to make a comparison even possible.
@StephanLehmke We had a bug report about expl3 messing up some calculations that came down to altering calc and changing from truncating to rounding. It was an 'edge case' situation, and we could have worked around it I guess, but that's not a priority for us.
@StephanLehmke I'm not sure if the rounding business came up in \ratio: the code we have that reimplemented it is rather more involved than your approach :-)
I have to take back the snide remark concerning precision of \ratio. I'm getting exactly the same results for calc and \dimexpr. OTOH, this means it might be backwards compatible after all ;-)
I seem to have a serious English problem in scientific writing. In the introduction of my article, I use "study" in the present tense, but "used", "was interested", "started to" etc. in the past tense, and I mix these. That's probably not quite correct, right?
@DavidCarlisle yeah, but "studied" sounds so "terminated" to me. Yes, it is terminated since the articles I cite are "finished", but it gives me the impression that we do something that is not "modern". But that's probably only me. I'll change these "study" to "studied" and just be happy :)
@PauloCereda Handling dtx is really easy, you just need a few elisp helper functions to add the boilerplate markup required. Oh no!, you don't use emacs. shame.
@StephanLehmke I tried one of your tests translating into a property list; it's really slow, compared to using the hash table (for 1000 objects a factor 10); basically a property list is a macro expanding to a sequence of items of the form \__prop_pair:wn <name>\s__prop {<value>}.
@egreg David K is very down on building arrays in TeX generally, but realism means you can't ignore the requirement to do this at least on a small-ish scale
@JosephWright I don't see why; if one uses property lists for “reasonable” things, the overhead is not too much. For big projects, one has better to define a specific data structure.
@JosephWright well, IMHO gnuplot by default produces a reasonable output, surely compared to E#c@|. It's not beatiful, but I think you can make it beautiful if you have some experience
@tohecz I've never really pursued gnuplot but the gallery I've seen doesn't suggest it's as easy to get good results as with pgfplots (for the plotting itself: calculations a different thing)
@JosephWright yes although when the first version of property lists was devised, the idea wasn't to improve speed but to store it at all when you only had a few hundred csnames left after loading the format....
@DavidCarlisle I know there is history: all those odd c-type variants we used to have
@DavidCarlisle Point being here that we might want to rethink how to do stuff if the decisions come out differently: having an interface for storing in a keyval-like structure is a 'good idea' I think
@egreg I don't think the situation is that bad. To give an example, DocScape creates a 733546 element "hash array" to "unfold" the XML data tree of a 70MB xml data file for processing. There are a lot of things that make DocScape slow, but this is not one of it (that is, DocScape can process huge amounts of data this way (much huger than that) without getting noticably slower than it already is).
@JosephWright In fact, if you think that in my tests, 100000 array elements are created and 1000000 accesses to them performed in a total of, say, 4 seconds, you can imagine that if you add actual processing of that data into the equation, the actual array accesses will be quite insignificant :-)
@StephanLehmke I can see that. One (minor) issue is that you're testing limits on LuaTeX, but pdfTeX is rather more 'tricky' in that regard as we don't have dynamic memory allocation.
@StephanLehmke I'll see how quickly I can write an alternative l3prop and do some tests
As some may know, I'm implementing a huge TeX macro package which has to do a lot of internal computations, and hence will give a lot less overall performance "per page" than, for instance, your usual LaTeX document.
Looking for quick wins in enhancing performance, I'm currently investigating wh...
To give another example: In DocScape, it happens very often that a "variable" containing a token list is "extended", for instance when building a text or tabular structure from data (note there is no expandable code in DocScape at this point ;-)
That is, something like var := var + x. The usual way is to implement this like \edef\var{\unexpanded\expandafter{\var}\unexpanded\expandafter{\x}}.
But if a lot of these extension happen, the quadratic performance becomes quite noticeable (this is where you normally cue in an ultracomplicated expandable implementation :-)
My simpler solution is to use something like \def\var{\varii}\edef\varii{\noexpand\vari\unexpanded\expandafter{\x}}, of course with proper counting of the "subvariable chain", garbage collection etc. The performance gain for large structures is dramatic.
@JosephWright Yes I wasn't suggesting this as a solution to anything; just pointing out that creating a huge amount of macros wasn't noticable at all performance-wise.
@JosephWright Well, maybe there should be a "chain mode", and after a "linearization": You would create many macros locally for the performance when you add them, but proccess them again (in O(n)) to avoid having them in the end
btw, English policy question (amongst scientist): If someone never signed a mail to you just by his first name and uses "Regards, John Doe", but he says "Dear Tom". Can you just say "Dear John" in a mail, or better stuck with "Dear Prof Doe" or "Dear John Doe" ?
@JosephWright Looking at this from a complexity-theoretic pov, it would basically come down to storing huge amounts of data and then doing most of the computation without involving them, which sounds improbable.
Still, I'd be interested in some concrete test results on this...
@tohecz I agree with Joseph too. It seems to me that only if you sign an email with just your first name do you expect to be addressed by your first name.
@PauloCereda So the problem with transparency was in \fill[fill=transparent] (0,1.2) circle (2pt) which was enforcing all my little figures have the same height :)
@Ahm No. You have to manually issue a \chaptermark or \sectionmark command, if you only want to change the heading. But the real answer is: use short titles with classicthesis.
@tohecz In my notes I also have \sim_{H} and \mathrel{{}_{H}\sim} for the equivalence relations in a group defined by a subgroup H: $a\sim_{H}b$ when $ab^{-1}\in H$, $a\mathrel{{}_{H}\sim}b$ when $a^{-1}b\in H$.
@Tobi: Your effort is really appreciable. Refer: tex.stackexchange.com/q/147991/11232 But in cases like this, the OP may say I don't know where to start. In such scenario, shall we try to add some code just as an illustration? I will do it for this question. Pour in your opinions. :-) And we are doing it only for some time if we are doing it.
@egreg Thanks for the hint. :-) (Adding one more beer to the count). I just woke up but still sleeping a bit ;-)
@HarishKumar I noticed \lipsum[1] and tried changing it to \lipsum*[1] so as to avoid the final \par, but the output was the same, so I tried doubling the OP's input. Then I decided to look at the manual. @DavidCarlisle will surely say something about this insane practice.
