@egreg yes been at it too long I stopped coding a while ago and started adding comments to the existing code. I think xii is easier to follow:-) (and \bm )
@egreg Heavens no! I once traveled back from SP to Analândia in a carnaval Friday. The sambódromo is a couple of squares nearby the bus station. The place was crowed, my bus arrived almost 2 hours late, there was no place to sit down, a lot of drunken people surrounding me, and heavy rain pouring... When my bus finally left, I heard screams in the terminal because of a blackout. The traffic was terrible, I arrived home amost 8 hours later. :)
@PauloCereda Once I had a game in Mestre (the town near Venice, where the bridge starts); we went by train on carnival Saturday and it was quite an endeavor to get off the train.
@JosephWright Did you figure out what was going on inside unicode-math?
@DavidCarlisle — I wanted to try testing variant selectors before complaining about them… but I can't find a font that supports them! I'm assuming that Mac OS X's font rendering can handle it…
@WillRobertson Well I can see that you load the same font in several ways, but I don't understand why
text 0 \EU1/latinmodern-math.otf(0)/m/n/10 select font "[latinmodern-math]/OT:s
cript=math;language=DFLT;"
text 1 \OML/cmm/m/it/10 select font cmmi10
text 2 \EU1/latinmodern-math.otf(1)/m/n/10 select font "[latinmodern-math]/OT:s
cript=math;language=DFLT;" at 10.00015pt
text 3 \EU1/latinmodern-math.otf(2)/m/n/10 select font "[latinmodern-math]/OT:s
cript=math;language=DFLT;" at 9.99985pt
@WillRobertson ^^^
@WillRobertson What's the logic here? Why not just load the font once and set it for all of the text slots?
script 0 \EU1/latinmodern-math.otf(0)/m/n/7.01236 select font "[latinmodern-mat
h]/OT:script=math;language=DFLT;+ssty=0;" at 7.01236pt
script 1 \OML/cmm/m/it/7.01236 select font cmmi7 at 7.01236pt
script 2 \EU1/latinmodern-math.otf(1)/m/n/7.01236 select font "[latinmodern-mat
h]/OT:script=math;language=DFLT;+ssty=0;" at 7.01247pt
script 3 \EU1/latinmodern-math.otf(2)/m/n/7.01236 select font "[latinmodern-mat
h]/OT:script=math;language=DFLT;+ssty=0;" at 7.01224pt
script 4 \nullfont select font nullfont
@WillRobertson Yes, sure: I'm surprised though that the Unicode font data doesn't include 'if this is superscript then scale it by ...': surely that's how e.g. Word does it
@WillRobertson Anyway, what I was trying to work out yesterday ...
@WillRobertson Well there is: TeX selects either \textfont or \scriptfont, so why don't the Unicode engines simply ignore the \scriptfont entirely and scale the text font using the scaling that I suspect must be in the font itself (as in Word or whatever there's not concept of picking a separate font)
@WillRobertson I'm thinking of the engine not the macro layer
@WillRobertson Yes, sure, but again the engine should be able to sort this you'd imagine: after all, the Unicode math font layout wasn't designed for TeX at all
@WillRobertson The core thing I'm trying to understand is what the correct settings are for load-unicode-math-data. In a Unicode set up each font slot has got all of the glyphs, but unicode-math still uses three of the text font slots (and doesn't set \textfont1). What's the logic here?
It's for compatibility with things like amsmath and so on that might query the values in the font. So I'm pretty sure you can remove them for unicode mathematics, but it's good to keep them there for ecosystem compatibility
By the way, I'm still working on a 2e stripped down unicode maths loader. I've run into an issue I'm trying to debug with Lua code and font defining callbacks. Might hold me up for a bit.
Is this actually a useful thing do you think? It would allow a "sensible default" for XeLaTeX and LuaLaTeX, but I'm not sure whether it might break legacy maths packages.
I guess legacy maths package by definition define the entire maths machinery, so perhaps they'd just overwrite everything anyway and we can all get along
I imagine we might want something similar for loading text fonts.
@WillRobertson One of the reasons I was keen to work out how @topskip's loader works. I guess to be truly useful I need to sort out TeX ligatures and system font loading there: both doable without needing all of luaotfload
@WillRobertson Actually loading the font is OK I assume: it's the additional properties that look more tricky (to me)
@WillRobertson So it would be as sensible to pick \textfont0 as \textfont1?
@JosephWright Yeah, I don't think the TeX community is going to move away from stuff like that. It would be quite nice to switch to something besides `' for quotes, so that we could use ` as an active character. But I think that'll just have to be something that a user chooses to do if they like.
@JosephWright Logical markup is good but I'm not a fan of \enquote all over the place, so having some shorthands (exactly like in csquotes!) I think is a good idea.
@WillRobertson I had a look over luaotfload but the bit that does ligatures is rather complex (it's activated as part of a much bigger block, not a section on its own)
@WillRobertson Probably some active approach would be good (active are fine provided we set them all up in one go)
@WillRobertson I suspect we'll want " active for babel-like shortcuts (yes, Unicode input is handy but there are places it's easier to use a macro-based approach)
@JosephWright Yeah, I've thought about perhaps providing a set of them (like `, @, #(?), * (?), ") and then have sensible ways to define and redefine them. OTOH, as David (?) said the other day, there's not too much problem with setting any catcode other symbol to be active for better macros for "—" and so on. So I don't know.
@WillRobertson Just easier for things like file names if we know what's active, so people can do \newcommand\foo{my-file-name@1} and get the expected result for \input{\foo} even if @ is active
@WillRobertson Feel free to send it over to me (now, if we used Git natively we could have a branch in expl3 for this ...)
@JosephWright This is a good example of where \protected doesn't go far enough. We could implement a heavy duty \protect-like layer on top to make that stuff work
@WillRobertson Yes we could, but I'd rather not. Of course, we'll be providing whatever mechanism is set up for making @ active, so we can arrange that the underlying code has a reliable list of actives
@JosephWright Thanks for the offer, but just in case it's something silly I'll slog away at it a little more. The problem is deep in the guts of the NFSS with its delayed loading of maths fonts and so on. David's callback to make virtual fonts works great for plain TeX-style font loading, but as soon as I tried in a maths font everything goes haywire
@JosephWright I'm not so sure that we shouldn't have a more rigorous approach to "typesetting states" that this sort of system would fall into. I forget the examples now, but I remember there was a brief discussion about doing things like
@WillRobertson Hmm, one has to assign \textfont2 and \textfont3 and that restriction doesn't seem to be lifted by the Unicode engines
@WillRobertson I get the feeling no-one has really looked at this properly (Hans doesn't actually do a lot of math mode work whilst Jonathan K. presumably just made minimal changes)
@DavidCarlisle Will outlines above why unicode-math sets various families/\fontdimen values, and that's all to do with working with 2e
@DavidCarlisle I think the math mode support in XeTeX/LuaTeX hasn't really been fully 'tried out' in a set up that doesn't need to work with plain/2e conventions
@DavidCarlisle For example, you have to set up \textfont2 and \textfont3 even though with a Unicode engine you only need one font
@DavidCarlisle Similarly, you have to set up \scriptfont even though for a Unicode font there is data in the font about scaling and (presumably) other software using such fonts doesn't have to load them (at least) twice (Word is the obvious)
@JosephWright yes but at least in luatex you must be able to avoid having to actually load the font data twice surely and just scale it in the font structure?
@JosephWright do you have a "released" 0.89, can you try luatex '\showthe\meaning\coprod \end' for me (rebuilt of texlive svn) I'm still getting \mathchar"1"03"000060 but I hoped to get \Umathchar"1"03"000060
@WillRobertson ah so it could be made to work then...
@WillRobertson making it work in math is a detail for the unicode-math maintainer
@WillRobertson makes it a bit tricky to go back to unicode and say we are using a pre-unicode math layout engine written in 1980 and it doesn't understand the selector characters..
@DavidCarlisle Maybe we could ask them to add a new OpenType feature (specifically, rather than just +ss01) so that a well behaved maths font should support either a global switch with a font feature, or a local switch with a variant selector.
that should be "both...and..." instead of "either...or...."
@WillRobertson but that's a different "them" (font designers rather than UTC) note stix2 already had a request to use less OTF features and make glyphs available via direct input as css can't (by standard although some mozilla extensions) set OTF features so the request was to put all glyphs that are accessed as stylistic variants also into the PUA (although that idea wasn't universally popular)
@WillRobertson why don't we take that example (extended to have both text and math versions) and then go back to the email list (or perhaps better Murray's blog comments) and say the variant selectors are tricky in math layout and despite the fact VS1 has been in unicode since Unicode 3.2 it isn't supported at all in Math renders as far as we can tell.
@JosephWright @WillRobertson meanwhile I suppose I should up the kernel date and modify xetex for conditionaly allowing more character classes on a new enough xetex...
@JosephWright haven't checked to be honest but I ran make hoping to get a new luatex and it rebuilt xetex after I rsynced yesterday, so something has changed:-)
anyway plan today was to wade through more of amsmath:(
@JosephWright @DavidCarlisle @WillRobertson: I just wrote Javier a mail about it. (While imho polyglossia will have to adapt its code, the babel documentation could be clearer regarding the commands of "babel in the format" and "babel package".)
All of the problems I had were exactly what Joseph was asking me about in unicode-math. It's not TeX's \font that won't allow a font to be loaded twice, it's NFSS!
# 2: class, one of:
#
# N - Normal - includes all digits and symbols requiring only one form
# A - Alphabetic
# B - Binary
# C - Closing - usually paired with opening delimiter
# D - Diacritic
# F - Fence - unpaired delimiter (often used as opening or closing)
# G - Glyph_Part - piece of large operator
# L - Large - n-ary or large operator, often takes limits
# O - Opening - usually paired with closing delimiter
# P - Punctuation
# R - Relation - includes arrows
@DavidCarlisle polyglossia repo issues, for example
@JosephWright I just automatically set all glyphs with "arrow" in their name to have a delcode that will allow them to stretch if the font supports it — it seemed sensible at the time :)
@JosephWright Perhaps people shouldn't be using fonts for drawings graphics!!
@WillRobertson mathml (where much of that comes from) just bails out and says that whether or not any character can stretch is font and system specific. the syntax allows you to specify stretching on anything but it may or may not stretch
@WillRobertson which reminds me, when you asked the other day if world was ready for lua/xelatex to default to unicode-math my reason for saying no wasn't the macro layer, but that I haven't a lot of confidence that the underlying spec is clear enough or that either of the engines is doing something sensible in all cases.
@WillRobertson but if You Fred and Khaled have trouble working out what the spec means to make a root sign, do worry whether the spec is as clear and useful as it might be...
@WillRobertson yes lua itself is pretty fast luatex I think is slow because of all the inter-language communication rather than either language being slow itself
@WillRobertson that's sort of my day job as well, for some of our library we added "vectorised" special functions which calculate a special function for all elements of an array instead of just one, in the fortran interface it's more or less pointless just saving one loop, but in matlab it's an order of magnitute faster to just pass the data to fortran once rather than loop in matlab and pass each value separately
@WillRobertson yes although I don't know if that is Word or just if cambria doesn't have VS1 glyphs still I suppose I could post and let Murray answer that
@DavidCarlisle Yes, I've spent quite some time vectorising my own Matlab code :) The trick is knowing when to stop so that you can read the code when you come back to it in 6 weeks time…
% Prepared by Peter F. Patel-Schneider, liberally using the ideas of
% other style hackers, including Barbara Beeton.
% This style is NOT guaranteed to work. It is provided in the hope
% that it will make the preparation of papers easier.
% NOTE: Some laser printers have a serious problem printing TeX output.
% These printing devices, commonly known as ``write-white'' laser
% printers, tend to make characters too light. To get around this
% problem, a darker set of fonts must be created for these devices.
@JosephWright "/OT" means OpenType (XeTeX only), as opposed to Graphite or AAT. "script=math" turns on the OpenType math features. "language=DFLT" is almost certainly redundant :)
@JosephWright That's important. It's a luaotfload thing:
luaot oad has two OpenType processing modes: base and node. base mode works by mapping OpenType features to traditional TEX ligature and kerning mechanisms. Supporting only non-contextual substitutions and kerning pairs, it is the slightly faster, albeit somewhat limited, variant. node mode works by processing TEX’s internal node list directly at the Lua end and supports a wider range of OpenType features. e downside is that the intricate operations required for node mode may slow down typese ing especially with complex fonts and it does not work in math mode.
@JosephWright When I translated it from NFSS the first time, I didn't preserve the original structure of the code (with reference to 2e macros, e.g.). I now think that was a mistake.
@JosephWright We have very different philosophies on that but in this case I agree :) Have you read the essay about how starting from scratch basically killed Netscape?
@JosephWright Your code is always better than mine so I can't exactly comment too forcefully!!
@WillRobertson Well (in seriousness) I've concluded that for xor starting again would be a waste and unlikely to work, hence the desire to get David and Frank to do some new work on it
@WillRobertson For the FSS, the existing code is probably quite a good starting point so I'd likely go with it, but try to address the areas that could be updated/improved
@JosephWright Yes. From memory where I was stumped originally was trying to stick too closely to NFSS methods and then realising we'd be breaking compatibility if we had new names for, e.g., \f@family. So I hope with a revisit we can develop a new interface that can hook into 2e code in l3in2e but use "proper" internal naming schemes in the format.
I mean, we definitely know the code is rock solid, but it's pretty hard to follow :) A rewrite nowadays might eliminate some of the misdirections/obfuscations in there
@WillRobertson I'm thinking that any rewrite here is like xgalley: only usable with code that knows about it, and working on 2e as we can't yet do real testing without it
