@PabloGonzálezL not sure I understand. the ctan readme has to be installed somewhere you can't just use it for ctan (a ctan rule not an l3build one)

@DavidCarlisle I explained myself badly (a thousand apologies). The file for ctan is correct, the problem is for the local installation...it shouldn't be installed in doc if I don't want it.

@UlrikeFischer docinit_hook' didn't work :(

@UlrikeFischer @DavidCarlisle With a "mix" between 'docinit_hook', 'demofiles' and 'typesetdemofiles' I have succeeded. I don't look like I'm very "fit for purpose" but for now it's the best I can do :)

@PabloGonzálezL It has to be in doc, that's where the CTAN readme belongs, so this is essentially hard-coded

@PabloGonzálezL l3build installdoesn't install documentation unless you give --full

@PabloGonzálezL I likely need to look again at README handling; we started with our own set up, then expanded to cover the case where there are two README-XXX files, but with the assumption they are all in the top-level directory (a common pattern)

@PabloGonzálezL Local installation and creating a CTAN archive use the same code: the idea is that you get locally exactly what goes into a TDS-style zip file

@PabloGonzálezL Cutting down to essentials, I get a build.lua that says

-- Build config for demopkg

-- File location and identification
module = "demopkg"

-- Configuration of files for build and installation
sourcefiledir = "./sources"

installfiles  = { "demopkg.sty","example.tex" }
typesetdemofiles = {"example.tex"}
tdslocations  = {"doc/latex/demopkg/example.tex"}

textfiles = { "sources/CTANREADME.md" }
ctanreadme = "CTANREADME.md"

@PabloGonzálezL Features that seem lacking in l3build here are (1) documentation about how the README file is treated and (2) a packeddemofiles variable to cover doc files that have to be extracted but which should have their source file installed (the usual assumption is that packed files are only extracted-and-installed if they go into the 'live' tree)

@PabloGonzálezL A new variable would deal with installfiles, typesetdemofiles and tdslocations in the above, leaving just packeddemofiles = {"example.tex"}

@PabloGonzálezL On the README, I think we need more documenation

@PabloGonzálezL BTW, I realise for your use case you need more of the Lua you have in your 'real' demo, but I was trying to get it down to the core issues

@PabloGonzálezL I hope overall you've found l3build to be a reasonable approach; you can code these things up yourself, but we hope we are providing a good starting point and common tools

@PabloGonzálezL At some stage, I might ask to add your demo to the main l3build repo: we have a few 'common layout' cases there

@PabloGonzálezL I guess I could add textfiledir to go with sourcefiledir and docfiledir, as that would make it clear that text files have their 'own home'

@JosephWright all of which reminds me I failed to get hyperref ctan zips to make when I last looked (so it's disabled again at the moment) I forget the details but it was again as far as I recall a matter of forcing extracted things to go in unexpected places. (one could always just change the sources to fit but it's a good test if you can generate the same distribution zip contents without changing the source.

@DavidCarlisle Sure, yes; it's a question of how to provide the flexibility

@DavidCarlisle If you could check what the issue is, we can see if it ties up with @PabloGonzálezL's use case and what might be adjusted

@JosephWright yes will look later

quack

@DavidCarlisle, @UlrikeFischer Interesting questions on the TL list

@PauloCereda boo!

@JosephWright dvi specials or tc fonts or both:-)

@JosephWright \documentclasss?? ;-) But while looking at it: Doesn't the --recorder option show tfm files?

@DavidCarlisle I'd forgotten about Frank's changes: they, much more than expl3, show up

@JosephWright as was to be expected ;-) I'm quite confident that we will see here a number of questions ...

@JosephWright something seems off with de-alt under pdflatex:

\documentclass{scrartcl}
\usepackage{xparse}
\usepackage{iftex}
\iftutex
\else
 \usepackage[T1]{fontenc}
\fi
\ExplSyntaxOn
\NewDocumentCommand\uppercasetesting{m}
  {
    \tl_analysis_show:x { \text_uppercase:nn { de-alt } { #1 } }
  }
\cs_generate_variant:Nn \tl_analysis_show:n { x }
\ExplSyntaxOff

\begin{document}
\uppercasetesting{Grüßeẞ}
\end{document}

Log output:

>  G (the letter G)
>  R (the letter R)
>  <c3> (active character=macro:->\UTFviii@two@octets <c3>)
>  <9c> (active character=macro:->\UTFviii@invalid@err <9c>)
>  S (the letter S)
>  S (the letter S)
>  E (the letter E)
>  <e1> (active character=macro:->\UTFviii@three@octets <e1>)
>  <ba> (active character=macro:->\UTFviii@invalid@err <ba>)
>  <9e> (active character=macro:->\UTFviii@invalid@err <9e>).

Under LuaLaTeX:

>  G (the letter G)
>  R (the letter R)
>  Ü (the letter Ü)
>  ẞ (the letter ẞ)
>  E (the letter E)
>  ẞ (the letter ẞ).

@JosephWright somehow for pdfLaTeX de-alt doesn't seem to be considered at all.

@Skillmonlikestopanswers.xyz It's not

@UlrikeFischer Indeed

@Skillmonlikestopanswers.xyz pdfTeX can't show the capital Eszett as it's not part of the T1 font encoding, so nothing happens: only T1 is covered

@JosephWright so you just shortcut the inputenc mapping, ok, that explains it. I was just curious.

@JosephWright if the target is T1, why is the result SS and not \char"DF?

@JosephWright \SS is in T1 ?

@Skillmonlikestopanswers.xyz Not exactly, no

@DavidCarlisle Huh?

@JosephWright t1enc.dfu: \DeclareUnicodeCharacter{1E9E}{\SS}

@DavidCarlisle Which is wrong ...

@DavidCarlisle Nope

@UlrikeFischer Because Unicode say Eszett maps to two S characters in the default mapping

@JosephWright ah sorry that's SS. Hmm now I'm confused, where did the capital ss go:-)

@JosephWright I'm sure I remembered discussion of a capital es-zed at the time the ec fonts were being made, but I see it ended up as SS

@Skillmonlikestopanswers.xyz unicode.org/versions/Unicode12.0.0/ch05.pdf

@JosephWright but "DF is the uppercase of ß regarding the uc/lc code. That is doesn't look right in the ec fonts shouldn't matter. One could create pdflatex fonts from one of the newer open type fonts which inserts here the 1E9E glyph.

> The German sharp s character has several complications in case map-ping. Not only does its uppercase mapping expand in length, but its default case-pairingsare asymmetrical. The default case mapping operations follow standard German orthogra-phy, which uses the string “SS” as the regular uppercase mapping for U+00DF ß latinsmall letter sharp s.

@UlrikeFischer As a starting point for what to cover, I had to make some choice about codepoints outside the 8-bit range; going with only those covered by T1 is meant to give some sense of predictablity

# The German es-zed is special--the normal mapping is to SS.
# Note: the titlecase should never occur in practice. It is equal to titlecase(uppercase(<es-zed>))

00DF; 00DF; 0053 0073; 0053 0053; # LATIN SMALL LETTER SHARP S

@JosephWright well it may be the best that can be done though, \SS is defined as being the uppercase of \ss so arguably the fact that it looks like SS not B is a font choice. The other possible mappings would be nothing (so an error) or a real SS I suppose.

@DavidCarlisle There's a mapping for \ss specifically (taken from \@uclclist), but here we are not talking about that but rather about ß

@JosephWright yes I understand this. But T1 does cover the uppercase of ß. That means even in T1 you have the choice if you want to go ß -> SS or ß ->\char"DF. It shouldn't matter how \char"DF actually looks.

@UlrikeFischer \char"DF isn't captial Eszett ..

@UlrikeFischer 1E9E;LATIN CAPITAL LETTER SHARP S;Lu;0;L;;;;;N;;;;00DF;

@JosephWright yes I was replying to your comment that the inputenc mapping \DeclareUnicodeCharacter{1E9E}{\SS} is wrong.

@JosephWright well it is, it just has a different shape to the one you expected.

@JosephWright but it could be. I could create a pdflatex font which moves a 1E9E there. Or I could move a duck there. But it still would be the "uppercase ß codepoint".

I'm so happy that I could spark this discussion. Sorry @JosephWright :)

@Skillmonlikestopanswers.xyz so do you think this is a capital es-zed with a weird shape, or two S weirdly pushed into a single codepoint?

\documentclass{article}

\usepackage[T1]{fontenc}

\begin{document}

[\ss\SS]
\end{document}

@DavidCarlisle I'd say if the code point exists, it should be used. It's the font which should decide what to display. So this is just a very weird looking capital es-zed

@Skillmonlikestopanswers.xyz That's the line or reasoning @UlrikeFischer and I were making....

@DavidCarlisle I know, I've read your messages.

@Skillmonlikestopanswers.xyz Yes but it was worth highlighting that I was agreeing with @UlrikeFischer (rare event:-)

@DavidCarlisle just so you can later blame her, admit it!

@Skillmonlikestopanswers.xyz I think we can blame Jörg Knappen here for not reading the future.

My reputation is currently made of two palindromes, is it twice as good as yours, @DavidCarlisle?

@Skillmonlikestopanswers.xyz I checked with my punctuation expert @barbarabeeton, and the comma is mis-placed. (and anyway my rep is made up of 6 palindromes)

@DavidCarlisle no, it's not. The comma separates the two palindromes perfectly, whereas in yours neither 445 nor 544 is a palindrome.

@DavidCarlisle No, it's lower-case!!

@UlrikeFischer \char"DF is the codepoint for lowercase; the starting point for all the expl3 code is the input is Unicode-compliant

@JosephWright but not in T1

@JosephWright so if you start with U+00df and uppercase choosing to get to U+1e9e then typeset using T1 shouldn't you have the option of ending up with \char"DF which coincidentally is the number you started with

@JosephWright as David says: not in T1. It is accidental that the number of the T1-code point for an uppercase ß is identical to the unicode number of the ß itself. I mean take another char where unicode and T1 differ:

\documentclass{article}
\usepackage[T1]{fontenc}

\begin{document}
U+119: \char166 ę

U+118: \char134 Ę

\ExplSyntaxOn
\text_uppercase:n {ę}
\text_lowercase:n{\text_uppercase:n {ę}}
\ExplSyntaxOff

\end{document}

@DavidCarlisle Ahahaah. I'm reading you comment for \dagger (+1 for the comment) tex.stackexchange.com/questions/528262/… ahahahahahahahh

@UlrikeFischer @DavidCarlisle The point is though that I needed some sense of coverage that was reasonable for 8-bit. So the position currently is that a change is made only if both codepoints have glyphs in T1. There is no uppercase Eszett in T1, so no attempt is made to convert in the de-alt set up. It's the same as askying why Cyrllic or Greek isn't attempted with 8-bit engines

@JosephWright yes the general stance is as expected, so the only point of contention is the status of this statement "There is no uppercase Eszett in T1" which is sort of true or sort of false, depending...

@JosephWright Hm. I'm not sure how to explain myself. Let's look at the T1-fonttable. Position "41 is "Capital A", Position "86 is "Capital E with ogonek". How would you describe Position "DF? For me it is "odd looking Capital ß". But what is it for you?

@UlrikeFischer Where are you looking?

@JosephWright simply at a T1-table. e.g.

\documentclass{book}
\usepackage{fonttable}


\begin{document}
\xfonttable{T1}{cmr}{m}{n}
\end{document}

@UlrikeFischer Looks like SS, i,e. not a Unicode codepoint at all ...

@JosephWright you are describing the look not the position or the meaning of the glyph. If I create a T1-version of dejavu sans and use the 1E9E glyph of dejavu for this position, how would you describe the position then?

@JosephWright It is specified as being the uppercase of es-zed and unicode is not supposed to care about glyph shapes only their "meaning"

@DavidCarlisle Yeah ... that's not totally realistic, is it

@DavidCarlisle Is there a proper list of the T1 positions semantically?

@JosephWright But I think here, it is in fact really the case.

@JosephWright one of the original cork encoding papers I suppose, but texdoc cork shows the encoding file and:

 0xD0
       /Eth /Ntilde /Ograve /Oacute /Ocircumflex /Otilde /Odieresis /OE
       /Oslash /Ugrave /Uacute /Ucircumflex /Udieresis /Yacute /Thorn /Germandbls

@DavidCarlisle Hmm, OK

@UlrikeFischer, @DavidCarlisle Ok, Ok, I'll change things

That is DF is /Germandbls

@JosephWright someone will complain (there is really no right or wrong here)

@JosephWright well you have to think about historically. Imho it was quite foresighted to realize that a code point for Capital ß would be needed in future, they only missed the look it would have.

@UlrikeFischer for EC fonts that is the case, but now any new opentype font mapped to T1 for pdftex still gets an SS in that slot, I wonder if it would be safe to say that fonts "should" use a capital es-zed glyph....

@DavidCarlisle well if the code does ß -> SS for language de, and ß -> whatever is in "DF for de-alt, it would be in sync with the open type behaviour.

@UlrikeFischer not so much thinking about uppercaseing but a simple direct use of \SS or \symbol("DF} would it be OK in a pdflatex T1 encoded document if they made a B-thing instead of SS-as-a-single-glyph-thing

@DavidCarlisle I looked at it. If a font has a glyph in 1E9E it could be mapped to the "DF position when using otftotfm (and probably should), but I couldn't find many sensible candidate fonts, Sourcesans pro perhaps. As default otftotfm fakes the "SS" combination.

@DavidCarlisle yes, as I said: whatever is in position "DF, may it be a SS or something looking like a tree or a beta ...

@Skillmonlikestopanswers.xyz -- @DavidCarlisle is taking my name in vain. Commas are problematic, but I recuse myself from this one.

@barbarabeeton would I do such a thing?

@DavidCarlisle -- Would you not? <grin>

@DavidCarlisle I've just discovered that TL'19 includes zip.exe, so everything we need for l3build is now in the TeX system (new PC, no GOW at present)

@JosephWright is zip a new addition this year?

@JosephWright I'd prefer .tar.gz though :)

@DavidCarlisle Seems so

@Skillmonlikestopanswers.xyz You are not CTAN

@JosephWright Do they prefer zip? I always sent them tar.gz archives (except when using l3build...) and no one ever complained.

@JosephWright how much of texfaq.org do you control? could you make a fr.texfaq.org?

@Skillmonlikestopanswers.xyz according to documented interfaces they only accept zip

@DavidCarlisle from ctan.org/help/upload-pkg:

@Skillmonlikestopanswers.xyz They require zips for the TDS-style case

> Please package your contribution as a ZIP archive with extension .zip or a gzip-ed tar archive with the extension .tar.gz or .tgz.

@DavidCarlisle @StefanKottwitz could do that

@DavidCarlisle Perhaps I can, actually: I'd have to check

@Skillmonlikestopanswers.xyz oh so they do now (they must just unpack it and repack it again though as you can only download the ctan or tds archives as zip I think?

@DavidCarlisle I think they only allow the TDS as zip (as mentioned by @JosephWright)

@DavidCarlisle, @UlrikeFischer PL2 sent

@JosephWright thanks

@DavidCarlisle After a bit of fun, so is PR2

@MarcelKrüger after the long discussion today about the Capital ß: could one implement a general "if font hasn't 1E9E use SS instead" substituation in luaotfload?

@UlrikeFischer Useful discussion

@JosephWright sorry again for restarting it :)

@Skillmonlikestopanswers.xyz It's fine: give me about 5 minutes and I'll have adjusted the code

@Skillmonlikestopanswers.xyz We are getting close to beign able to replace \MakeUppercase, I think

@JosephWright it looks promising. Did you benchmark against \MakeUppercase yet? (I think I'm a benchmark addict)

%    \emph{Note} that this implementation is subject to change!  At
%    the moment we're not providing any way to extend the list of
%    uc/lc commands, since finding a good interface is difficult.
%    These commands have some nasty features, such as uppercasing
%    mathematics, environment names, labels, etc.  A much better
%    long-term solution is to use all-caps fonts, but these aren't
%    generally available.

@Skillmonlikestopanswers.xyz Not worth it: performance-wise, \MakeUppercase will win by a country mile. That's really not the point here.

@JosephWright I know, I'm just genuinely interested.

@JosephWright Can I move a discussion below an answer to chat before the "move this discussion to chat" message appears?

@PhelypeOleinik Er, I guess so ...

@PhelypeOleinik I certainly can ;)

@Skillmonlikestopanswers.xyz Let me fix the current issue and I'll report

@JosephWright but you're a wizard!

@JosephWright Ooh, mod powers :-)

@JosephWright I couldn't find an obvious button. On second thought, I'll just hope the comments don't extend long enough for me to need that :-)

Thanks!

@UlrikeFischer Have you tried if that already works with HarfBuzz?

@Skillmonlikestopanswers.xyz For a simple \MakeUppercase{foo}:

3.66e-6 seconds (9.47 ops)
2.3e-4 seconds (960 ops)

for the L2 and L3 versions

@JosephWright that's only factor 100... :)

@MarcelKrüger what?

@UlrikeFischer Ups, I was trying to refer to replacing ẞ with SS if the former is missing. (I thought that it might work because I thought that SS is the compatibility decomposition of ẞ, but I was mistaken)

@UlrikeFischer For mode=node it's easy, I will have to think about the best way to do such things with mode=harf.

@Skillmonlikestopanswers.xyz Yes, it's not super-tight but the code is pretty well-established

@Skillmonlikestopanswers.xyz Key is it does the Right Thing

@UlrikeFischer (@Skillmonlikestopanswers.xyz, @DavidCarlisle) I wonder if there is an 'official' name for the change from SS to grosses Eszett: something like de-2017?

@JosephWright: Regarding your latest question about case... consider this example:

\documentclass{book}

\usepackage{lipsum}

\begin{document}

\sloppy
\chapter{A chapter before Chapter~\ref{ch:two}}
\lipsum[1-50]
\chapter{Second chapter}
\lipsum[1-50]

\end{document}

LaTeX spits outs warnings in capital letters...

Chapter 1.

LaTeX Warning: Reference `ch:two' on page 1 undefined on input line 8.

[1


{c:/texlive/2019/texmf-var/fonts/map/pdftex/updmap/pdftex.map}]

LATEX WARNING: REFERENCE `CH:TWO' ON PAGE 1 UNDEFINED on input line 10.

[2] [3]

LATEX WARNING: REFERENCE `CH:TWO' ON PAGE 1 UNDEFINED on input line 10.

[4] [5]

LATEX WARNING: REFERENCE `CH:TWO' ON PAGE 1 UNDEFINED on input line 10.

[6] [7]

LATEX WARNING: REFERENCE `CH:TWO' ON PAGE 1 UNDEFINED on input line 10.

[8] [9]

LATEX WARNING: REFERENCE `CH:TWO' ON PAGE 1 UNDEFINED on input line 10.

@JosephWright "§ 25 Ergänzung 3" of "Rechtschreibung 2017"

@Skillmonlikestopanswers.xyz Er, I mean a BCP 47 name ...

@JosephWright oh.

@JosephWright I don't think so, it was not a switch to a new spelling like the ngerman, but only an addition "allowed is too to use ...". In view that many fonts don't have the char it wouldn't make much sense to enforce it.

@JosephWright I don't think there will be a single one, as Germany, Austria, Swiss, Liechtenstein, South Tirol, and parts of Belgium are affected, which used to have their own subtags, iirc.

@Skillmonlikestopanswers.xyz Yes, that's probably the next thing to sort

@UlrikeFischer I thought as @Skillmonlikestopanswers.xyz said it was part of Rechtschreibung 2017, so is mandated ...

@JosephWright no, it is not mandated. That collection of rules states something like "You might also use ẞ instead of SS". Both are allowed.

@Skillmonlikestopanswers.xyz BCP 47 tags are divided into parts, so it's possible to combine 'things'; I likely need to work on splitting out locale, language, variants, ...

@JosephWright literal according to Wikipedia: „Bei Schreibung von Großbuchstaben schreibt man SS. Daneben ist auch die Verwendung des Großbuchstabens ẞ möglich. Beispiel: Straße – STRASSE – STRAẞE.“ – translation: "When writing in all-caps one uses SS. Alternatively the usage of the capital ẞ is possible, too. Example: Straße – STRASSE – STRAẞE."

@JosephWright: Not sure whether the capitalization in the .log is something you're concerned about.

@Skillmonlikestopanswers.xyz There's a reason my working title was 'alt'

@Werner Huh??

@JosephWright Scroll up a bit, there's an example. I tried with expl3's case changing and the output is correct. Didn't trace it to see exactly what happens...

@Werner Would be a good example: go for it ;)

@PhelypeOleinik Right, spotted it

@JosephWright :) The name is good, just leave it like that :)

\documentclass{article}
\usepackage{fontspec}
\usepackage[turkish]{babel}
\begin{document}
\end{document}

@JosephWright @UlrikeFischer @WillRobertson ^^^

@PhelypeOleinik Oh drat

@PhelypeOleinik yes, was just looking.

@PhelypeOleinik fontspec is not involved it errors also with pdflatex

@UlrikeFischer Hm... Indeed

@JosephWright active : when backend code is loaded?

@JosephWright @UlrikeFischer This line in babel.def seems suspicious:

\g@addto@macro\@uclclist{\reserved@b{\reserved@b\bbl@uclc}}%

@PhelypeOleinik Yes!

\reserved@b doesn't seem to come from anywhere...

@PhelypeOleinik It's a throw-away

@JosephWright Yes, but I don't see any other part of the code defining or using it... In fact, at the time the code is in that line, \reserved@b still contains \\SetCase [{}] from the optional-argument checking.

@JosephWright I'm pretty convinced that that line is wrong...

@PhelypeOleinik It's relying on low-level behaviour of \Make...case implementation

@JosephWright Ah :(

@PhelypeOleinik Need a \tl_if_single_token:n test

But moewe is right, it compiles on miktex, so the problem is quite new.

@PhelypeOleinik, @UlrikeFischer I have a fix

@JosephWright what was the problem?

@UlrikeFischer babel is using low-level tricks

@JosephWright and why did they break now? Who do we blame? ;-)

@JosephWright Example added.

@JosephWright Great!

@UlrikeFischer babel injects a trick into \@uclclist, aiming to get \let \reserved@b \reserved@b then it's own code. But that means we no long have an even number of entries in \@uclclist, and the assumption that we will have breaks down

@JosephWright I see that you have a fix latex3, but should babel be corrected too? It doesn't sound like something it should do. Or is there a sensible hook missing?

@UlrikeFischer I've added a test for single-token condition

@UlrikeFischer Well, \@uclclist is very well-established and only used in one place. The source does say that there is no good interface (see ltfinal.dtx)

@UlrikeFischer, @PhelypeOleinik Of course, I'd literally just sent an update to CTAN, so there will be another in the morning ...

@JosephWright CTAN will love you.

@UlrikeFischer Oh, they have for years

@UlrikeFischer It was easier with Robin ...

@JosephWright ;-) I hope to see some of them at Dante and will invite them to some wine ...

@UlrikeFischer Lots of wine ...

@UlrikeFischer Might see other people too ;)

@JosephWright ?

@JosephWright for every update until the meeting from now on one?

@UlrikeFischer Er, they are used to me I think (I've said before I'm probably #1 uploader)

Hence if I don't say, they just guess it's me

@JosephWright yes, just joking. I tested the fix, looks fine.

@DavidCarlisle tex.stackexchange.com/a/528365/73

@DavidCarlisle A feature request ;)

@UlrikeFischer -- Do you know anyone with the power to hypnotize?

@barbarabeeton why?

@UlrikeFischer -- Power of suggestion ... "Be gentle with @JosephWright and @DavidCarlisle when they upload to CTAN."

@JosephWright I'm currently taking a look at your commit regarding de-alt, and am not entirely happy with the wording: github.com/latex3/latex3/commit/… sounds like it is not available for the TU engines.

@JosephWright well that's sort of the purpose of the lists, you can't list everything by default:-)

@barbarabeeton oh that is already working.

@Skillmonlikestopanswers.xyz The text says that everything is available in TU ...

@JosephWright yes, it does, well hidden in the paragraph above it.

@JosephWright you could perhaps add an also: available also using pdflatex ...

@UlrikeFischer exactly what I had in mind.