Hi @JosephWright -- can you please check whether this is supposed to be failing in TL2022 or not?

\documentclass{article}

\usepackage{CJK}%字體(Chinese/Japanese/Korean)

\begin{document}
\begin{CJK*}{UTF8}{bsmi}

\MakeUppercase{字體字體}

\end{CJK*}
\end{document}

(I boiled this down from a template at overleaf.com/latex/templates/… that uses revtex-1 and is newly failing for uppercasing section titles.)

@yo' pdfTeX?

@JosephWright yep

@yo' The package is redefining the active chars such that they are not \protected: one can fix of course but really this needs to be addressed by the package authors - I guess I'll report

\documentclass{article}
\usepackage{etoolbox}
\usepackage{CJK}%字體(Chinese/Japanese/Korean)
\begin{document}
\begin{CJK*}{UTF8}{bsmi}
\ExplSyntaxOn
\tl_map_inline:nn{字體字體}{\robustify{#1}}
\ExplSyntaxOff
\MakeUppercase{字體字體}
\end{CJK*}
\end{document}

@JosephWright ah :(

\documentclass{article}
\usepackage{CJK}%字體(Chinese/Japanese/Korean)
\makeatletter
\endlinechar \m@ne
\def\CJK@namepppdef#1{
  \CJK@active\protected\def#1{
    \CJK@@@
    \ifx\protect \@typeset@protect
      \expandafter\expandafter\expandafter\CJK@XXX
       \expandafter\string\expandafter#1
    \else
      \noexpand #1
    \fi}}
\endlinechar `\^^M
\makeatother
\begin{document}
\begin{CJK*}{UTF8}{bsmi}
\MakeUppercase{字體字體}
\end{CJK*}
\end{document}

\documentclass{article}
\usepackage{CJK}%字體(Chinese/Japanese/Korean)
\usepackage{etoolbox}
\makeatletter
\renewcommand*\do[1]{%
  \patchcmd{#1}{\CJK@active\def}{\CJK@active\protected\def}{}{\ERROR}%
}
\docsvlist{\CJK@namedef,\CJK@namepdef,\CJK@nameppdef,\CJK@namepppdef,\CJK@nameppppdef}
\makeatother
\begin{document}
\begin{CJK*}{UTF8}{bsmi}
\MakeUppercase{字體字體}
\end{CJK*}
\end{document}

@yo' ^^^ I could arrange to firstaid CJK using something like that if required

@JosephWright No need to. We'll simply keep the template at TL2021 and then bump it up in TL2023 where hopefully this (together with romannum and babel) will be fixed.

@yo' The roman business is fixed with an expl3 change: I've adjusted an internal loop so the lack-of-termination isn't an issue

If these get fixed reasonaly soon, we might still patch our TL2022 to include them.

@yo' I've reported and offered a patch

@JosephWright cool! If you have an L3 update that helps with the roman stuff, we can probably implement it as well, given it doesn't break anything :)

@yo' Latest l3kernel fixes that, doesn't that I know of break anything else :)

@JosephWright ah ok. If it were possible to isolate this one change (or if you didn't change anything else), it's probably cool and we'll consider it once we have the current image out.

(We really need to ship now!)

@JosephWright btw, just as a side note: Most failures we've seen in templates for the new TL are from things like \MakeUppercase{\Large #1} which seem to fail with an infinite loop.

@yo' Yes, that's not entirely a surprise - it's situations where the font commands are not robust - works fine with the standard classes/font setup

@JosephWright oh right! I was curious about what the people are doing wrong :D I guess we should include that in our blog post

@yo' Try

\documentclass{article}
\begin{document}
\MakeUppercase{\Large Hello world}
\end{document}

which works fine as \Large is robust

@JosephWright right

@yo' I've looked at the issue and I don't have a good solution: places where setting \protect is required are not doable using the expansion approach I've implemented

@JosephWright no worries. I'm just thinking about how to phrase it for our blog post. Currently we have this:

(where I'd like to remove the last sentence in favour of "You don't need to load inputenc at all as it's loaded automatically since 2020 with the [utf8] option)

@yo' Like I said, if you use current l3kernel then the infinite loop is fixed, so there is only the issue of non-robust \Large, etc.

@yo' Agreed

@JosephWright ok, we'll keep it for now but I'll push Eric for yet another test :)

Gosh!

\chapterlineswithprefixformat #1#2#3->\setstretch {1.2}\textupper {#2#3}
#1<-chapter
#2<-{\usekomafont {chapterprefix}{\csname chapterformat\endcsname \setlength {\@tempskipa }{\csname scr@chapter@innerskip\endcsname }\chapterheadmidvskip }}
#3<-\interlinepenalty \@M Introduction Chapter Name\@@par

(with \newcommand\textupper[1]{\MakeUppercase{#1}})

@yo' Yes that sort of thing is tricky

@JosephWright but the solution seems fortunately simple (and the tool for that is there, so it's really a template bug): \chapterlineswithprefixformat #1#2#3->\setstretch {1.2}#2\textupper {#3}

@yo' David is just fixing ucs, so if people really need the additional functionality they will be able to use it. But the second sentence should better be "in a typical utf8 encoded document loading inputenc is not needed at all as utf8 is the default in latex since 2018 anyway".

@UlrikeFischer ok thanks! Still, for the time being, not loading anything is the way, and we'll write it that way in the blog post :)

@JosephWright hmm I made ucs.sty make characters protected but currently then need a hyperref package hook to unprotect them again (@UlrikeFischer)

@DavidCarlisle oh you mean cjk would then need that too?

One issue is that starting yesterday, we had to reallocate people to different projects, so we don't really have much dev capacity for more testing, patching etc. now.

@UlrikeFischer possibly, not checked, or possibly the existing un\protect loop is OK for CJK's utf8 version

sourcefiles={"*.ins","*.dtx","config/*.ucf","data/*.*","utils/*.*"}
flatten=false

@JosephWright is ^^^ supposed to make a data directory in the ctan zip (or how am I supposed to copy data utils and config unchanged to ctan ?

@DavidCarlisle no, doesn't work:

\documentclass{article}
\usepackage{CJK}%字體(Chinese/Japanese/Korean)
\makeatletter
\endlinechar \m@ne
\def\CJK@namepppdef#1{
  \CJK@active\protected\def#1{
    \CJK@@@
    \ifx\protect \@typeset@protect
      \expandafter\expandafter\expandafter\CJK@XXX
       \expandafter\string\expandafter#1
    \else
      \noexpand #1
    \fi}}
\endlinechar `\^^M
\makeatother
\usepackage{hyperref}
\begin{document}
\begin{CJK*}{UTF8}{bsmi}
\section{字體字體}
\MakeUppercase{字體字體}
\end{CJK*}
\end{document}

@JosephWright Please, which version of the kernel has the update to \MakeUppercase? Is it available via tlmgr already?

(found it now, sorry for noise)

Making the ctan build with make is tempting:-)

@yo' :)

@yo' Quite understand: we (team) have a rolling release other than during the TL freeze, so our situation is different to yours

@JosephWright and we also have 1.5M MAU :-)

@yo' MAU?

@JosephWright monhtly active users

@yo' Oooh: are you supposed to tell me that? (Seriously - should I delete?)

@yo' Logically, the team have those plus everyone else of course ;)

@JosephWright Marine Amphibious Unit what Tom said :)

@JosephWright I don't think this is much of a secret...

@yo' OK: I've only seen total registered users in for example the Overleaf blog

@JosephWright yep, that one is much more public, but still this is not a real secret.

Not anymore :)

@JosephWright we corporate ducks are very good at keeping secrets

@yo' @PauloCereda You don't like numbers in environment names, do you?

@samcarter new or old editor?

@yo' My settings say "You are enrolled in the Beta Program" or how can I find out of it is the new or old editor?

@samcarter if you see both "Source" and "Source (Legacy)" then you're on the new editor. As a Beta participant you should be there certainly.

However, to answer your question: We do have some issues with the new parser. We had to write it from scratch as there was nothing that would provide the neeed functionality. And the parser seems to have some glitches...

@yo' Ah, I see both and Source is selected

@samcarter yeah, so it's the new one, and this is a bug :-/

@yo' In case you'd like the code to reproduce: tex.stackexchange.com/a/653358/36296

(in legacy it is correctly coloured)

@UlrikeFischer l3build ctan just about works now (I get two readme files, not sure why) but otherwise packages the right things, I think.

@DavidCarlisle this could more or less work, one only need to handle the substitutions for spaces etc better (I didn't try yet with the protected ucs:

\documentclass{article}
\usepackage{CJK}
\makeatletter
\endlinechar \m@ne
\def\CJK@namepppdef#1{
  \CJK@active\protected\def#1{
    \CJK@@@
    \ifx\protect \@typeset@protect
      \expandafter\expandafter\expandafter\CJK@XXX
       \expandafter\string\expandafter#1
    \else
      \noexpand #1
    \fi}}
\endlinechar `\^^M
\makeatother
\usepackage{hyperref}

\ExplSyntaxOn\makeatletter
\cs_generate_variant:Nn\str_gset_convert:Nnnn{NVnn}
\def\HyPsd@CJKhook{}
\def\HyPsd@expand@utfvii{}
\def\HyPsd@ConvertToUnicode#1{}

@UlrikeFischer ImDeutschengibteskeineLeerzeichen

@DavidCarlisle and in english no accented chars, we should combine both.

@UlrikeFischer then you get Chinese

@DavidCarlisle, @UlrikeFischer, @MarcelKrüger I think github.com/latex3/latex3/pull/1120 is basically 'ready' (the only thing not covered is extended pictograms): would welcome any thoughts

@UlrikeFischer I see I need to cover T5

@UlrikeFischer Fits with my supposition on the team list (that we likely need to do all of Unicode even with pdfTeX)

@JosephWright ;-).

@UlrikeFischer You want to answer or should I?

@JosephWright Better you I have no idea what exactly needs to be done, I only made a smaller example (and tested that it works in texlive 2021).

@UlrikeFischer Cool

@JosephWright @UlrikeFischer github.com/LaTeX-Package-Repositories/ucs/commit/…

@DavidCarlisle there is a bee in it ;-)

@UlrikeFischer gone

@DavidCarlisle I see a few more typos. I will correct them ok?

@UlrikeFischer yes fine, thanks

@DavidCarlisle is Upper/Lower by design in uppercase?

@UlrikeFischer apart from typos do you think we need to say something different?

@UlrikeFischer er probably not

@DavidCarlisle perhaps mention that it tries to work with hyperref too?

@UlrikeFischer I guess an update to cover T5 is in order (@DavidCarlisle)

@UlrikeFischer yes I was wondering about mentioning hyperref, do you want to add something in the second item?

@DavidCarlisle we tried to make it work with hyperref, blame Ulrike if something fails?

@UlrikeFischer I did consider using your name instead of mine in \author just in case of blame, (or we could use latex-team?)

The BachoTeX meeting is cancelled.

@mickep I'd seen :(

@JosephWright Did you plan to go?

@UlrikeFischer It should have been \tikz\bee; :)

@mickep No, never been

@UlrikeFischer, @DavidCarlisle Case changing strikes again: tex.stackexchange.com/questions/653491/…

@JosephWright I just wanted to ask you if you enjoy \MakeTitlecase ;-)

@UlrikeFischer Edited :)

@JosephWright blame @yo' and @PauloCereda for not providing \MakeTitlecase ?

@DavidCarlisle :)

@DavidCarlisle I wonder when people will notice that \text_map_inline:nn is a way to solve the listings UTF-8 issue ...

@DavidCarlisle wait a day?

(with fingers crossed nothing goes wrong with the deploy!)