@UlrikeFischer Danke schön

@Cattleya Well I just like this friendly chat. :D I am interested in programming overall, but let's see if you can pull me into LaTeX (lol)! TNB stands for The Nineteenth Byte, the main room for Code Golf Stack Exchange.

We can post small pictures again, it seems: meta.stackexchange.com/a/399292/252737

@Rmano yeah!

I found by chance that Unicode works with PDFLaTeX when the upper-half of 8-bit chars is set to active but fails if not. Where can I find some documentation about this? Is there more to it than just setting the upper-half of the 8-bit range to active?

@JasperHabicht inputenc, but nowadays built into LaTeX

@JosephWright I see

@JasperHabicht @DavidCarlisle's the man to talk to - it's his idea from the early 1990s :)

@JasperHabicht Unicode (UTF-8) works by having the upper half of the 8-bit range as the bytes of multi-byte codepoints, so only the classical ASCII range uses one byte-per-codepoint

@JasperHabicht The need to make things active is just the pdfTeX implementation of UTF-8, really

@JosephWright The point is, in order to allow things like \t in a JSON file to be parsed as something (even if another character follows), I somehow need to change catcodes. I was reading the part in interfaces3 on cctab where \c_document_cctab is defined and looked that up in the code. Now, I am wondering whether I do thnigs properly ... e_e

@JosephWright Yes, this is why \c_document_cctab only does this setting for PDFTeX, not for other compilers

@JasperHabicht I know - I wrote that code :)

I was wondering whether having set these to active might be a bad idea ... So, maybe I should at least provide some switch or something (or test for the current encoding). But if it is fine, then ... well, no switch is needed.

@JosephWright I know, I just wanted you to know that I know =)

@JasperHabicht You only need actives for typesetting - if you are dealing with programmatic data, it really doesn't matter

@JosephWright Oh, right! ... So, maybe I don't need that at all ... hm, I will look more into this. Thanks!

@JasperHabicht I don't follow the issue with tabs ...

@JosephWright You can have strings like "foo\tbar" in JSON. But if you read that in with TeX, it will complain, as it finds \tbar which is undefined.

Other way could be to search for these things and add spaces ... would require regex, I assume ...

@JasperHabicht Well that's a different issue: likely best to make \ active in the reading context as a command taking one argument

\begingroup
\catcode`\/=0 %
/catcode`/\=/active
/xdef\#1{/detokenize{\}/noexpand/detokenize{#1}}
/endgroup

@JosephWright Okay, I will look into this approach.

I currently set t to other for parsing and later change it back to letter (after \t etc. being a control symbol at this point have been replaced)

@JasperHabicht Depending on what you want to do, you could then arrange for \<x> to expand to whatever the real char is, if you set up a simple lookup approach - really depends what makes sense in the context

@JasperHabicht Also works well - you don't really need any 'letters' here - could just detokenize the lot

@JosephWright Yes, I think I understand now much better how things work. First, I thought detokenization breaks unicode support (it does, but now I know why =D), so I switched to not doing that. But then, things developed in a way that I need to do a rescan before typesetting anyways ...

@JasperHabicht making them active just allows them to be defined, you then need to define them to do something. It is not just unicode though latex2e has always been this way, as soon as you load inputenc so supporting 8 bit code pages such as latin 1 also requires all the top half to be active.

@DavidCarlisle So, this means, it won't hurt for them to be active? In the inputenc it says: "The inputenc package makes the upper 8-bit characters active and assigns to all of them an error message. It then waits for the input encoding definitions to change this set-up." ...

@JasperHabicht surely you shouldn't be reading json with a catcode 0 \ shouldn't you be reading in verbatim catcodes so str type in l3 ?

@DavidCarlisle Yes, that is what I originally did. This broke Unicode, so I switched to tl. This was probably a stupid decision, since if I do a rescan, everything will probably be fine again, right?

@JasperHabicht since 2018 inputenc is essentially pre-loaded and set to utf-8 so they are all active and if you chaneg that things will break (for typesetting) in a local environment it doesn't really matter much but I would have assumed that you want all the 8 bit chars active with teh defaultinputenc definitions and everything else catcode 12

@JasperHabicht same as l3sys-query we use tl not str and then just make the lower half catcode 12 (which is a sort f approximation to Unicode str for 8 bit engines (@JosephWright i wonder if we should standardise or document that somewhere)

@DavidCarlisle Yes, kind of. Except for curly braces (except I revert my code to a very early state or rewrite how objects are parsed)

@DavidCarlisle Yes, this is essentially what I am doing now: I set up a new cctab that sets more or less everything to 12, only braces to 1, 2 and upper half of 8-bit to active

@JasperHabicht how do you cope with json strings like "aaa}{bbb" if the braces are not catcode 12?

@DavidCarlisle good point, I think I currently don't ... So, maybe I really will rewrite the code for parsing objects

@DavidCarlisle Thanks for the input! I think with this, I can now rewrite parts of the code to make it more robust and also somehow cleaner. Since I did not play around much with catcodes, this is somehow work-in-progress and learning by doing (and therefore still a beta version =P )

@DavidCarlisle I said something in the code docs, but we could perhaps extend

@mickep for microtype, i have used the kerning options a few times, ie sec. 5.4 in microtype manual. actually the feature matrix for pdfluatex is not as bad as i remembered, though i haven't tested yet

\documentclass[%crop,
 margin=0pt,
tikz,
%convert={outext=.svg,
%command=\unexpanded{dvisvgm \jobname}},
%%multi=true,
]{standalone}
\usepackage{lmodern}
\usepackage{tikz}
\usetikzlibrary{decorations.markings}
\usetikzlibrary{backgrounds}

%  \usetikzlibrary{patterns.meta}
%  \usetikzlibrary{patterns}
\usepackage{svg}
%\usepackage{txfonts}
%\usepackage{newtxtext,newtxmath}


% Card-Sizes (default)=======================
\pgfmathsetlengthmacro\cardlw{20px}% card line width

\pgfmathsetlengthmacro\framelw{10px}%

@tsj Interesting. I thought everything was there also for luatex...

paulo at frankfurt in ~
➜ uname -a
Linux frankfurt 6.8.7-300.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Apr 17 19:21:08 UTC 2024 x86_64 GNU/Linux

@samcarter ^^ ooh :)

paulo at frankfurt in ~
➜ distro
Name: Fedora Linux 40 (Workstation Edition)
Version: 40
Codename:

@PauloCereda oooooooooooooooooooooooooooooooooooooooh

@samcarter <3

ooh hai

@PauloCereda Oh, dodos like triangles.

@mickep ooh but these are scary triangles

@PauloCereda Now I feel obtuse...

@mickep oh no

also triangle puns :)

@PauloCereda Indeed, we only need David to come out of his corner to join...

@PauloCereda also, BachoTeX in a week!

@mickep Have a lot of fun!

On the Dante mailing list it was just mentioned that DEK comes to Venice this autumn grafematik2024.sciencesconf.org

@samcarter Thanks! I'm sure it will be great! Very nice place to be.

@mickep I hope you will keep us updated on all the fun stuff happing there :)

@samcarter Oh, live reports in the chat. :)

@mickep yeah :)

@samcarter Nice!!

@DavidCarlisle How do you feel about decent - barely loose - almost loose - loose - very loose? :))

@PauloCereda next time, paste the XKCD link so we can see the mouse-over text!

@Skillmon But \cs_new:cpn { __jsonparse_parse_ \c_left_brace_str :w } #1 \q_stop {...} won't work, would it? I am either doing things wrong or it just can't work ...

@JasperHabicht show more code, just from this one line I'd say it should work.

@Skillmon okay, then ... I'll try again ...

So, I already have this working:

\str_set:Nn \l_tmpa_tl { [ abc ] }

\cs_new:cpn { __jsonparse_parsex_ [ :w } #1 \q_stop {
  \exp_last_unbraced:Ne
    \__jsonparse_parsex_array_begin:w #1 \q_stop
}

\cs_new:Npn \__jsonparse_parsex_array_begin:w [ #1 \q_stop {
  \tl_analysis_log:n {#1}
}

\cs_if_exist_use:cTF { __jsonparse_parsex_ \str_head_ignore_spaces:e { \l_tmpa_tl } :w } {
  \l_tmpa_tl \q_stop
} { F }

And I thought, I could do in analogy:

\str_set:Nn \l_tmpb_tl { { abc } }

\cs_new:cpn { __jsonparse_parsex_ \c_left_brace_str :w } #1 \q_stop {
  \exp_last_unbraced:Ne
    \__jsonparse_parsex_object_begin:w #1 \q_stop
}

\cs_new:Npn \__jsonparse_parsex_object_begin:w \c_left_brace_str #1 \q_stop {
  \tl_analysis_log:n {#1}
}

\cs_if_exist_use:cTF { __jsonparse_parsex_ \str_head_ignore_spaces:e { \l_tmpb_tl } :w } {
  \l_tmpb_tl \q_stop
} { F }

But I get a loop which prevents me from getting closer to the issue

I think that this somehow results in unbalanced braces ...

maybe I am just too tired today ... =D

Hm, or maybe the issue is with the \c_left_brace_str in the p part?

@JasperHabicht \cs_new:Npn \__jsonparse_parsex_object_begin:w \c_left_brace_str #1 \q_stop defines a command with argument delimited by \c_left_brace_str not delimited by a catcode 12 {

try \use:e{\exp_not:N\cs_new:Npn \exp_not:N\__jsonparse_parsex_object_begin:w \c_left_brace_str} #1 \q_stop @JasperHabicht

@JasperHabicht what @DavidCarlisle said, use \exp_last_unbraced:NNo \\cs_new:Npn \__jsonparse_object_begin:w \c_left_brace_str #1 \q_stop { ... } instead.

@Skillmon Yes, that's it!

Thanks! Cool. It works again .. now with catcode 12 (mostly). Now part two of the rewrite comes: the escape sequences ... but this is something for tomorrow (so in about a minute)

@JasperHabicht now that it is tomorrow: How's it going?

@Skillmon =D

@Skillmon your clock is wrong

@DavidCarlisle nah, I'm just not a kobold witch.

@DavidCarlisle What does your clock say? :)

@JosephWright I had the idea of somewhat porting expkv-cs into ltcmd by providing an argument processor \SplitKeyval{<key1>=<default1>, ...}. Worth pursuing or should I drop it?

@yo' 23:06 obviously!

@DavidCarlisle That also looks non-proper. I'm sure the astronomical clock at Greenwich shows 22:06 (now :08) ;)

@yo' we make allowances for summer

@DavidCarlisle traitors

@yo' it's (still) st george's day, you should be nice to English people

@DavidCarlisle Isn't St Georges 24th, so tomorrow for you?

Ah dammit, it's only Czech who celebrate it on the 24th, that's weird.

@yo' dates and time wrong, there is really no hope for you