4 hours later…
12:35 PM
@PauloCereda and among the replies: twitter.com/Marty_MCA/status/1555683502850465793
2

2 hours later…
2:47 PM
How feasible is it to look ahead for an \end{enva}\end{envb} pattern? (as opposed to \end{enva} something \end{envb})?

3:15 PM
@DavidPurton who's doing the looking? \endenva looking for \end{envb} or \begin{enva} looking for \end{enva}\end{envb} ?

Actually I want to do it just before an \endlist inside \endenva
\endlist is the last thing inside \endenva
I want encourage a page break in one case and prevent it in another (perhaps by setting \@endparpenalty)

@DavidPurton \def\endenva{....\@ifnextchar\end{check for envb}{}}

But it seems hard to know if an \end{envb} is what is coming up next. or if there's another paragraph in between

@DavidPurton \@ifnextchar would skip white space but not a blank line (\par)

hmmm. I think I need to ask a proper question. I have trouble getting this to work even after the \endlist. I think because \end{enva} just calls \@@_end_enva: And there's too many tokens that get in the way.
Actually, it doesn't even work for me if I put the \@ifnextchar\end{}{} right at the end in the environment definition itself.

3:33 PM
@DavidPurton ah you're so modern:-) you could put the lookahead in the end after hook so it was in the tokens you are seeing not before them, alternatively (and probably better) don't do source code lookup (which will break with any user definitions hiding the markup) get the outer end to add or not penalties depnding what is on the main vertical list at that point

@DavidCarlisle Ah, you lost me :(. The problem with just using penalties is the list. enva is a list and by default lists encourage breaks after them. That's fine for me most of the time. Except if enva is the last thing before the main envb ends.
In the latter case I want to insert some extra text that definitely must be on the same page as previous text.
Can't be helped. I'll keep thinking and maybe ask a question on main site

@DavidPurton yes but look at how \addpenalty works, it inspects the skips just added, if you had a distinguished skip eg 2sp as a marker you would know whre you are and do what needs doing

ooo. I will look

3:50 PM
@DavidPurton actually I'm just looking at the same problem as I need for the tagging. I have to insert a structure end in the one case but not the other. I'm considering something with \@doendpar and a look back from the end of the outer environment.
@DavidCarlisle that is actually a good idea. I should check that.

4:21 PM
@DavidCarlisle grumph. The "accidental ascii" means that a percentchar is converted into the string \376\377\000%, which is okay if used directly, but has its drawback when written to the out file and then read back ;-(

4:32 PM
@UlrikeFischer exactly why UTF-8 was invented, ASCII Only occurs in UTF-8 when you want that character:-)
@UlrikeFischer well either lean on @JosephWright to give you % in octal, or make sure there are no comments in the .out and locally set cacode % to 12?
@UlrikeFischer I guess mis-matched { would mess you up as well?

4:45 PM
@DavidCarlisle it would if hyperref wouldn't execute \HyPsd@SanitizeOut@BraceLeft\Hy@tempa before writing to the file. Which means one could add more stuff there. But I would prefer that \str_convert does it right from the start.

@UlrikeFischer yes but what is right? if you start from }%{ then converting that to \str_whatever will produce same in utf8 and same plus null bytes for utf-16, neither safe for reading back from a file

@DavidCarlisle I want to get \376\377\000\173 (or whatever number is right). If I set this conditional to always return false then it works:
\prg_set_conditional:Npnn \__str_if_escape_string:N #1 { TF }
{
\prg_return_false:
}

@UlrikeFischer it would be easy to always use octal but then blub gets to be 4 times longer, not sure how easy it is to get just certain ascii characters in octal, a few more ifs somewhere....

5:04 PM
@DavidCarlisle one could change the range. currently the test leaves everything from "21 to "7E, with "27 to "7A one would leave one most of the candidates for troubles.

@UlrikeFischer you looked at the code? shocking

@DavidCarlisle the documentation is so short.

@UlrikeFischer sounds like a good plan though

@DavidCarlisle I think that would make sense. Even if I could handle bookmarks, \pdfstringdef is used in other places too and there is really no need to create potentially problematic output with % and & and #. (And if ^ or _ make problems one could remove them too later.)

@DavidCarlisle May I again trouble you with a question regarding this \pgfkeys stuff?

5:18 PM
@JasperHabicht you can ask but don't be fooled by my gold badge, I know nothing:-)

@DavidCarlisle So, you essentially meant that using any pgf macros globally, this is bad because I (as I am not the PGF maintainer) don't really know what these macros do. This would, of course, be true for any other macro that I don't really know the defintion of, right?
So, if I create a PGF key that uses .code to \gdef a macro \mykey to story my value in ... this would be save.

@JasperHabicht You can count the places where it is safe to use \global before a high level setting on the fingures of one hand, and the places where it is safe to use \globaldefs ar even more scarce. So if it works in that instance it works, but only by luck.

I see. Then I think I got it. It is hard, anyways to understand in which context this is needed by the questioner. Thanks for clarification!

@JasperHabicht but actually the real answer for that question is as Joseph implied in comments: "Do not do that" minipage is by design a scope for definitions, if you want the definition to be visible outside the minipage, do not make it inside the minipage.

@DavidCarlisle yes, this is actually it. it does not make sense to define something globally inside a scope already from the perspective of good coding

5:28 PM
@JasperHabicht Sometimes the right answer is not the answer the questioner wants to hear:-)

@DavidCarlisle indeed. so i leave my answer like it is, although I am not too happy with it. but in the end, it says "do not do it" =D

5:40 PM
lualatex doesn't like the replacement character ;-(
\documentclass{article}
\begin{document}
\Uchar"FFFD
\end{document}
@DavidCarlisle ^^^ "String contains an invalid utf-8 sequence."
:61778097 well actually I used \char_generate:nn (through the char command from yesterday.)

@UlrikeFischer works in xelatex

@DavidCarlisle That's not really a consolation :-(

@UlrikeFischer ^^^^fffd works
@UlrikeFischer so \Uchar should be fixable in Lua

@DavidCarlisle I trust @MarcelKrüger for find a work-around but I guess one should report it anyway?

@UlrikeFischer I have a feeling it's known but yes
@UlrikeFischer I see if you scroll past you get Missing character: There is no � (U+FFFD) in font [lmroman10-regular]:+tlig;! so it does work, despite the error message
@UlrikeFischer he'll just say "use xetex"
@UlrikeFischer "feeling it's known...."
3

lulatex doesn't seem to accept U+FFFD even when correctly encoded as UTF-8, which seems wrong but anyway you can remove it using the input buffer callback, here changing it to ? \documentclass{article} \begin{document} \directlua{ function zzz(buff) return string.gsub(buff,"^^^^fffd","?")...

6:03 PM
@DavidCarlisle ;-)

6:21 PM
\directlua{
local func = luatexbase.new_luafunction'Uchar'
lua.get_functions_table()[func] = function()
local ch = token.scan_int()
token.put_next(token.new(ch, ch == 32 and 10 or 12))
end
token.set_lua('Uchar', func)
}

6:55 PM
@MarcelKrüger yes you can fix \Uchar from lua although fixing U+FFFD in tex text input looks harder without fixing the C sources?

7:11 PM
@UlrikeFischer it's useful to have the C source to hand:
(/c/tmp/cc532.tex
! Blame Ulrike.
l.4
�
?

@DavidCarlisle I'm famous!