@NicolaTalbot, @DavidCarlisle, @egreg: OK. Thanks for answering.

Advertisement for those who love Mathematica but have limited budget: msgcache.wolfram.com/holiday/2013/08/2013082890/b

Hey! I've been able to review a first post! ;-)

@egreg :-)

@DavidCarlisle Do you know where a BITNET/Internet gateway can be reached? IIRC, scrambling characters was more a JANET problem.

@egreg well i was on janet and routinely got characters dropped, recollection of what the outside world was doing is vague:-)

@DavidCarlisle: you can harvest rep here:

hi there :)

any beamer specialists here?

oh never mind, looks like I fixed it :)

The latest answer to the following question is spam. Please vote.

@Kurt good call!

@tohecz :-)

Always good to see bugs go unreported for 20 years.....

@DavidCarlisle And a very important one! ;-)

@egreg I blame @JosephWright

@Kurt Zapped

@DavidCarlisle He was just a teenager at the time, completely unaware of the existence of LaTeX. But it's his fault nonetheless, he should have caught it.

@egreg exactly

@JosephWright I've found a bug in \tl_set_rescan:Nn: trailing spaces are gobbled. :(

@egreg Wouldn't surprise me: I don't really like the whole rescan business

@egreg I will of course take a look

@SamWhited Depends what you want to do, of course

@egreg Can you give me a demo?

@JosephWright Here it is

No matter how many space tokens you have at the end, they get gobbled.

\begingroup\edef\x{\endgroup
 \noexpand\rescan\noexpand\x{}{ab\space\space\space}}\x

X\x X

I got bitten trying a patch with regexpatch: a macro with a trailing space in the replacement text doesn't pass the \checkpatchable test because of this gobbling.

@egreg OK, so this is a fundamental part of \scantokens, then?

@JosephWright Indeed I'm afraid it is a problem with \scantokens

And it can be explained: e-TeX treats it as a line in an input file, so spaces at the end are removed. :(

@egreg presumably you just need to put some token at the end, scan then remove

@JosephWright So it's not a bug, but a feature

@DavidCarlisle Yes, that's a good idea.

@DavidCarlisle We already have that, though

@egreg Back with my 'we should drop this from expl3' case :-)

@JosephWright Oh, no! Don't!

@egreg \scantokens really doesn't work well, I've found

@JosephWright A good idea with a bad implementation

@egreg Yes, but that doesn't mean we should keep it in expl3: there seem to be better ways of tackling issues

Probably LuaTeX's \scantextokens is good.

@JosephWright can't be right though as that would preserve spaces at the end in principle.

No, also \luatexscantextokens gobbles trailing spaces.

@DavidCarlisle Am working on it!

@egreg Just fixes the fact that you have the EOF business, I think

OK, something like

\cs_set_protected:Npn \__tl_set_rescan:NNnn #1#2#3#4
  {
    \group_begin:
      \exp_args:No \etex_everyeof:D { \c__tl_rescan_marker_tl \exp_not:N }
      \tex_endlinechar:D \c_minus_one
      \tex_newlinechar:D \c_minus_one
      #3
      \use:x
        {
          \cs_set:Npn \exp_not:N \__tl_rescan:w ####1 @ \c__tl_rescan_marker_tl
            { \exp_not:N \exp_not:o { ####1 } }
        }
      \use:x
        {
          \group_end:
          #1 \exp_not:N #2
            {
              \exp_after:wN \__tl_rescan:w

seems to do the trick: add an arbitrary token to the end of the input, set up the 'clean up' function to remove it, tidy up

Have to check the other functions too

I'll look to sort this later today in the SVN

CTAN update needed once fixed?

@JosephWright after you've fixed \renewcommand:-)

@DavidCarlisle I never commit to LaTeX2e ;-)

@JosephWright I get ! File ended while scanning use of \__tl_rescan:w

Really?

My test file, verbatim:

\documentclass{article}
\usepackage{expl3,trace}
\ExplSyntaxOn
\cs_set_protected:Npn \__tl_set_rescan:NNnn #1#2#3#4
  {
    \group_begin:
      \exp_args:No \etex_everyeof:D { \c__tl_rescan_marker_tl \exp_not:N }
      \tex_endlinechar:D \c_minus_one
      \tex_newlinechar:D \c_minus_one
      #3
      \use:x
        {
          \cs_set:Npn \exp_not:N \__tl_rescan:w ####1 @ \c__tl_rescan_marker_tl
            { \exp_not:N \exp_not:o { ####1 } }
        }
      \use:x
        {
          \group_end:

Issue logged as github.com/latex3/svn-mirror/issues/154

@JosephWright @ can be a letter when the rescanning takes place and it is most of the times a patch is needed.

@egreg Yes, I know :-) I need to make some adjustments to the above to make sure that whatever token we pick has the same catcode in \__tl_rescan:w as when used as an end-marker.

@JosephWright :)

@JosephWright use a quark not a catcoded token oops I guess not:(

@DavidCarlisle It is quite tricky, as something like \etex_scantokens:D { @ } \c__tl_rescan_marker_tl fails (with \everyeof{\noexpand}, at least)

@JosephWright sometimes I wonder why we treat the tex engine with so much respect. In any normal environment we'd just fix the underlying code.

@egreg I can sort it if I use the end marker explicitly, but using it as a tl is proving to be more difficult. Will try harder after work.

@DavidCarlisle We can't ;-)

Or at least I certainly can't, and it wouldn't get accepted by the distributors anyway

@JosephWright Don't worry too much. I can always add a test with a trailing space token before giving up and say a macro is not patchable.

@JosephWright we could ask taco to fix it in luatex and then just use that.

@egreg I can fix it, as I say, but there is still an issue with for example \rescan\x{\catcode\@ = 15}{ab }`, whatever token you decide to use as an end marker

@DavidCarlisle You convince Frank that's a good plan, then :-)

@JosephWright Users are advised that a special such and such token with category code such and such is added, so it can't be used when rescanning.

@JosephWright If Khaled does as he hinted in his interview and brings xetex font handling to luatex, I think we should seriously consider it.

Not completely general, but I wouldn't bother: there are other limitations and a category code 8 @ seems not so useful.

@DavidCarlisle Then we have to start work again

@DavidCarlisle This would be a great advancement.

@egreg Certainly

@JosephWright that's life. But not being unicode based is going to be increasingly an embarrassment the further we get into 21st C.

@DavidCarlisle I'm sure you've had this discussion with Frank before

@DavidCarlisle The question then though is why not go all the way and adopt something like Jonathan Fine's suggestion of not doing the programming in the typesetter at all

@JosephWright well not so often (and it was nts or omega, which isn't quite the same:-)

@DavidCarlisle Possibly: I've certainly suggested that a reasonable position would be not be as accommodating of using pdfTeX with varied input encodings as is done by LaTeX2e

@JosephWright because you can make lualatex process the vast majority of latex documents. (and the model of macro expansion within the document isn't necessarily all wrong, just because implementing it on a 7bit engine with grudging extensions for 8bit encodings, may be)

@DavidCarlisle That's different to starting from scratch in Lua, though

@JosephWright yes context isn't all written in lua either.

@DavidCarlisle There is I'm afraid no way that a LaTeX3 format will process any existing LaTeX2e documents, so that doesn't really help

@DavidCarlisle Not yet

@DavidCarlisle Also, Hans didn't start from scratch with a more-or-less finished LuaTeX

@JosephWright yes I'd expect all packages to break but there probably isn't much need to break a clean latex document body.

@JosephWright once you break the model of freezing software forever then it is never finished.

@DavidCarlisle I mean finished as in 'production usable'

@DavidCarlisle They started rewriting ConTeXt partly to help write LuaTeX

@DavidCarlisle Most real documents use package code in the body, and there are some LaTeX2e things I would like to see removed or at least deprecated formally

Part of the question there then is how XML-like you go (for example, are multiple optional arguments still acceptable or should everything be keyval)

@JosephWright sure

@JosephWright probably both at least in transition

@DavidCarlisle Yes, my guess too

Anyway, at present XeTeX doesn't have Lua and Lua doesn't have the XeTeX font loader, so we are not quite in that position

@egreg What's your take, then, on the scantokens business? Do I document that trailing spaces are removed or fix with the limitation that whatever I pick as a trailing char can then only be rescanned with a limited number of catcodes?

@JosephWright Probably both options. ;-) But the latter seems better. Maybe you can ask on the list.

@egreg Yes, I think that's best. I'll present the options :-)

@egreg I'm going to ask the team as a whole first: Bruno might have a cunning plan

@JosephWright This is another good idea. In my application I can add myself a string, since I can assume that numbers have category code 12. So I can add 0 and remove it with a \regex_replace_once:nnN, since I know it's there.

And, actually, with regex_replace_once:nnN it doesn't matter what catcode it has. ;-)

I think we should add this to our Text building blocks:

...or maybe not...

@JosephWright My plan seems working. :-)

Everybody left....

Some progress on my "CTAN history" project. Still in beta, but you get the idea. URL is still temporary and not 100% uptime: speedata.de:8089

Close as OT (see self-answer)

@Werner rather not I'd say. It would suggest people to over-use it

How do I mark a straight line at regular intervals in tikz?

@kan something like \draw (0,0) -- (5,5) \foreach \x in {0,...,10}{node[pos=\x/10] {A\x}}

@tohecz Hmm, I need something like: 0, 1/3, 2/3, 1, 4/3, ..., 2

So, how do I do that?

@kan then put it instead of 0,...,10

@tohecz No, so I should put all the points right? Isn't there intelligent way to make the system understand the pattern?

@kan I dunno. Moreover, I'm not able to make it understand fractions...

this is all I have, sorry, I'm certainly not a TikZ expert:

\documentclass{article}

\usepackage{tikz}
\usetikzlibrary{calc}

\begin{document}

\begin{tikzpicture}
\draw (0,0) -- (1,1) \foreach \x in {0,0.33333,0.5}{ node[pos=\x] {A\x}};
\end{tikzpicture}

\end{document}

@tohecz Hah! this is exactly my problem!

Anyway, I looked up the manual and there is something that works!

Hmm! Now, another question: how do I define a path?

\def\mypath{<stuff-here>}

What is the syntax for <stuff-here>?

@kan Not surprisingly, this works

\documentclass{article}
\usepackage{tikz}
\def\mypath{(0,0) -- (1,1)}

\begin{document}
\begin{tikzpicture}
\draw \mypath ;
\end{tikzpicture}
\end{document}

@egreg Hey, but for more complicated things, like a concave arc? ;-)

@kan I don't understand.

@egreg OK! that was silly! But, I would like to join three paths.

Like I have a line, an arc, followed by a line. How would I do that?

it seems like a lot of this stuff people do by defining things absolutely on the page but i've got something like 8 nodes already all defined and i'd rather not redo everything.

@egreg thanks fixed (I've never actually used \widowpenalties:-)

@DavidCarlisle I did! And so discovered the trick. ;-)

@JonBeardsley node [above] ?

@egreg you mean like you test answers? wow:-)

oh yeah, thanks! and if i use node [above, fill=white] i get the break in the arrow

thanks so much!

@DavidCarlisle No, I used \widowpenalties in the past.

tex.stackexchange.com/search?q=widowpenalties+user:4427

@DavidCarlisle Actually the search engine is ‘smart’ and so it finds also \widowpenalty.

@egreg is it possible to search for \widowpenalties but not widowpenalties, ?

@DavidCarlisle If I type "\widowpenalties" only one is found.

ah with quotes, that cuts things down t just 7 hits:-)