It seems inputlineno has very little to do with the source code line and I suppose that there exists no equivalent for source code line values. Kinda sad, makes certain things impossible to do without resorting to very ugly hacks. Could a macro be used that wraps the entire document and figures out the source code lines(counts the newlines) then passes the block back to tex? E.g., suppose you were given the block of code you have above, could you write a robust true line(ending in newline or whatever) counter macro? Then a \sourcelineno could potentially be created.

These things are very simple to do with a normal programming language. Surely the TeX can do it too? If X is the token stream then when \sourcelineno is encountered it simply has to know how many characters has been processed from the start and how man characters up to the nth newline. (then it is a simple matter to get the source code line... arg_k newLine[k-1] <= currCharIndex < newLine[k]. newLine[k] is simply the count of the num of characters up to the kth line. (may ways to do it of course...)

@AbstractDissonance You're forgetting that TeX breaks pages asynchronously with paragraphs. A page break mostly happens when a paragraph has already entirely built in memory, so when the output routine is called, \inputlineno has already been set to the line number where the paragraph ended. Other programming languages don't typeset paragraphs and pages, so you're comparing apples and oranges.

@AbstractDissonance inputlineno is all about the source code line. Line 85 has been read and it is the blank line on that line that executes \par that causes the page breaker to be output. Since you hooked the logging in to the output routine, what else would you expect? If you added \typeout{\the\inputlineno} on every line of text you would see each line being processed before the paragraph was broken in to lines.

@AbstractDissonance I extended the answer here to show \inputlineno reporting line 15

@DavidCarlisle I'm sorry but the logic makes no sense to me. I realize that tex processes things a bit different but in standard programming things work differently. AfterEveryPage is a macro(a function) called after every page has been "created" by TeX, right? That is, it processes the .tex input file and forms the output(which is a pdf in my case) and for each page in the pdf that is generated, AfterEveryPage is executed. This way we can add watermarks or other things to the generated pages after they have been generated. Correct so far or not?

I'll assume I'm right to continue to the next issue. \thepage is a macro that returns the current page that has been generated. We could use it, for example, along with AfterEveryPage to number the pages. Right? Ok, if that's correct \inputlineno returns the line number for the input source code? If so, by the same logic as \thepage, we could have a macro AfterEveryInputLine that would be called after each input line has been procssed, and say, turn .'s in in to !. (it may not be robust but in some cases it should work) Is that right? In fact with such a macro we could apply it to your example

We don't have a macro to do that of course. But I'm actually only interested in mapping the line of a source code to the page that it appears(if it does) in the pdf. AfterEveryLine would allow that to work but all we have is AfterEvery page. Now, the way I see it is that somewhere in the example you gave there are "imaginary" \newpage's that are inserted because the tex code has went beyond the page. Those "imaginary" \newpages could be called, like you have done with typeout directly. But we don't want to do it have to manually find the spot where the break is by comparing it with the pdf.

So, we use AfterEveryPage to find it for us(that is it's purpose) and stick the typout in that macro. BUT when we do this we do not get the same results as if we manually did it(Just like how your AAA typeout returns 15 but AfterEveryPage's version(the same purpose) is much different). This is illogical to me in standard programming interpretation because it means that the macro AfterEveryPage doesn't expand \thepage and \inputlineno the same way as the AAA version(even though they are the same code). I realize that macros in tex behave different than typical computer procedures.

and I realize that macros have the ability to alter the context from what one interprets and this gives tex some power and flexibility that it wouldn't otherwise have. OTH, it makes certain simple things difficult and "tricks" or complex methods must be used. This is fine, I don't mind that, but regardless of how tex works the results must be interpretable in a logical manner. One can't expect another to know everything about the tex architecture to realize that basic logical reasoning(e.g., implication, substitution principles, etc) cannot be used as normal.

I do not believe that you have explained sufficiently to me why the two cases are different. My mind is seeing \AddEverypageHook{\typeout{Page = \thepage, Line = \the\inputlineno}} and \typeout{AAA:Page = \thepage, Line = \the\inputlineno} should function identically except that AddEverypageHook should be automatically executing the macro(inserting it in to the stream as if I typed it manually, as a function call would work in C/C++/C#/D/F#/php/perl/python/asm/JS/etc/etc/etc) at the end of each page in the output.

This is obviously not the case and I don't understand why except that in tex, the macro is effectively not substituted "inline". I have not yet figured out what it actually does. It seems that macros act as state machines and they can be nested. The outer state for some things(such as inputlineno) is not the same as the inner state. I can see that this might be ok, but then the question is how to get the outer line number. I do not believe this is the problem though and I have ideas but nothing conclusive. I'll shut up now. Hopefully I've explained my thinking enough for you to help me.

@AbstractDissonance really I think you are mistaken in your analysis. You have \inputlineno in the output routine that function is not called until it is invoked by \par on line 85, so I fail to see why you expect to see any other number. it would be the same in C or any other language.

As you are using luatex you can use the process_input_buffer callback to get and store more info about your input e.g. something that connects the line number with a \label. But be aware that it is dangerous to inject code in more or less arbitrary place: you could e.g. be in the middle of a command argument or a verbatim environment. In my opinion it will not work as you want it to work. Side remark: I'm not a little boy but a rather old woman. So say "she" when you mean me.

@DavidCarlisle I assume by output routine you mean the AfterEveryPageHook. If it is involved by a \par on line 85 then it is obviously the correct behavior. But \par is not a page. And it is obvious that AfterEveryPageHook is being called per pdf page(not tex page). That is, if your logic was absolutely correct, we would only have one output from AfterEveryPageHook, not N = #pages. So, While I may be missing something or getting something backwards, you still have not explained the logical flow that is taking place.

If you are saying that AfterEveryPageHook is only called once at the end of \par and that \inputlineno is for that place that it is being called, then ok! I get that. But then it makes \inputlineno nearly useless because it doesn't get the parent source. While some programs C type programs do stuff like this they are generally useless. For example D has __LINE__ which does one you want(gets the current line it is used at). And if used in a function it doesn't generally help. But you can set it up so that you can get the line of what actually called the function.

And so what we are talking about is scope. You are saying that \inputlineno is of local scope, not global. It does not return the outer most source code line number but the line number were the token shows up. Well, this seems useless to me because I know exactly the line number it showed up at so why would I need to use it in the first place? Instead of typing \inputlineno I could type 43, for example. Sure it is more robust but it doesn't help me write more "intelligent" programs. Instead, I would like/need something \inputcalleelineno. Which is limiting but solves my problem.

@AbstractDissonance no not at all!!!! You have put \inputlineno into the output routine so your typeout shows you which line of your document triggered that function. that is line 85 (the blank line). How can you know when writing the function where it will be called. Your output tells you a lot of information. the page breaker was called first on line 85 and output several pages, then it was called again on page 87 (\end{document}) to output the last page. How could you know that in advance when adding the typeout to the output routine?

@DavidCarlisle Because you told me it happened on the implicit \par on line 87. Look, this should be simple to explain ;) I might be very dense here but I think I'm missing a fundamental difference between how tex works and a normal programming language(for some reason I treat tex as a sort of C like program). If I were to put a break point on the AfterEveryPageHook and I were to step through the code(one char at a time I suppose), what lines do the AfterEveryPageHook inter at? You are telling me it is \par on line 85. But hat means it happens only once.

Yet it is called 7 times. and \thepage has a value that changes correctly with the output page. So what you are saying to me, in my own mind, simply does not make sense. You are telling me the hook gets called one time(or two due to the end of the document) yet it clearly get's called several times, one for as many pages that are in the pdf/output(not the input/source). Let's try to be clear here. AfterEveryPageHook is really "AfterEveryPageInSourceHook" or "AfterEveryPageInOutputHook"? Because the values of \thepage suggest the second while \inputlineno suggests the first.

@UlrikeFischer Ok, thanks. This looks like it might be usable(I'm assuming it will be called for each line(line ending in '\n' or '\r\n' in the source tex file). I only need to know correlation between the lines in the source and the pages. I do not actually need to modify anything so it shouldn't be a problem. The lines are used so I can automate \includeonly. To do that I have to know what has changed and where. While I do change some code(move it to a separate file and cache it) that already works. Is there any lua hook that is called when a block/group is entered for every block?

@AbstractDissonance Sorry I give up: this site doesn't work as a discussion forum and you are not picking up anything I'm trying to explain (which isn't surprising as this comment format doesn't help) but you are misunderstanding the tex control flow and so mis-interpretting the debugging information the typeout is providing. But if you have more questions better to ask a new question.