@JeffSchaller FWIW I think it's a perfectly reasonable issue to raise. Just one of the many gray areas out there.

I had a similar question/issue not long ago, though I forget the details. In that case too, it apparently qualified as an answer.

The bottom line is, I think, that what is an answer and what is a comment, is not well defined.

@ToxicFrog Mind if I ask you about the Lua C API? Not really on-topic here.

Or I suppose I could try my luck on IRC. Though I don't really find it a satisfactory medium. And the chat is not archived.

@FaheemMitha go ahead

If you do want to try your luck on IRC, I believe #lua on libera.chat is the canonical channel these days, although I haven't been a regular there for years

So, in the following answer (technically LuaTeX)

I don't understand the necessity of the line

lua_remove(L, -2);

Why pop the function before it is called? I tried with and without and it works both ways (unless I made an error).

For context, texio is a Lua module embedded inside LuaTeX. It's not part of Lua proper.

Or perhaps embedded inside the Lua library used with LuaTeX. If it makes a difference.

I just managed this afternoon to understanding the beginnings of this Lua stack thing.

So I'm sure there is much I don't understand.

And while we're on the subject of things I don't understand, how does lua_call know what function to use? Or is the case that only one function can be on the stack at one time?

taking the short answer first: lua_call knows what function to call because you tell it how many arguments the function takes

So it pops that many arguments and then calls what's on top of the stack

That's why the protocol for using it is "push the function, then the arguments in direct order, then lua_call"

The arguments must immediately follow the function to be called on the stack.

@ToxicFrog I see. That's a very clear description.

So if you push two functions in sequence, along with their corresponding arguments (in order), and then do lua_call, it will execute the more recent function?

It depends on how many arguments you tell it to pop.

Like, functions that take other functions as arguments are a thing

If you push onto the stack, say, [f, 1, g, 2, 3], where f and g are functions

lua_call(L, 2, 1) will leave f and 1 on the stack, call g(2,3), and leave the stack as [f, 1, <one return value from g()>]

lua_call(L, 4, 1) will pop all of those from the stack, call f(1,g,2,3), and push the first return value of f() onto the stack

@ToxicFrog Yes, good point. And Lua functions are first class objects. For some value of "first class".

And I suppose return values automatically get pushed?

You should read the documentation for lua_call, since it sounds like you haven't: lua.org/manual/5.3/manual.html#lua_call

Return value behaviour depends on the third argument to it.

@ToxicFrog LOL. You're the second person to tell me that. No, I read it. Otherwise I would have absolutely no idea how this worked.

@ToxicFrog I'm assuming that there is only one argument.

...only one argument to what?

I have people explaining the same thing to me in TeX SE Chat. Which you wouldn't know if you're not there.

I am not.

Did you mean "only one return value"?

@ToxicFrog Sorry, assuming the function only has one return value, not multiple.

Behold, I quote from the manual:

> The function results are pushed onto the stack when the function returns. The number of results is adjusted to nresults, unless nresults is LUA_MULTRET. In this case, all results from the function are pushed; Lua takes care that the returned values fit into the stack space, but it does not ensure any extra space in the stack.

I'm not sure I'll ever get used to Lua having multiple return values.

@ToxicFrog OK.

If the third argument is 0, no return values are pushed. If it's 1 or more, exactly that many return values are pushed; if the function didn't return that many the extras are filled in with nil. If it's LUA_MULTRET exactly as many return values as the function returned (which may be 0) are pushed.

As for your other question, you should read this: lua.org/manual/5.3/manual.html#lua_CFunction

The code you have provided is, in fact, wrong with or without the call to lua_remove, but may work by coincidence

@ToxicFrog Which question was that? Sorry, I lost track.

@ToxicFrog Oh. Which part is wrong?

"why does it work with or without lua_remove"

The signature for a lua->C call is: the C function gets its arguments from lua on the stack, and to return back to lua, pushes its return values and returns the number of return values R; the top R items on the stack are returned to lua and everything below them is discarded.

In your case, without the lua_remove call, the stack as your function executes looks like:

[texio] -> [texio write_nl] -> [texio write_nl "Hello World"] -> (lua_call 1 0) -> [texio] -> return 1 -> texio table returned to lua

With the lua_remove call, it looks like this:

[texio] -> [texio write_nl] -> (lua_remove -2) -> [write_nl] -> [write_nl "Hello World"] -> (lua_call 1 0) -> [] -> return 1 -> stack underflow

I do not recall if the lua spec says what happens if you stack underflow on return from a C function; it may be implementation or even compile-time-option dependent.

But you are definitely not meant to do it.

So that's the part that's wrong.

The return 1 part?

You mean if you're already on the top of the stack, you don't return 1?

what?

Ok, let's try one more time

Do you understand what return 1 means in that context?

Actually, the bit I was concerned about with removing texio, is that I thought write_nl depended on it. The people in TeX SE said it didn't. Because on the lookup was done, the lookup table could be discarded.

@ToxicFrog Assume I don't.

@FaheemMitha then you should read this again: lua.org/manual/5.3/manual.html#lua_CFunction

Because it's explained there, and I also explained it earlier in this conversation.

@ToxicFrog I haven't read that section yet, no.

Meaning lua_CFunction, I assume.

Well, you should, because that's the part that explains how you write C functions that are callable from lua, which is what you are doing in that answer.

@ToxicFrog OK, will do. Thank you.

> the bit I was concerned about with removing texio, is that I thought write_nl depended on it.

It does not. All you need the table for is fishing a reference to the function out of it. Once you have that reference you don't need a reference to the table anymore; even if the table got garbage collected (which it won't, because it's a global) the fact that you have a reference to the function on your stack will keep it safe.

That said, you also do not need to lua_remove, because any extra debris left on the stack will be cleaned up for you when you return back to lua as long as you properly obey the lua->C calling convention.

@ToxicFrog Yes, I understand that now.

It is good form to clean up the stack after you when writing C functions to be called by other C functions, but when writing C functions to be called by Lua the interpreter handles some of that for you.

@ToxicFrog Yes, I'll leave that part out.

@ToxicFrog OK, so it seems that in this case no result is returned (it just does a print statement), so it should return 0.

Yes.

Thank you for the explanations. I think I understand things better now.

@ToxicFrog So by "stack underflow" you are referring to the fact that "return 1" misinforms Lua that the value at the top of the stack is the return value of that function, when it fact it has no return value?

So Lua will pop that value, and try to use it as the return value for that function?

Which presumably would not end well in general.

Yes. Expanding on that:

- if there is something on top of the stack, it will be returned, and program execution will continue normally, except you returned some unexpected value from a function that wasn't meant to return anything. Probably the caller ignores it and everything is fine, but it's untidy, especially if they were doing something that assumed the function would return nothing!

- if the stack is empty, you just told it to pop 1 value from an empty stack; if you compiled with -DLUA_APICHECK it should assert() immediately, if you did not it might segfault, return a value from another function's stack frame, or something else entirely

@ToxicFrog Yes, those all sound undesirable. I suppose I should test all these things in debug mode. I tried -DLUA_APICHECK when compiling the shared library, but it didn't seem to make any difference to the TeX run. Maybe I used it wrong.

Hmm. Probably it needs to be in the code itself. I'll look at it after dinner.

Sorry, I got it wrong, it's LUA_USE_APICHECK

And it needs to be defined when compiling lua itself

Since it affects the emitted code for the lua_* and luaL_* functions

i.e. it's a setting for liblua, not individual libs you write that interoperate with it.

lua.org/manual/5.3/manual.html#4 it is mentioned in the intro to chapter 4.

@ToxicFrog Maybe that's the Lua debug version in Debian.

I suspect that debian -debug packages are just "the same thing but built with -O0 and debug symbols", but I'm not very familiar with it; it may be that they also add package-specific debug options.

@ToxicFrog I suspect that both functions in tex.stackexchange.com/a/370904/3406 (namely callprint and printfromc) should return 0. Am I right? I've only changed the first one.

@ToxicFrog Perhaps so. I could download the source and check.

@ToxicFrog It looks like you are right.

@FaheemMitha lua.org/manual/5.3/manual.html#pdf-require has information on the signature of module loaders

tl;dr best practice is to return the module object itself, but if the module is accessible via other means (e.g. has been made global) you can return nothing.

@ToxicFrog That second function is a module loader?

Clearly I have more reading to do.

Yes.

Possibly you shouldn't be editing someone else's code without understanding it.

@ToxicFrog I just made the one change.

With regard to the module return value, I could not find anything other than:

> If the loader returns any non-nil value, require assigns the returned value to package.loaded[modname]. If the loader does not return a non-nil value and has not assigned any value to package.loaded[modname], then require assigns true to this entry. In any case, require returns the final value of package.loaded[modname].

But all the examples I've found return 1.

I'm not clear of the implications of "require assigns the returned value to package.loaded[modname]".

@ToxicFrog ^^

@FaheemMitha so, when you do require('foo'), what it does is:

- search for a library named foo

- load it and call the loader function, which for C libraries is the luaopen_foo() function

That remembers me that I should update one of my lua plugins for mpv

- if the function doesn't return anything, set package.loaded.foo = true and return true; the library is assumed to have set a global 'foo' or something

- if it does return something, set package.loaded.foo = <that thing> and return it

The latter has been considered the best practice for lua libraries for ~15 years at this point; the library is expected to not set any globals, and instead return the library table itself; the caller can then assign it to a global or a local as they see fit

So the idiom on the caller side is (e.g.) local lfs = require 'lfs'

From the perspective of library authoring, this means that, unless you are doing something really weird, your luaopen_foo() function should pack all the user-accessible parts of the library into a table (typically using luaL_newlib, although you can do it by hand if needed) and return that table without setting any globals

Or, if for some reason you need to support the old-shool usage idiom, pack it into a table and assign that table to a global, then either return the table as well or return nothing

In the case of the given question, the module loader is following the modern idiom, i.e. create a new table and return it without assigning it to a global

So return 1 is correct there.