Yes, the implementation of stateful iterators in Lua. And how it interacts with the Lua feature of returning multiple values.

@FaheemMitha note that "There is no guarantee about the types of the results [...] this method could return nil as a valid result."

In your example, if a call to fetch returns nil for the date column, the loop will exit; the cursor hasn't run off the end of the table, so it's still open

I would expect it to be closed when garbage-collected, but lua doesn't have precise destructors so that will probably happen after con:close(), resulting in the "still has open cursors" error

Consider using the variant of fetch that takes a table to fill in

(more generally, lua uses nil as the first return value from an iterator to indicate that iteration is complete, rather than having something like python's StopIterationException; thus, you can't use an iterator in a for loop to lazy-yield a sequence that contains nils unless you're ok stopping at the first nil. You instead need to wrap it in a guaranteed non-nil value (as cur:fetch(t) does) or use a different loop construct.)

@ToxicFrog I don't know where you are quoting from.

@ToxicFrog What variant of fetch is that?

@FaheemMitha the same luaSQL documentation you quote from in "A call to fetch after the last row has already being returned will close the corresponding cursor". The same paragraph, in fact.

Oh, there are different versions of fetch?

@ToxicFrog Ah, yes. LuqSQL. I momentarily forgot it existed. It has different versions of fetch?

Read all the documentation for cursor:fetch that you link to in your gist -- you can either call it without arguments and get the individual fields back as return values on the stack, or you can pass it a table as its first argument and it will write the row contents into the table and return the table.

(unless there are no rows left, in which case it leaves the table untouched and returns nil)

@ToxicFrog I see. So the latter is perhaps better for my purposes?

In table-filling mode there's an optional second argument that tells it whether to use numeric indices when filling the table, the names of the various columns, or both.

And OK, I'll go back and reread it again.

If there is any possibility that the Date column could be nil, yes.

@ToxicFrog Oh, OK. Looks like I missed a lot.

@ToxicFrog Do you use Lua?

Yes.

I've never used luaSQL, but I read the documentation you linked to, and I've been using Lua for various things on since 2003-ish.

@ToxicFrog Oh. That's a long time.

Thank you for taking the time to help. I'm in the middle of sending an email right now, but I'll look at the documentation again shortly.

It is! I must confess I'm not as up to date on recent (5.3/5.4) versions as I would like, since I mostly use luajit these days (which is basically 5.1 with some 5.2/5.3 stuff backported where possible) -- basically juust the bare minimum needed to update libraries I maintain to support recent versions. But the loop iteration behaviour hasn't significantly changed in any of the 5.x versions :)

@ToxicFrog Yes, that stateful iterator behavior seems pretty much baked in.

I'm not a fan of Lua, and unlikely to become one. But I use it because of LuaTeX.

Fair.

It shows up in a lot of places because it is very successful in its niche (i.e. embedded, runtime-loadable, simple language)

@ToxicFrog Erratic library support is particularly annoying. And along with that a weird culture of reinventing the wheel.

I suspect those are both tied together; lua's design is intentionally minimalist (you're meant to be able to include it its entirety in an existing project without a significant impact on memory usage, startup time, etc), which means the selection of built-in libraries is very sparse, especially when compared to something designed for standalone use like python

So, for a lot of use cases, library support is down to which lua users have the combination of time, need, and knowledge to make a library filling that need

And since such libraries are not built in, it's easy for someone else with the same need to write their own library to fill it without being aware of existing libraries in the same niche

(Or, as in my case with vstruct, to be aware of them but conclude that none of them fill that need properly for the task at hand -- when I wrote vstruct there were at least three other struct packing/unpacking libraries already but they only supported standard C types)

There have been various attempts to put together a "canonical set of lua extension libraries" (e.g. luadist) but historically it's been a very "now there are 14 standards" kind of situation.

@ToxicFrog Yes. most of this is in line with my experiences with Lua.

I wonder why it's not an option to have a collection of "standard" Lua libraries which are not built-in.

At a guess, the lua developers would rather focus on developing lua the language and its core libraries.

It's not like python; supporting standalone, general-purpose use is not part of their objectives in the first place.

@ToxicFrog In my use case, the data comes from SQLite. And then gets handed over to TeX to be (partially) printed. So, if I write it to a Lua table first, it will add a separate layer of indirection, but perhaps that's the cleanest way to proceed. Alternatively, I could make sure an index is returned from SQLite (that was the direction I was leaning), since DB tables always have indexes. You just have to include them when calling the iterator.

@ToxicFrog In some respects, usage in LuaTeX is standalone, general use. You could want to link to a wide variety of Lua libraries in theory.

Anyone else struggling with nvidia v460 driver?

Right, but my point is that that's not really what the Lua developers care about, it's explicitly not part of their scope. They aren't the people who work on LuaTeX.

Or ... I suspect it's that one.

Making sure an index is returned would mean minimal changes to the existing code, so that sounds like a reasonable approach.

Upgraded kernel+nvidia 4 days ago. After that computer has went into complete lock 2-3 times a day :/

Adding the table has hard-to-assess performance implications that largely depend on how many columns you have and their types; if this is happening as part of TeX compilation I suspect the difference isn't enough to matter one way or another, so the main concern is the process of prying open the table to get the contents into TeX afterwards.

@ToxicFrog Yes, I realise they don't care about LuaTeX. Currently, even most TeX users don't care about LuaTeX. It hasn't yet caught on in a big way. Though it's probably the future of TeX, unless someone else manages to embed Python (for example).

@ToxicFrog That extra level of indirection isn't really an issue. It's just a question of which was to go.

nod

@ibuprofen Sounds unpleasant. I'd downgrade for now. And then maybe experiment with a test installation.

Every 6 months or so it's the same all over. Some kernel+nvidia release and system goes completely bonkers. Bad for a month+ then some release and ok for a few months then a new release and all goes to the dogs again.

What distribution?

ubuntu lts 18.04

@ibuprofen Well, proprietary hardware has its problems, yes.

@ibuprofen I use Nvidia too (there aren't a lot of options). And don't have any driver issues.

Are you using a high-end card?

fairly

I might be using the legacy drivers. I forget what those are.

GeForce GTX 1070 Ti

But I see the word legacy everywhere.

@ibuprofen GeForce GT 610 here.

Why not just use an older card.

last time was around April, and before that around nov/des

I'm not sure if there are any reasonable alternatives to NVidia these days. It used to be ATI, which I've never used.

Well, I have spent money on it

But things don't see to have changed much for video cards in the last 20 years.

@ibuprofen And now you're wasting time not being able to use your computer. Did you check how well the drivers worked on it before your purchase? That's a good defense strategy when dealing with Linux drivers.

Sure. Checked out. It worked fine for a couple of years.

@ibuprofen Oh. Well, that's frustrating.

I wonder if my video drivers will one day go phut too. The perils of proprietary hardware. And software, really. Since the driver is proprietary too.

Any chance that a non-3-year-old non-ubuntu OS would have better support?

@FaheemMitha so is your CPU, I'm sure

and probably the rest of your computer

@ibuprofen Actually, a 2018 distribution is quite old. Is there a reason you're not using something more recent?

it's probably LTS

I installed LTS because I wanted LTS.

Ah, yes. LTS.

I always end up tweaking the system, and the thought of upgrading is a pain lol.

It's my last Ubuntu install though.

What's the plan for the next one?

Not sure.

I had some trouble setting up BIND etc. due to all the systemd things on Ubuntu + I do not like snap :P

I like apt so likely something Debian based.

And I do not like Wayland, not that I have looked much at it.

I have X11 on my debian, because at one point Wayland just broke, and I switched. That's the extent of my knowledge in the matter.

I utilize various X tools like xrandr, xdotools, xprop, xwininfo, nested x - not to mention some automation with OpenCV, keyboard automation etc. which does not exist under Wayland.

huh