8:58 AM
Please check gist.github.com/fmitha/5b56efb42c6023db71d81c2aacaef4e4 if you are a TeX user. And preferably also a Python user, though that isn't so important.
This might be a bug in the Python library Pandas.

9:19 AM
@FaheemMitha why are you doing this "insert into latex" dance when you can just look at the output of the python script and check the labels?

@AndrasDeak One can do that, certainly. But it's less work just to check the message.

OK
This is like telling people to listen to the sound of the exhaust pipe on a car, rather than just looking under the car to see the hole in the drum

Can you reproduce the error?

I'm on mobile for now
When I'm on laptop I'll look under the car

LaTeX error messages are very standard things. Oh, and the Python script just produces one label.
It's just in the wrong place.

9:23 AM
"Oh, and the Python script just produces one label". So it's not a pandas bug?
@FaheemMitha ah
@FaheemMitha it's not an error

@AndrasDeak Well, warning message. Depending on your definition of what an error is.
@AndrasDeak Oh, it's definitely a Pandas bug. IMO. Assuming it's still reproducible in 1.3.0.

@FaheemMitha no, errors are errors and warnings are warnings

I have 1.1.5. I guess I could backport 1.3.0, but it would be more work for me.
@AndrasDeak Whatever you say.

@FaheemMitha Just pip install the master branch in a venv, if you really want to know

@AndrasDeak That sounds like work too.

9:26 AM
This backporting things is just making your own life miserable
@FaheemMitha three commands in bash

The Debian Pandas package has an alarming number of patches, so the chances are that they wouldn't apply cleanly, anyway.
@AndrasDeak And what about undoing it, so that it doesn't screw up my Python installation?

@FaheemMitha that's the venv part.

@AndrasDeak I don't think it does.
@AndrasDeak I'll check it in a bit. AFK for a while.

@FaheemMitha if you're willing to humor me for once I can tell you how to use a venv to show you how easy it is
But AFK here too

9:55 AM
I keep getting my messages posted twice here. The second time it says it failed.
@AndrasDeak I'm not sure, but this might be fixing a different issue.
But @TorbjørnT on TeX SE reports he doesn't get an error. So I guess it's been fixed. Though I think I might as well check myself before forgetting about it.

@FaheemMitha sounds exactly the same to me
> Currently DataFrame.to_latex() has a longtable argument, which writes the DataFrame to latex using the syntax of the longtable package. When used with the label and caption arguments, it writes a latex table with all of these features.

However, the caption and label are both placed between a \begin{longtable} line and \endhead. As a consequence, the table appears twice in the listoftables and leads to warnings about the reference being multiply defined

the label positioning is still wrong. Though of course I could be wrong.
@AndrasDeak Yes, that description appears approximately correct, though it needs to appear below \endfoot too. But of course the person who fixed it may have taken care of that.

10:19 AM
Oh. No, looking at it again, it's correct. My mistake.

@FaheemMitha OK, so it's really just 3 commands. Go into a directory where you want to create a (for now, temporary) directory for the venv.

@AndrasDeak OK. Does it need to be in Python's path, or can it be any directory?

Then, first create and activate the venv:
$python3 -m venv tmpenv # tmpenv will be the directory name and the name of the venv$ . tmpenv/bin/activate # sourcing this file activates the venv. You'll see the prompt change. If you want to leave it, just run deactivate which is a bash function defined on activation. As long as you're in the venv, any pip and python commands relate to the venv, without affecting your global env. Try issuing pip freeze, it should be (almost) empty.

Now install the newest pandas release (1.3.something?)
@FaheemMitha it should absolutely be outside python's path

@AndrasDeak OK
@AndrasDeak OK, can you tell me where it will install, and how to remove it first?
The venv, that is.

@FaheemMitha it installs everything into ./tmpenv, wherever that is. And to remove it you do rm -rf ./tmpenv. It helps if you deactivate the venv first, but that's not critical, as the env can be deactivated later and it only affects the running shell.
when the venv is not active, it's just a directory with stuff inside
Trust me, it's a no-brainer. My grandmother could use a venv, safely. Well, if she were still with us.

10:26 AM
@AndrasDeak OK, trying it.

"activation" itself is a very lightweight thing, it sets some paths and changes the prompt. You could even run the python file inside ./tmpenv/bin/python, and that would have the same effect without the prompt. It's very safe.
you can look in the ./tmpenv/bin/activate script to verify that

@AndrasDeak Does it matter what directory the activate command is called from?

@FaheemMitha I don't think so
yeah, no, I have a bash alias that activates wherever I am
alias venvme='source ~/virtualenvs/py3.9_main/bin/activate'
(and it works)
be back later, breakfast

Yes, I see (tmpenv) at the beginning of the prompt now.
pip freeze
pkg-resources==0.0.0
Odd command choice, freeze.
Wow, it's installing a wagon load of stuff.

10:51 AM
So, just to be clear, this shell will now use this installed venv, until it's deactivated?

Yup
Not sure how it interacts with running processes like your latex example
First include the new tex output manually

OK, it's fixed. And it follows the layout that I quoted from that issue.

excellent

So . tmpenv/bin/activate to deactivate?

And now you know why I recommend this workflow over backporting
@FaheemMitha no

10:57 AM
Nice to come across a bug that's actually been fixed. Then again, the maintainers of the big Python libraries seem to be quite proactive.

deactivate shell function

. tmpenv/bin/deactivate to deactivate, sorry.

@FaheemMitha absolutely
@FaheemMitha also no

I regularly come across bugs that are years old.
And bugs I report are regularly ignored. Though I don't report many these days.

. something is the same as source something
You need a function call
\$ deactivate

11:00 AM
@AndrasDeak Oh. OK, thank you. I'm back to the regular prompt. And I trust my regular Python installation will ignore those other pip installed Python packages?

Yes
You can pip3 freeze again

OK, good.
@AndrasDeak It's now showing a whole bunch of stuff. I didn't realise I had some much installed with pip. Or maybe it shows the Debian packages too? For example, it shows Mercurial, which I certainly didn't install with pip.
@AndrasDeak Backporting is fine. But I realise it's not everyone's cup of tea.

@FaheemMitha thank you
@FaheemMitha yes, the global env includes system packages
That's why it's bad to sudo pip install

@AndrasDeak OK. Thank you for the confirmation.

@FaheemMitha it can only be a lot more effort than using a venv
Using a venv gives you the newest releases with no effort

11:13 AM
@AndrasDeak I wouldn't dream of it. Though these days my mental faculties seem to be dropping rapidly, so who knows. I did something very stupid and dangerous last week, for example.
@AndrasDeak Backporting is much more work, of course.

Some things can't be pip installed or have non-python dependencies. Those are harder.

But once it's done it's part of your system, and shouldn't cause any issues.
The last of my grandparents died in 1982, I think.
I can barely remember her.

I'm sorry

11:43 AM
So my very strong recommendation would be to default to a virtualenv, possibly with a kind of alias I showed. I'd even prefer the effort of figuring out how to use the virtualenv's python in build scripts with as little effort as possible.
@FaheemMitha yeah, I figured. But you'd have to go to all that extra effort with every new release of the library that you want to update to. With a virtualenv it's as simple as pip install -U pandas to update a package.

@AndrasDeak I don't usually need to update very often. I'm also not a heavy Python user currently.
And this might be an option for Python, but for lots of other things it isn't an option.
I suppose one can try to run stuff in chroots, but it all winds up getting rather messy, I imagine. Though I've not tried. The stuff that I upgrade, I mostly backport. Sometimes I install things with local tools like pip3 and luarocks, but I really try to avoid that.

Yes, I mean specifically python

Unfortunately, Luarocks on Debian is itself horribly out of date.
@AndrasDeak If I become a heavy Python user again, and need to have lots of very up-to-date packages, then maybe I'll switch to using a different approach.
These days, I occasionally write the odd Python script.
The Python ecosystem seems to have got larger since I first started using Python in 2004.

11:58 AM
absolutely

@AndrasDeak Anyway, thank you for your assistance with this.
@AndrasDeak BTW, do you happen to use the longtable package?

12:27 PM
No problem. And no, I'm not familiar with it.

3 hours later…
3:45 PM
@AndrasDeak Thinking of you -> thewire.in/history/…
@AndrasDeak OK. The reason I asked was that the longtable documentation is obviously deficient in some respects. But unsurprisingly, David Carlisle isn't receptive to the idea of changes. Regardless, I thought I might work on a patch.

2 hours later…
6:15 PM
@FaheemMitha Horthy in our case is an interesting choice. He was pretty fascist, after all he allowed the first anti-jew laws to be made. But the real, hardcore nazis came after, led by Szálasi. Our current fascists are working hard to whitewash Horthy's memory for some reason.
our current regime ticks quite a few boxes in Eco's list
@FaheemMitha can you show specific examples of deficiencies? I've seen you be confused of situations that seemed clear-cut to me, so I wonder if he's reluctant because he thinks the shortcomings aren't in the documentation.

6:45 PM
@AndrasDeak Do you have the longtable manual to hand?
I was specifically talking about the first para of Section 3: Captions and Headings
I'm planning to write a critique and a modified version, but I don't know if I will actually get around to it.
@AndrasDeak 1995?
@AndrasDeak Ours ticks a lot of them, though I haven't systematically gone through Eco Umberto's list.
Some of the things these creatures say is like they're quoting from a manual. It's so transparent and comical.
Currently the "Parliament" is more or less shut down, because the Opposition and the "Government" are fighting about the "Government's" refusal to reply as to whether they're responsible for the Pegasus thing. In fact they're refusing to have a discussion about it at all.

7:04 PM
@FaheemMitha I didn't, but I do now
@FaheemMitha seemed like that was when Eco published his list nybooks.com/articles/1995/06/22/ur-fascism

@AndrasDeak Oh.
Paywalls.
> I elaborated with rhetorical skill on the subject “Should we die for the glory of Mussolini and the immortal destiny of Italy?” My answer was positive. I was a smart boy.
LOL.

@FaheemMitha yeah, our government majority sabotaged the official investigation into Pegasus. They say there's nothing to talk about and that's it.
@FaheemMitha OK. All but the last sentence seems clear to me, without trying to use it in practice.

@AndrasDeak Hmm. Exactly the same reaction here. "It's a non-issue". That's a verbatim quote, I think.
@AndrasDeak Ah. Well, the explanation seems very incomplete to me.
Of course, it may be my mental deficiences. But without further data it's hard to know.
If you are willing to review a patch, I can post it here, if and when I get around to it.

I can try, but it would also help if you can pinpoint what use case it doesn't explain enough.
I don't have the bandwidth to try and play with example tables to see how it behaves

@AndrasDeak Well, my patch would explain the rationale.
Barbara Beeton said she would also take a look, so hopefully that's 2 people.
I had an argument in TeX SE on the subject a few days ago.

7:16 PM
Rationale for what?

@AndrasDeak For the patch. If one thinks documentation is unclear, it's helpful to first explain why one thinks it's unclear.
That would be before the actual patch. Assuming I could modify David's prose in a way that he would find acceptable (unlikely).
The best case scenario is that he would concede that I had a point and add a sentence or two (also unlikely, but not quite so unlikely).

ah
OK, feel free to ping me, at worst we'll be none the wiser after I've read it

The TeX set aren't very cooperative when it comes to documentation. Their general attitude is that it's all perfectly clear, and if it isn't, you should just read the source code.
That's all well and good if one is one of the few people on earth who can read and understand TeX code comfortably. My guess is that number is a few thousands.
I'm not one of them.
@AndrasDeak OK. Thank you.
Oddly, they don't have a problem with TeX SE. Which is thousands of questions and answers, in part explaining all that perfectly clear stuff. But they don't seem to like it in the actual documentation.
Of course, some documentation is better than others. Both the KOMA-script and PGF/Tikz manuals are quite detailed, for example.

7:35 PM
all 1300 pages