/dev/chat

8:58 AM

Please check gist.github.com/fmitha/5b56efb42c6023db71d81c2aacaef4e4 if you are a TeX user. And preferably also a Python user, though that isn't so important.

This might be a bug in the Python library Pandas.

Andras Deak

9:19 AM

@FaheemMitha why are you doing this "insert into latex" dance when you can just look at the output of the python script and check the labels?

Faheem Mitha

@AndrasDeak One can do that, certainly. But it's less work just to check the message.

Andras Deak

OK

This is like telling people to listen to the sound of the exhaust pipe on a car, rather than just looking under the car to see the hole in the drum

Faheem Mitha

Can you reproduce the error?

Andras Deak

I'm on mobile for now

When I'm on laptop I'll look under the car

Faheem Mitha

LaTeX error messages are very standard things. Oh, and the Python script just produces one label.

It's just in the wrong place.

Andras Deak

9:23 AM

"Oh, and the Python script just produces one label". So it's not a pandas bug?

@FaheemMitha ah

@FaheemMitha it's not an error

Faheem Mitha

@AndrasDeak Well, warning message. Depending on your definition of what an error is.

@AndrasDeak Oh, it's definitely a Pandas bug. IMO. Assuming it's still reproducible in 1.3.0.

Andras Deak

@FaheemMitha no, errors are errors and warnings are warnings

Faheem Mitha

I have 1.1.5. I guess I could backport 1.3.0, but it would be more work for me.

@AndrasDeak Whatever you say.

Andras Deak

@FaheemMitha Just pip install the master branch in a venv, if you really want to know

Faheem Mitha

@AndrasDeak That sounds like work too.

Andras Deak

9:26 AM

This backporting things is just making your own life miserable

@FaheemMitha three commands in bash

Faheem Mitha

The Debian Pandas package has an alarming number of patches, so the chances are that they wouldn't apply cleanly, anyway.

@AndrasDeak And what about undoing it, so that it doesn't screw up my Python installation?

Andras Deak

Reported and fixed in 1.2 github.com/pandas-dev/pandas/issues/34360

@FaheemMitha that's the venv part.

Faheem Mitha

@AndrasDeak I don't think it does.

@AndrasDeak I'll check it in a bit. AFK for a while.

Andras Deak

@FaheemMitha if you're willing to humor me for once I can tell you how to use a venv to show you how easy it is

But AFK here too

Faheem Mitha

9:55 AM

I keep getting my messages posted twice here. The second time it says it failed.

@AndrasDeak Fine, go ahead.

@AndrasDeak I'm not sure, but this might be fixing a different issue.

But @TorbjørnT on TeX SE reports he doesn't get an error. So I guess it's been fixed. Though I think I might as well check myself before forgetting about it.

Andras Deak

@FaheemMitha sounds exactly the same to me

> Currently DataFrame.to_latex() has a longtable argument, which writes the DataFrame to latex using the syntax of the longtable package. When used with the label and caption arguments, it writes a latex table with all of these features.

However, the caption and label are both placed between a \begin{longtable} line and \endhead. As a consequence, the table appears twice in the listoftables and leads to warnings about the reference being multiply defined

Faheem Mitha

In the example given: github.com/pandas-dev/pandas/issues/…

the label positioning is still wrong. Though of course I could be wrong.

@AndrasDeak Yes, that description appears approximately correct, though it needs to appear below \endfoot too. But of course the person who fixed it may have taken care of that.

Faheem Mitha

10:19 AM

Oh. No, looking at it again, it's correct. My mistake.

Andras Deak

@FaheemMitha OK, so it's really just 3 commands. Go into a directory where you want to create a (for now, temporary) directory for the venv.

Faheem Mitha

@AndrasDeak OK. Does it need to be in Python's path, or can it be any directory?

Andras Deak

Then, first create and activate the venv:
$ python3 -m venv tmpenv # tmpenv will be the directory name and the name of the venv
$ . tmpenv/bin/activate # sourcing this file activates the venv. You'll see the prompt change. If you want to leave it, just run `deactivate` which is a bash function defined on activation. As long as you're in the venv, any pip and python commands relate to the venv, without affecting your global env. Try issuing `pip freeze`, it should be (almost) empty.

Now install the newest pandas release (1.3.something?)

@FaheemMitha it should absolutely be outside python's path

Faheem Mitha

@AndrasDeak OK

@AndrasDeak OK, can you tell me where it will install, and how to remove it first?

The venv, that is.

Andras Deak

@FaheemMitha it installs everything into ./tmpenv, wherever that is. And to remove it you do rm -rf ./tmpenv. It helps if you deactivate the venv first, but that's not critical, as the env can be deactivated later and it only affects the running shell.

when the venv is not active, it's just a directory with stuff inside

Trust me, it's a no-brainer. My grandmother could use a venv, safely. Well, if she were still with us.

Faheem Mitha

10:26 AM

@AndrasDeak OK, trying it.

Andras Deak

"activation" itself is a very lightweight thing, it sets some paths and changes the prompt. You could even run the python file inside ./tmpenv/bin/python, and that would have the same effect without the prompt. It's very safe.

you can look in the ./tmpenv/bin/activate script to verify that

Faheem Mitha

@AndrasDeak Does it matter what directory the activate command is called from?

Andras Deak

@FaheemMitha I don't think so

yeah, no, I have a bash alias that activates wherever I am

alias venvme='source ~/virtualenvs/py3.9_main/bin/activate'

(and it works)

be back later, breakfast

Faheem Mitha

Yes, I see (tmpenv) at the beginning of the prompt now.

pip freeze
pkg-resources==0.0.0

Odd command choice, freeze.

Wow, it's installing a wagon load of stuff.

Faheem Mitha

10:51 AM

So, just to be clear, this shell will now use this installed venv, until it's deactivated?

Andras Deak

Yup

Not sure how it interacts with running processes like your latex example

First include the new tex output manually

Faheem Mitha

OK, it's fixed. And it follows the layout that I quoted from that issue.

Andras Deak

excellent

Faheem Mitha

So . tmpenv/bin/activate to deactivate?

Andras Deak

And now you know why I recommend this workflow over backporting

@FaheemMitha no

Faheem Mitha

10:57 AM

Nice to come across a bug that's actually been fixed. Then again, the maintainers of the big Python libraries seem to be quite proactive.

Andras Deak

deactivate shell function

Faheem Mitha

. tmpenv/bin/deactivate to deactivate, sorry.

Andras Deak

@FaheemMitha absolutely

@FaheemMitha also no

Faheem Mitha

I regularly come across bugs that are years old.

And bugs I report are regularly ignored. Though I don't report many these days.

Andras Deak

. something is the same as source something

You need a function call

$ deactivate

Faheem Mitha

11:00 AM

@AndrasDeak Oh. OK, thank you. I'm back to the regular prompt. And I trust my regular Python installation will ignore those other pip installed Python packages?

Andras Deak

Yes

You can pip3 freeze again

Faheem Mitha

OK, good.

@AndrasDeak Condolences for your grandmother.

@AndrasDeak It's now showing a whole bunch of stuff. I didn't realise I had some much installed with pip. Or maybe it shows the Debian packages too? For example, it shows Mercurial, which I certainly didn't install with pip.

@AndrasDeak Backporting is fine. But I realise it's not everyone's cup of tea.

Andras Deak

@FaheemMitha thank you

@FaheemMitha yes, the global env includes system packages

That's why it's bad to sudo pip install

Faheem Mitha

@AndrasDeak OK. Thank you for the confirmation.

Andras Deak

@FaheemMitha it can only be a lot more effort than using a venv

Using a venv gives you the newest releases with no effort

Faheem Mitha

11:13 AM

@AndrasDeak I wouldn't dream of it. Though these days my mental faculties seem to be dropping rapidly, so who knows. I did something very stupid and dangerous last week, for example.

@AndrasDeak Backporting is much more work, of course.

Andras Deak

Some things can't be pip installed or have non-python dependencies. Those are harder.

Faheem Mitha

But once it's done it's part of your system, and shouldn't cause any issues.

The last of my grandparents died in 1982, I think.

I can barely remember her.

Andras Deak

I'm sorry

Andras Deak

11:43 AM

So my very strong recommendation would be to default to a virtualenv, possibly with a kind of alias I showed. I'd even prefer the effort of figuring out how to use the virtualenv's python in build scripts with as little effort as possible.

@FaheemMitha yeah, I figured. But you'd have to go to all that extra effort with every new release of the library that you want to update to. With a virtualenv it's as simple as pip install -U pandas to update a package.

Faheem Mitha

@AndrasDeak I don't usually need to update very often. I'm also not a heavy Python user currently.

And this might be an option for Python, but for lots of other things it isn't an option.

I suppose one can try to run stuff in chroots, but it all winds up getting rather messy, I imagine. Though I've not tried. The stuff that I upgrade, I mostly backport. Sometimes I install things with local tools like pip3 and luarocks, but I really try to avoid that.

Andras Deak

Yes, I mean specifically python

Faheem Mitha

Unfortunately, Luarocks on Debian is itself horribly out of date.

@AndrasDeak If I become a heavy Python user again, and need to have lots of very up-to-date packages, then maybe I'll switch to using a different approach.

These days, I occasionally write the odd Python script.

The Python ecosystem seems to have got larger since I first started using Python in 2004.

Andras Deak

11:58 AM

absolutely

Faheem Mitha

@AndrasDeak Anyway, thank you for your assistance with this.

@AndrasDeak BTW, do you happen to use the longtable package?

Andras Deak

12:27 PM

No problem. And no, I'm not familiar with it.

Faheem Mitha

3:45 PM

@AndrasDeak Thinking of you -> thewire.in/history/…

@AndrasDeak OK. The reason I asked was that the longtable documentation is obviously deficient in some respects. But unsurprisingly, David Carlisle isn't receptive to the idea of changes. Regardless, I thought I might work on a patch.

Andras Deak

6:15 PM

@FaheemMitha Horthy in our case is an interesting choice. He was pretty fascist, after all he allowed the first anti-jew laws to be made. But the real, hardcore nazis came after, led by Szálasi. Our current fascists are working hard to whitewash Horthy's memory for some reason.

our current regime ticks quite a few boxes in Eco's list

1995. Not bad.

@FaheemMitha can you show specific examples of deficiencies? I've seen you be confused of situations that seemed clear-cut to me, so I wonder if he's reluctant because he thinks the shortcomings aren't in the documentation.

Faheem Mitha

6:45 PM

@AndrasDeak Do you have the longtable manual to hand?

I was specifically talking about the first para of Section 3: Captions and Headings

I'm planning to write a critique and a modified version, but I don't know if I will actually get around to it.

@AndrasDeak 1995?

@AndrasDeak Ours ticks a lot of them, though I haven't systematically gone through Eco Umberto's list.

Some of the things these creatures say is like they're quoting from a manual. It's so transparent and comical.

Currently the "Parliament" is more or less shut down, because the Opposition and the "Government" are fighting about the "Government's" refusal to reply as to whether they're responsible for the Pegasus thing. In fact they're refusing to have a discussion about it at all.

Andras Deak

7:04 PM

@FaheemMitha I didn't, but I do now

@FaheemMitha seemed like that was when Eco published his list nybooks.com/articles/1995/06/22/ur-fascism

Faheem Mitha

@AndrasDeak Oh.

Paywalls.

> I elaborated with rhetorical skill on the subject “Should we die for the glory of Mussolini and the immortal destiny of Italy?” My answer was positive. I was a smart boy.

LOL.

Andras Deak

@FaheemMitha yeah, our government majority sabotaged the official investigation into Pegasus. They say there's nothing to talk about and that's it.

@FaheemMitha OK. All but the last sentence seems clear to me, without trying to use it in practice.

Faheem Mitha

@AndrasDeak Hmm. Exactly the same reaction here. "It's a non-issue". That's a verbatim quote, I think.

@AndrasDeak Ah. Well, the explanation seems very incomplete to me.

Of course, it may be my mental deficiences. But without further data it's hard to know.

If you are willing to review a patch, I can post it here, if and when I get around to it.

Andras Deak

I can try, but it would also help if you can pinpoint what use case it doesn't explain enough.

I don't have the bandwidth to try and play with example tables to see how it behaves

Faheem Mitha

@AndrasDeak Well, my patch would explain the rationale.

Barbara Beeton said she would also take a look, so hopefully that's 2 people.

I had an argument in TeX SE on the subject a few days ago.

Andras Deak

7:16 PM

Rationale for what?

Faheem Mitha

@AndrasDeak For the patch. If one thinks documentation is unclear, it's helpful to first explain why one thinks it's unclear.

That would be before the actual patch. Assuming I could modify David's prose in a way that he would find acceptable (unlikely).

The best case scenario is that he would concede that I had a point and add a sentence or two (also unlikely, but not quite so unlikely).

Andras Deak

ah

OK, feel free to ping me, at worst we'll be none the wiser after I've read it

Faheem Mitha

The TeX set aren't very cooperative when it comes to documentation. Their general attitude is that it's all perfectly clear, and if it isn't, you should just read the source code.

That's all well and good if one is one of the few people on earth who can read and understand TeX code comfortably. My guess is that number is a few thousands.

I'm not one of them.

@AndrasDeak OK. Thank you.

Oddly, they don't have a problem with TeX SE. Which is thousands of questions and answers, in part explaining all that perfectly clear stuff. But they don't seem to like it in the actual documentation.

Of course, some documentation is better than others. Both the KOMA-script and PGF/Tikz manuals are quite detailed, for example.

Andras Deak

7:35 PM

all 1300 pages

Transcript for