/dev/chat

6:25 AM

@Tim Most of statistics, except the very theoretical/mathematical areas, are relevant and useful for industrial applicaions. It's an applied science, after all.

@Tim Multivariate statistics is quite a general term. The others are quite big and/or basic areas too.

@Tim No.

@Tim Statistics is fundamentally an applied science. The point of the field is to apply it.

But statistics is not for everyone. One statistician told me that you need to enjoy analyzing data. I think he was at least half right.

It's not the most intellectually elevated area. And it can often seem messy and/or boring.

Of course, if you have the freedom to choose what you work on, it's much better. Not everyone does.

You do need to be comfortable doing computations. Usually writing code to do so.

Andras Deak

10:27 AM

@FaheemMitha nice!

@FaheemMitha to be fair TikZ is a lot more than tex

Faheem Mitha

@AndrasDeak TikZ is TeX. I'm not sure what you mean.

Andras Deak

I mean that it's a package with a 1300-page manual. An ecosystem of its own. Sure, it's tex, but I also wouldn't say that keras and pytorch etc. are "python" for ML :)

If you want to do that you have to be more than a tex expert, you have to be a tikz expert.

Tim

10:44 AM

I found the books on multivariate statistics and books on machine learning cover the same thing.

Andras Deak

11:01 AM

@Tim then you can cross one book from your long list

Faheem Mitha

11:28 AM

@Tim No, they don't. They're completely different areas.

There might be some overlap, I guess. Machine learning is a pretty big area. And multivariate statistics isn't so much a topic as a classification.

@AndrasDeak It depends on what you mean by "a lot more than tex". TikZ is part of the TeX ecosystem.

To some extent it is its own thing, sure. Like say Pandas or some other sufficiently large Python library.

Or like Ggplot2 for R.

Andras Deak

@FaheemMitha you might disagree with my assessment, but that's still what I meant :P

Faheem Mitha

@AndrasDeak Hard to disagree with such a subjective statement.

For a TeX user you don't seem very active on TeX SE. One question.

With an answer by Ulrike.

Andras Deak

Yeah, I'm most of an end user. And I haven't used too much of it lately. I've mostly brushed off my knowledge to abuse beamer layouts

my ugliest child :D i.stack.imgur.com/twsdj.png

I can always solve my problems by googling. That one question I have is an exception, which I first asked in chat

Faheem Mitha

Yes, one should only ask a question if the answer isn't already available.

But answers to the questions I ask are sometimes not available.

@AndrasDeak Looks good.

Currently wondering how I should save a Pandas Dataframe to disk. matthewrocklin.com/blog/work/2015/03/16/Fast-Serialization

Sigh. Managing data is very glamorous.

Andras Deak

It's very bad from a teaching perpective. But I added a dozen overlays to show the students one part at a time. Made it acceptable.

Faheem Mitha

11:44 AM

@AndrasDeak What is bad from a teaching perspective? The slide?

I wonder if I should just use CSV. It's boring and uncool, but at least human readable.

Andras Deak

@FaheemMitha yup

Faheem Mitha

@AndrasDeak What's the course topic?

Andras Deak

11:58 AM

Statistical physics

@FaheemMitha my impression is that if csv is an option then anything works. But I'm not a big data guy.

Faheem Mitha

@AndrasDeak Ok. Graduate or undergrad?

Andras Deak

Under

Basically intro

Faheem Mitha

12:16 PM

Ok.

I wonder if I should use a tab as separator or not. I haven't in the past, but a comma is hardly the perfect separator.

ilkkachu

12:37 PM

Whitespace separated text files (the way awk interprets them by default) are nice in that you can add whitespace to have the columns line up for human reading. (Or just tab-separated if you need spaces in the values.) With commas, nobody ever does that, and even if you did, the commas would look ugly there (plus the comma has other meanings too, the semicolon would be a much better separator)

terdon

@FaheemMitha Do. For the reasons ilkkachu mentioned. It just makes everything so much easier when you have tabs so you can also have spaces within the fields if needed. I almost always use tab-separated files for storing data.

Faheem Mitha

@terdon Tab doesn't look too bad, actually.

terdon

yeah, it comes out almost aligned most of the time with minimal effort

Andras Deak

1:13 PM

but then you no longer have a csv ;)

ilkkachu

I've heard it called TSV then. Not like CSV would be that accurate even with commas if quotes are still allowed.

terdon

@AndrasDeak Why would you need one? Any tool that can read CSVs I've ever heard of can also read TSVs.

Andras Deak

plus the only tool you need is vim

terdon

Vim? Never heard of it. Is it some sort of cheap emacs clone?

:P

ilkkachu

Hmm, Microsoft Excel: Save As... -> "CSV (Comma delimited) (*.csv)" -> Save. Then the result looks like this: 1;abc;;;

terdon

1:25 PM

Although I do all my work in awk really

Andras Deak

awk is very nice

ilkkachu

Those aren't commas...

Andras Deak

Doesn't excel offer you to choose an actual separator?

ilkkachu

I thought it did, but I didn't find it in this version. At least not in the Save dialog. Tab-delimited was there as a separate option, though.

Andras Deak

Have you considered switching to linux? ;)

ilkkachu

1:29 PM

Well, I have one right next to that other one.

But, whatever driver Xubuntu is using for that old Radeon card, the GUI lags. Like way more than I've ever seen on Windows. And stuff like that is a pain to debug. Also most games run nicer on Windows ;)

Of course, Freeciv runs on Linux too (haven't checked other unixen)

Andras Deak

as far as games go I've only tried openttd

ilkkachu

I remember some friend playing that years and years ago

Andras Deak

Loved it, and the open one is as good as the original.

ilkkachu

I've never touched it. Mostly I play stuff from the light and indie end.

And let RMS or anyone say what they will, I don't mind the fact they're not free :D (maniacal laughter)

Freeciv is a pretty good Civ 2 clone, too. There's some slower games running in longturn.net

slower as in one turn per day, but rules modified for more action within a turn

Andras Deak

I'm not much of a gamer at all. Spent a lot of time playing Red Alert, Total Annihilation and Transport Tycoon in my childhood. That's probably it. Well, apart from some ancient gems such as F29 Retaliator and Stunts

ilkkachu

1:41 PM

excellent stuff, most of that :)

terdon

1:53 PM

@ilkkachu Real civ also runs on Linux. Civ VI for example.

But man... freeciv. That takes me back! It was even better than civII as I recall.

ilkkachu

@terdon yes. Though given my experience on Linux desktops, as mentioned, I'm not that ecstatic to try...

terdon

2:18 PM

@ilkkachu It works absolutely fine. I've been playing it on Linux since it came out.

Had a couple of issues at first, but they were easy to fix and I've spent more hours than I care to admit playing it.

Faheem Mitha

3:07 PM

@ilkkachu Excel is generally best avoided unless you can't.

And I'm not a fan of spreadsheets generally. But I'm sure I've said that here before.

ilkkachu

3:19 PM

Excel is actually quite fine, compared to many other things, IMO.

Within limitations, naturally. Building a very complex application on top of Excel formulas and macros might not be "fine" any more.

Andras Deak

I'd be the last person to praise microsoft, but excel and word are not bad at what they're for

not sure if excel is still intentionally buggy to be compatible with ancient software, though :P

terdon

3:37 PM

Grrrr. Excel is the work of the devil.

Worst bloody tool in the world. Although, OK, not really the fault of Excel. People insist on using it for things it isn't designed to do. Especially in science. There are people who think Excel is the right tool to use for complex math, plotting data, you name it. And it can do it, barely, but really not well.

Andras Deak

Yeah, that's not excel's fault. Same way that not a single person in administration seems to know that tabulators are a thing in word.

Need to center something? Use spaces! Need a dotted line? Kill that period key!

and never press it a multiple of 3 times because a series of ellipses would look too acceptable

terdon

I hate word. I dislike all programs that assume I don't know how to type and change what I typed to their vision of what I should have typed.

Andras Deak

Do you mean actual spelling, or typesetting?

terdon

And writing large documents in it is just a pain. Although to be fair, I haven't tried that in 20 years or so, not since discovering LaTeX.

@AndrasDeak Both. Either. Any.

If I wrote aBa, I don't want you to change that to aba, or ABA or ABBA.

Andras Deak

because I'm pretty sure you can disable all autocorrect shenanigans with a click

ilkkachu

3:42 PM

with a number of clicks, at least.

terdon

Yeah, you can, but I shouldn't have to :)

Andras Deak

Yeah, maybe 3? Something like...tools, spellcheck/autocorrect, and there are probably a few tickboxes in there

I wouldn't know, I don't use it

terdon

And it's easy to miss if you just open word once every decade. You haven't bothered to change the setting and it just silently changes what you typed to something else. The gall!

ilkkachu

but yes, most of them are rather annoying when on by default, esp. since it's not that obvious how to undo the effects. (you can use 'undo' but it feels a bit weird)

Kusalananda

@terdon My current project at work involves transforming a 10 MB Excel spreadsheet with about 10 years worth of rabbit breeding/breeders data to a PostrgeSQL database! You can't even begin to understand how frustrating this is.

terdon

3:44 PM

It's often an issue for me since there are various terms in bioinformatics, from gene names to tool and database names that use specific capitalizations.

Andras Deak

@terdon So imagine my face when I was told by a journal that I should submit my response to their proof of our paper in an adobe-annotated pdf

terdon

@Kusalananda Oh believe me, I can. I really, really can.

@AndrasDeak eeek!

ilkkachu

Word is way worse than Excel, though. And not just because of the easy autocorrections.

Andras Deak

had to install windows in a VM just for that

I don't even install adobe on my linux because the last time I did that 10 years ago it acted just like malware

ilkkachu

But really, Excel allows that run-of-the mill administrator to actually do some smart-ish number crunching even without knowing programming.

terdon

3:45 PM

Excel also corrects stuff. And that's a big problem in biology. People give lists of gene names as supplementary material for a paper, for instance, and in their wisdom, they save them as an excel file, and in its wisdom, Excel auto-"corrects" the names.

Kusalananda

genomebiology.biomedcentral.com/articles/10.1186/…

Andras Deak

And the dates too, I bet. Or things that look like dates. At least google spreadsheets dose that.

terdon

@ilkkachu Sure. And if we're being serious, it's pretty good for that sort of thing. I don't really have an issue with Excel, I have an issue with the people who insist on using it for bloody everythying!

@Kusalananda exactly

ilkkachu

@terdon yes, well, there one might consider using a dedicated tool. Or at least learn to use Excel that much what is needed to store strings as strings.

terdon

> The problem of Excel software (Microsoft Corp., Redmond, WA, USA) inadvertently converting gene symbols to dates and floating-point numbers was originally described in 2004 [1]. For example, gene symbols such as SEPT2 (Septin 2) and MARCH1 [Membrane-Associated Ring Finger (C3HC4) 1, E3 Ubiquitin Protein Ligase] are converted by default to ‘2-Sep’ and ‘1-Mar’, respectively.

ilkkachu

3:46 PM

it's the same with phone numbers. They're not numbers. They're strings.

Andras Deak

I suspect excel supports literals with a leading '...

ilkkachu

paint cells -> right-click -> "Format cells" -> "Text" -> OK

terdon

Sure, you can do it properly. But people insist on using a screwdriver (excel) to hammer a nail. So yes, it's possible to do, if you're careful, but it really would be wiser to use a hammer instead.

Kusalananda

> Journals that had the highest proportion of papers with affected supplementary files were Nucleic Acids Research, Genome Biology, Nature Genetics, Genome Research, Genes and Development and Nature (>20 %).

terdon

And then they complain too. "Booo, why won't your tool let me download my list of 5000000 results in excel?".

Um. Because the excel format can't deal with that? And it would break your computer if you tried anyway?

Kusalananda

3:49 PM

:-)

ilkkachu

@terdon well, you probably could hammer a nail with a screwdriver in some situations. But just try it with a screw and a hammer!

Andras Deak

@Kusalananda ugh

@ilkkachu when I was a teen I hammered a screw once...

ilkkachu

hmm, what is the row limit in excel currently...

Andras Deak

there were nice sparks and nobody lost an eye

ilkkachu

I didn't expect sparks.

Kusalananda

3:52 PM

@ilkkachu support.office.com/en-us/article/…

> 1,048,576 rows by 16,384 columns

ilkkachu

yep, I just found it.

So sadly, no, those 5 M lines indeed do not fit. :(

I think it used to be 65535 (or 65536) rows in old versions of Excel, and there I would have guessed because of a 16-bit number. But now, I'd guess they've just raised the limit "high enough".

Andras Deak

20th and 14th, respectively (powers of 2 I mean)

not exactly round :)

Kusalananda

Hmmm.... 16384 columns of at most 32767 characters... You could fit 5M lines if they were transposed.

ilkkachu

well, at least it's not because they wanted to shoehorn both into 32 bits.

Well, there you have it :D

Kusalananda

In fact, you would fit 512M lines.

It would only be good for storage though. You wouldn't be able to do much with it.

Andras Deak

3:57 PM

"I'm working with big data, I have to transpose to fit into an excel sheet"

coolest kid on the block :D

ilkkachu

someone will have nightmares after this.

terdon

@AndrasDeak You think that's a joke. But it isn't, it really isn't :(

I do work with big data and that's exactly the kind of thing some users want to do.

Andras Deak

I'm sorry for y'all :(

Kusalananda

Restrictive environments breeds innovation.

ilkkachu

Kinda like how people implement elementary binary logic circuits in the Minecraft world and such

Cute work, nice little waste of time and all that, but for practical applications there might be better ways.

Andras Deak

4:00 PM

or game of life implemented in game of life...

ilkkachu

@AndrasDeak well that does raise a bit of an eyebrow. I think there are some meme pictures online about how my face actually looked.

but yes of course. Why not. If it can be done...

Kusalananda

... or implement an OS on a machine with 4-64K word RAM.

PDP-7

The PDP-7 was a minicomputer produced by Digital Equipment Corporation as part of the PDP series. Introduced in 1964, shipped since 1965, it was the first to use their Flip-Chip technology. With a cost of US$72,000, it was cheap but powerful by the standards of the time. The PDP-7 is the third of Digital's 18-bit machines, with essentially the same instruction set architecture as the PDP-4 and the PDP-9. == Hardware == The PDP-7 was the first wire-wrapped PDP. The computer has a memory cycle time of 1.75 µs and add time of 4 µs. I/O includes a keyboard, printer, paper-tape and dual trans...

ilkkachu

80's home computers

some embedded devices today

Andras Deak

@ilkkachu oh yes. Glorious and terrifying youtube.com/watch?v=xP5-iIeKXE8

2

ilkkachu

depends on what it has to do, of course.

@AndrasDeak indeed. But it would have to be. There has to be an edit of that video somewhere where it loops back to the start from the end, revealing another layer of the same

Andras Deak

4:11 PM

(wasn't clear whether you're familiar with it already)

@ilkkachu no no no :P

ilkkachu

No, I hadn't seen it, thanks for sharing it. Now I'm not sure if I want to know how it works.

Andras Deak

If you want to look into it I'm only aware of the keyword: the OTCA metapixel. Never had the courage to go down the GoL rabbit hole.

ilkkachu

yep, I have other things I should do, too...

Faheem Mitha

4:30 PM

@Kusalananda I imagine you need to do a lot of data cleaning and validation.

Do you add a lot of checks on the PG side to weed out stuff?

It looks like Matplotlib is (probably) still the best way to plot in Python.

I wonder if should experiment with a pure TeX approach.

Andras Deak

My experience is that pgfplots is blood and tears but might be totally worth it if you want publication-ready figures and have two weeks to do that. If the result is more mundane or there's less time matplotlib is the next best thing.

...speaking as a python user, so take that claim within the bounds of python (but now I see you're also only asking about python so it's OK)

Faheem Mitha

@AndrasDeak No publication intended. I'm just trying to look at some financial data visually, that's all.

R is the other obvious possibility, but it makes me break out in hives, so there is that.

Andras Deak

yeah, I suggest matplotlib

pretty plots fast, especially if your data is already in python

Faheem Mitha

@AndrasDeak TeX isn't suited to data manipulation, but one could just feed it data, possibly via the Lua backend.

Anyway, matplotlib has had a TikZ backend for a long time, so there's that.

Andras Deak

Well, yeah. That's part of the blood and tears. I used to prepare my data from MATLAB, generate coordinates from that and just dump that into pgfplots.

@FaheemMitha it has?

Faheem Mitha

4:43 PM

@AndrasDeak Lua would be better than dumping and loading.

Andras Deak

Ah right, "PGF backend". I looked at it once, when I learned about matplotlib2tikz

Faheem Mitha

@AndrasDeak Yes, it's been there for a long time. I don't know how much it is used.

@AndrasDeak Right. Maybe it's called PGF backend. I don't remember.

Though Lua couldn't talk to Matlab.

Andras Deak

I'm happy with pdf exports so I never had to try

Faheem Mitha

@AndrasDeak Is Pandas as good as it gets for Python dataframes and the like?

It seems unlikely there is anything better. Or even a real alternative.

Andras Deak

If you mean pandas dataframes then yes ;) I have only tangential experience with pandas, don't use it myself. And haven't seen any other data libraries, such as dask or pyspark, so I can't comment.

Faheem Mitha

4:46 PM

@AndrasDeak Having TeX in the loop can be helpful sometimes. Particularly if embedding inside a TeX document.

Andras Deak

But yeah, pandas has become the go-to data library. My main impression is that it can do a lot of things, but there are too many ways to do things and too much fluff in the API.

Faheem Mitha

But what I'm doing right now is just for me, so...

Andras Deak

also way too much garbage code on Stack Overflow

Faheem Mitha

@AndrasDeak Unfortunately, that's a common disease. R has similar issues. Probably worse.

Andras Deak

at least my problems are within numpy so I'm happy

Faheem Mitha

4:47 PM

Plus R is a pretty wacky language, considered as a language. Last I checked, it still doesn't have proper tracebacks. You think they could manage that after 30 years.

@AndrasDeak Do you do numerical work?

Andras Deak

yup

Faheem Mitha

Research stuff or consulting stuff?

Andras Deak

Research.

Faheem Mitha

Ok.

Kusalananda

@FaheemMitha Parsing the manually curated data is a pain, and it's there we catch most of the manual errors. Once it's parsed and loaded into the database, all is well and we can enforce constrains automatically. We're doing this so that the group curating the data can start working with a custom web interface instead of with an Excel sheet.

Faheem Mitha

4:54 PM

@Kusalananda You don't use db validation to catch errors?

Kusalananda

You can't do that with data in an Excel sheet.

Later, when they start using the web interface, yes, that's what we do.

Andras Deak

sounds like definitely an investment

Faheem Mitha

@Kusalananda Not directly. But you could export it to csv/tsv or something, then try to import that into PG. Of course, you'd have to add suitable validation to the PG code.

I did such things once upon a time. It wasn't pleasant.

Biological data tends to be full of errors. Very dirty.

Andras Deak

that's soft sciences for you ;)

Faheem Mitha

I was checking some public insurance claim data recently, and that was full of errors too.

You'd think multi-million dollar companies could manage to have their public data pass basic sanity tests.

Kusalananda

5:03 PM

@FaheemMitha I'm exporting to CSV already, but that data is in no way fit to be directly included in a database as a single table. It needs to be normalized and the data contains various measurements and encoded stuff. Just a simple thing like each individual has a mother and a father (each referenced by their "number" in each offspring) has to be untangled and checked.

Andras Deak

Are you spared character encoding and timezone problems at least?

Kusalananda

@AndrasDeak Yes, that bit is the easy part.

Faheem Mitha

@Kusalananda Sure. I meant, you could use a frontend thingy like SQLAlchemy to read it in for you and do suitable checks.

Not necessarily as a single table. I'd assume some kind of schema.

Andras Deak

I suspect the problem is that a lot of the data is "not even wrong"

Faheem Mitha

I find automated checks useful for spotting problems. Of course, some unfortunate human still needs to fix them.

They should pay people a lot of money for messing around with data. Because it's usually no fun.

But I doubt they do. They probably just get grad students to do it.

Andras Deak

5:07 PM

You mean indentured slaves? Sounds good!

Faheem Mitha

@AndrasDeak It's not considered polite to call them that.

ilkkachu

Yes, you're not allowed to do that any more.

Andras Deak

On a more serious note, I only have perspective on some theorist and experimentalist PhD students. But the latter can get pretty bad: supervisors forcing the students to work on industrial projects for profit, rather than progressing with their PhD. Pretty bad.

I'm sure it varies a lot both with field and with supervisor

Kusalananda

@FaheemMitha In this case, we report issues back to the data providers and they fix it.

Faheem Mitha

@Kusalananda That's nice.

@AndrasDeak Lots of bad things happen in academia.

Andras Deak

5:22 PM

yeah, just ask Tim

Faheem Mitha

@AndrasDeak Read Academia SE. If a fraction of that stuff is true, it's pretty hair-raising stuff.

Andras Deak

Well, it's inhabited by people (and people are the worst), occasionally in environments that incentivise cutthroat behaviour, with few checks and balances on the lower levels.

Faheem Mitha

@AndrasDeak Yes, that's an adequate summation.

Well, except for "occasionally".

Also, I'd substitute "very few" for "few".

Andras Deak

@FaheemMitha That's too much of a generalization to be applied without a qualifier. In my immediate environment there's no kind of push to trample on others. A lot depends on local culture (by which I mean funding of research and whatnot).

Faheem Mitha

@AndrasDeak Ok. Well, I've seen a lot of abuse in my time. But I accept things may be different elsewhere. And maths is relatively free of such things. Of course, it has other disadvantages.

Andras Deak

5:30 PM

Plus my impression is that one's local experience is very strongly affected by one's boss. A good boss can let their subordinates thrive, and a bad boss can make their subordinates' life miserable. And it's also my impression that in academia this effect is much stronger, because you are very strongly coupled to your boss. So it's understandable to an extent that bad people in academia make for terrible bosses who do outrageous things to their subordinates.

but we rarely get "my supervisor is amazing" stories on academia.SE, so there's also a lot of selection bias

Faheem Mitha

@AndrasDeak Yes, that's true, of course. Also academics, particularly temporary/junior ones, don't have the same kind of job related protections that regular people do.

@AndrasDeak Yes, there is obviously selection bias. But I also don't think there are a lot of splendid supervisors around.

Math is a relatively sane corner of academia. Which may be why you haven't seen much of this kind of thing yourself.

Andras Deak

Ah, but I'm in physics :)

And I'm not saying it's not there, just that I'm lucky enough that my immediate environment is decent.

but this also implies that the problems are not universal

Faheem Mitha

@AndrasDeak Mathematical physics?

Andras Deak

No, just...physics. Computational condensed matter.

Faheem Mitha

@AndrasDeak It's definitely there. It's a huge and well-known problem.

@AndrasDeak Oh. So you do a lot of number crunching? What are your go-to tools? Fortran?

Andras Deak

5:43 PM

For the actual calculations yes, fortran (also note my similar-to-tex lack of fortran footprint on Stack Overflow :P). And I do all my post-processing in python. "Number crunching" sort of works, but it's not big data. Call it small data.

Faheem Mitha

Of course, huge is relative. We have climate change, nuclear weapons. Most people in the world are a paycheck away from homelessness or similar bad things. The world economic system is a catastrophe. So in that context, academic abuses perhaps are not so big.

@AndrasDeak So you don't need help with the Fortran?

Andras Deak

(I'm exporting matplotlib figures for work as we speak. And hah, I just noticed a "PGF for Latex" option in the export dialog)

Faheem Mitha

@AndrasDeak Works if you are using the figures in TeX. For example, the fonts match.

Coincidentally, I'm also trying to make a matplotlib figure.

Not working yet. It's been a while. I get a blank display. Maybe I need to do a close.

Andras Deak

@FaheemMitha not usually, no. But the code base is already there and I rarely have to develop it, and when I do I don't need anything fancy. The one real question I have on SO (the one I registered for) is roughly my first encounter with undefined behaviour, which I ran into during develoment.

@FaheemMitha if you need help let me know

Faheem Mitha

@AndrasDeak Thanks. I expect I'll manage. I'm lazy. I try to get my answers from SO answers, which isn't a great method.

Time to read the actual docs, perhaps.

Andras Deak

5:49 PM

There are some terrible posts out there.

although matplotlib is not as bad as pandas in this regard

Faheem Mitha

Indeed there are. Perhaps it doesn't like the dates on the x axis. What's the recommended way to handle that? No Python complaints though.

Andras Deak

Depends. Do you have actual datetimes?

Faheem Mitha

@AndrasDeak Oh. No, I guess I should convert to datetimes.

Andras Deak

Pyplot should support datetimes. If you have a blank figure make sure you're not setting any manual x limits.

If you have strings as data you get categoricals, i.e. ~enums, so stuff gets plotted at integer indices.

Faheem Mitha

Well, with a blank plot, that's clearly not the only problem. :-)

Currently I just have a list of strings that look like:

2020-05-15

Andras Deak

5:53 PM

@FaheemMitha well I've seen that. Users sets their limits from 100 to 300 because they are plotting prices in dollars. But it's left as strings so the actual data points range from 0 to len(data). The result is a blank plot.

Faheem Mitha

@AndrasDeak I see. Well, maybe that's the problem.

Andras Deak

plus the points would not be sorted, which can lead to extra fun results :)

If you convert to datetimes and temporarily remove limits it should start working, or at least you should see something. Then you might have to use some date formatter to get ticks where you want them (I've never plotted dates)

Faheem Mitha

@AndrasDeak Right.

@AndrasDeak Well, financial data are generally time series, which I have little experience with. I did take one crap course in grad school.

Ok, after converting got a graph.

Tim

9:23 PM

@FaheemMitha which reference books for your time series course?

Transcript for