Yeah. Although haven't been as active there due to school this summer. I hoped to hit 50k by the end of spring, but school work made me nope out for a while
@FaheemMitha Well, when I just started out with Linux and learning *nix over all, I went on a spree of trying out different distros, virtual machines and real installs. So no real reason, just learning and exploring.
@overexchange zgrep -ao '[^[:print:]]' test.tar.gz | od will grep all non-printable characters out of test.tar.gz and feed them through od. I get nul characters (\0) back, which are not ASCII.
@overexchange The tar archive format uses non-ASCII characters to create dividers between the files in the archive.
Anybody have an idea for some kind of statistical computation for Linux in general ? Going to be learning Matlab and R on my own this summer, so trying to find some kind of project that involves Linux
@Kusalananda Well, for the most part I do text processing stuff. Made a couple small indicator applets for Ubuntu's Unity desktop, one of which involved displaying disk usage for plugged in drives, so that's something I'd be interested in - block devices, drives, etc. Networking stuff is interesting but I'm not very strong in it - so that's something I'd be willing to pick up as a challenge to raise my level.
@SergiyKolodyazhnyy Well, that may be interesting. Maybe look for trends, extrapolation into the future and stuff like that? Frequency analysis over a few weeks of data?
Or an outdoor temperature sensor would be even more interesting?
I'm actually electrical engineering student . . . I've some experience with Matlab, but not enough to have a clue of what I'm doing. Hence the goal is to learn the ropes via a project. Plus the closer I get to graduation, the more I think about diversifying my skills. Data Science is pretty popular right now, so that's another thing I'm considering
@Kusalananda actually, that's a pretty good idea. I didn't even think about spreading it over a few weeks.
So, summary: I think you can do something simple that may be fun with a temperature sensor, definitely. To gain real stats knowledge requires some study. Faheem thinks you need a proper class, and I think you can probably pick the basics up without one.
@SergiyKolodyazhnyy There's some decent stats / R courses on EdX.org, for free that I would recommend. I've just done the "PH525.1x Statistics and R" one.
There was one stat course I took once I might have failed if I stayed in it. I dropped out. The teacher was terrible. He was old and completely out of it. They shouldn't have let him teach. I remember I got like 50% in the first test, because I couldn't make sense of the material. I was so humiliated.
@Kusalananda LWN regularly reports on kernel statistics. Who wrote the most lines of code, that kind of thing. But I was thinking of something rather deeper than that.
I know C, but not on the pro level. Only have been working PIC18 microcontrollers for a class and wrote a very very basic "ps" program for final project in C class itself
@FaheemMitha It's all linear algebra. Didn't understand most of the more difficult bits (it was my advisor's research), I worked mostly on the implementation.
Spread the word. In less than a week, this site will undergo a private beta evaluation. I had to postpone the pro tem nomination thread — typically posted today — because there simply isn't a large enough community signed on to support this site.
We need to increase your numbers. Typically we ne...
@terdon Scanning through the topics, it looks like this may be more useful for my colleagues than for me. :-/ I'm just the Unix wiz kid the real bioinformaticians call upon when they need me to do debugging or infrastructure.
@Kusalananda Well, I think there's at least one more person from that side of things:
The way the word bioinformatics is used these days pretty much suggests any part of biology that involves the use of computers to process data. Once upon a time bioinformatics dealt with the best ways to organised, store, and cross-reference biological data. It was fair to call it subfield of informatics. But most bioinformatician these days wouldn't know the first thing about databases, and as commonly used, itis a subfield of biology. — Ian SudberyMay 22 at 23:11
Also, that's great! The main issue we have here is precisely that the userbase is too small and there's not enough variety,
I would call infrastructure questions on topic (as long as they're somehow specific to a bioinformatics lab), so it'd be great to have someone with your knowledge.
I worked in a computational biology group for a bit. And also one of those awful clinical trials groups. Ugh. In both places I did stuff one could call bioinformatics.
If you know statistics and can program, it's quite likely you'll end up doing bioinformatics stuff. Though bioinformatics is one of those poorly defined terms. Like communist.
2
Typically biologists don't know statistics. And if they can program, they usually do so very badly. With shining counter-examples like our terdon excepted.
The only thing about bioinformatics stuff that I know is FASTA, because there's occasionally people coming with questions to parse that type of file in awk or something.
It is an interesting idea, biomart. We spent so much time working out a highly normalized database schema for the (ordinary) core databases, and along comes Arek and denormalizes everything! The mart databases are huge, but also blazingly fast to query.
I used to use Ensembl every day from around 2002 to 2010 or so, but have barely touched it since then. So I'm not up to speed on whatever they're up to these days. Not past visiting a few times a year to look at something specific.
I used UniProt and geneontology.org for most of my post doc. Don't really use any such resource in my current job. Weird, not that I think of it.
It's been a while.
How large are the Enseble data @Kusalananda? I mean the whole thing. If I wanted to make a full mirror of Ensembl, how much disk space would I need?
I would be very surprised if it isn't several TB at least. Although I might be getting too used to the volumes of data needed for next generation sequencing. But still, if our little startup already needs ~3T for our data, Ensembl must be a few hundred TB, I'd guess.
I'd like to learn which format it's commonly used for storing full sequence human genome (4 letters without quality score) and why.
I assume storing it in plain format would be very inefficient. I would think more about binary format like 2 bits per nucleotide.
Therefore, which format would be ...
@terdon Note that that's for setting up a complete set of databases and the web site only. Running the Ensembl pipelines would require lots more! Lots and lots more.
