« first day (1711 days earlier)      last day (3537 days later) » 

01:54
@Basj what's on your mind?
Hi @Anthon. What's happening?
Not much, cannot sleep, so I prowl the nets for things to edit or review
02:17
@Anthon That sucks. I know about the not sleeping thing.
02:35
@FaheemMitha I norally would read in bed, but my gf has the early shift, needs all the sleep she can get and while reading I might wake her up.
Done any interesting pythoning recently?
@Anthon You should come over to AU if you can't find anything here, we've always got things to edit and review :P
02:50
@Seth Thanks for the invite. I'm not sure why I spent so little time there. It might be the subconscious affront the orange theme poses for my latent monarchistic tendencies.
@Anthon I see.
@Anthon Not really. I did fix a depreciation warning in biopython a few days ago. They changed a function to handle large data better, I think. That was kind of a pain. And I'm seeing a SQLAlchemy warning in the same place I don't understand. Haven't fixed that one yet.
@Anthon o_O
;p
@Anthon I think you should stay here. :-)
Hi @Seth. Stop trying to lure him away. :-)
@Anthon is your gf a nurse or a doctor?
@FaheemMitha Hi! :P
@Anthon I don't get the orange reference.
@Seth What's happening?
02:58
Watching some penetration testing videos, playing some chess.
And talking to you ;P
And you?
@Seth Penetration testing videos?
@Seth Proofreading this freaking letter.
@FaheemMitha ya
@FaheemMitha :/
I hate writing..
8.30 am here. I'm trying to sync with the rest of humanity for the moment.
Which reminds me.. I have some to do D:
@Seth Well, here, it's more the content that actual writing that is the problem. Some nasty legal things.
Also, a url is getting cut off. Have to fix that.
LaTeX stuff.
I VTC this one as a programming question:
0
Q: How to fix Python ValueError:bad marshal data?

el_SalmonRunning flexget Python script in Ubuntu, I get an error: $ flexget series forget "Orange is the new black" s03e01 Traceback (most recent call last): File "/usr/local/bin/flexget", line 7, in <module> from flexget import main File "/usr/local/lib/python2.7/dist-packages/flexget/__init__.py", line...

@FaheemMitha do you think that SO guys would answer that?
@Braiam No idea. Do you think it is unsuitable for SO?
@FaheemMitha is definitively not about coding something... is more akin to "here's my core dump, find and fix the problem" which nobody likes
@Braiam Hmm, fair enough. Should I change my close vote? And if so, to what? It is definitely off topic here.
03:25
@FaheemMitha no nurses/doctors don't carry a gun
The House of Orange-Nassau (in Dutch: Huis van Oranje-Nassau, pronounced [ˈɦœy̯s fɑn oːˈrɑɲə ˈnɑsʌu̯]), a branch of the European House of Nassau, has played a central role in the politics and government of the Netherlands — and at times in Europe — especially since William I of Orange (also known as "William the Silent" and "Father of the Fatherland") organized the Dutch revolt against Spanish rule, which after the Eighty Years' War led to an independent Dutch state. Several members of the house served during this war and after as governor or stadtholder (Dutch stadhouder) during the Dutch Republic...
04:03
@Seth the other problem is that my rep on AU is so low I cannot review anything and only suggest edits
@Anthon Ah, a European political joke. Went over my head, I'm afraid.
@Anthon You could probably bring up your rep pretty fast. Not that I recommend it.
04:26
@FaheemMitha Somehow I assume the whole world associates dutchmen with orange colored soccer-hooligans (not that I watch soccer or any other sports)
@Anthon You do the whole world way too much credit.
In assuming they know much about anything.
@Anthon That's easily fixable, just ask @terdon! :D
He now has more rep than I do >.>
Which obviously means I have more of a life, right? Right? ;p
@Seth just answer regex questions on SO... you will feel dirty, but you will have more rep than terdon anyways
@seth I am sure Terdon has a life as well.
Getting to the top of a review queue on AU would be a challenge....
04:58
What is a suitable tinyurl thingy? One that won't be blocked by sites?
is bitly better than tinyurl?
05:12
@FaheemMitha Which sites are blocking those? SO?
@Anthon That's hypothetical blocking.
As in, I wouldn't want the url to be blocked.
 
8 hours later…
13:01
 
2 hours later…
14:37
The gzip format represents the input size modulo 2^32, so the uncompressed size and compression ratio are listed incorrectly for uncompressed files 4 GiB and larger. To work around this problem, you can use the following command to discover a large uncompressed file's true size: zcat file.gz | wc -c
I wonder how far off it is?
@Seth Probably. Sadly. :P
15:10
@mikeserv yes, gzip was written in 1992. Newer compression formats (bzip2/xz) don't have that low a limitation and some (like xz) can allow random access.
@StéphaneChazelas Yeah. It's a shame - that was pretty easy that way. I can't figure on how else to do it otherwise. I can't even tell what the other answers do, how are they splitting the output files reliably? If the top half of the file has a more regular data set than the bottom half, then the top half would be far smaller, I think.
I tried faking it - just piping the gz header at zcat - like the first 10 bytes (and several other amounts) and then sending the next Mb to >/dev/null, and then catting the rest at it, but it just kept telling me input violates the format.
@Braiam Thanks, but it doesn't seem terribly relevant. And I wasn't planning to use url shorteners on this site, anyway.
 
1 hour later…
16:23
@Anthon I certainly and unequivocally associate orange with dutch. I also have first hand experience with the football folks, more with the nice than the hooligans though (I encountered them in force in Bern, Switzerland during Euro 2008)
 
1 hour later…
17:42
@casey You met nice dutchman there? That a new definition for "Das Wunder von Bern" :-D
 
3 hours later…
20:51
@mikeserv What are you trying to do?
3
A: Fastest way of working out uncompressed size of large GZIPPED file

Stephen KittI believe the fastest way is to modify gzip so that testing in verbose mode outputs the number of bytes decompressed; on my system, with a 7761108684-byte file, I get % time gzip -tv test.gz test.gz: OK (7761108684 bytes) gzip -tv test.gz 44.19s user 0.79s system 100% cpu 44.919 total % ti...

I actually saw that earlier today - but I forgot to upvote it then. That is pretty cool. It's just that, well, that's what I was trying to avoid doing. You should offer that as anwer though.
0
A: How to efficiently split up a large text file wihout splitting multiline records?

mikeservI don't think you can do this - not reliably, and not the way you ask. The thing is, the archive's compression ratio will probably not be evenly distributed from head to tail - the compression algorithm will apply better to some parts than others. That's just how it works. And so you can't factor...

@StephenKitt ^ that was it.
The post history will show that I initially divided gzip -l's output by 4 before Stephane let me know it wouldn't work - I hadn't tried it on a 4gb file.
@mikeserv Ah right, I see...
So while the 4 line thing is cake - and it needn't even be blocked at a specific byte boundary to start with either (as dd can conv=block anyway) the 25% of stream thing just can't be done, I don't think. And really, I don't think that guy should, anyway.
No, I don't think it's possible either, not with gzip anyway.
21:10
Yeah, that makes two of us.
It's not clear from the question though whether the input is supposed to be processed in compressed form or not; the Python answer just works on the uncompressed file.
21:39
@mikeserv That would be possible on a filesystem that support cp --reflink and fallocate -c with 1-byte granularity.
Don't know of such a filesystem though
@StéphaneChazelas - which ones are those?
@StephenKitt - the Python answer just works on the uncompressed file? I couldn't even tell what those other answers did.
@StephenKitt - where does it put it?
btrfs can do cp --reflink, but not fallocate -c, ext4 can do fallocate -c but only with block granularity and can't do reflink yet. Don't know about things like nilfs2. One could always have a fuse filesystem on top of the full uncompressed file to fake the 4 files though. Though that'd be cheating.
Yeah. fuse is always cheating.
But on a btrfs filesystem you would just mark the file for gz compression and forget about it.
@StéphaneChazelas - is it ext4 that does the -c thing - is dig holes or something? I was look at that man page the other day.
I actually did man $(pacman -Qil util-linux | sed -n 's|.*/usr/bin/\(.[^ ]*\)|\1|') and just tabbed through all of them.
22:03
@StéphaneChazelas - dang, man. You're cryptic sometimes. How could you do it with fallocate or --reflink? You'd still have to uncompress it, wouldn't you?
@mikeserv Yes, I just mean make 4 reflinked copies of the uncompressed file (and record the line number/size... upon uncompressing) and collapse the unwanted parts in each of those files.
lwn.net/Articles/587819 for fallocate -c
@StéphaneChazelas - yeah, that's genius. collapse - that was it.
@StéphaneChazelas - what's i_size - is that for inode?

« first day (1711 days earlier)      last day (3537 days later) »