/dev/chat

7:44 AM

@Kusalananda st_size in struct stat measures a file’s size in bytes, and that’s what stat() gives... The type is off_t which is a “signed integer type”.

7:56 AM

@StephenKitt Too early in the morning. But, hmmm.... Which of the block and byte counts is the one stored on disk? Probably both?

@Kusalananda You always need the byte count in some fashion, the specifics presumably vary depending on the file system — it could be blocks + bytes used in the last block... Using block counts might result in “interesting” handling with sparse files, so I suspect most file systems store the apparent file size explicitly.

8:37 AM

@StephenKitt Thanks. So in any case, returning to @JohnnyApplesauce, it depends on the filesystem.

@Kusalananda and the size of int ;-)

@StephenKitt Well, the size given in each filesystem must be static, as disks may move between OSes (or booted with different OSes). So, for example, for ext [linux](https://www.nongnu.org/ext2-doc/ext2.html#i-size) is either 32 bit or 64 bit.

i_size

In revision 0, (signed) 32bit value indicating the size of the file in bytes. In revision 1 and later revisions, and only for regular files, this represents the lower 32-bit of the file size; the upper 32-bit is located in the i_dir_acl.

@Isaac I’m not sure I understand what you’re getting at...

I was alluding to the fact that JohnnyApplesauce asked whether the size was an int or a long; the answer to that depends on the size of an int on the given system.

(As an interface concern, not the value stored on the file system.)

8:57 AM

@StephenKitt Yes, answering in terms of ints and/or longs it depends on the OS. But a disk already formatted has some data already written that is independent of the OS that is used to read it. In ext4 it is 64 bit. Whether that is an int, long, or double I'll pass.

@Isaac of course. I was cracking a joke based on a literal take of JohnnyApplesauce’s question.

@Isaac BTW the reference for Ext4 internals is kernel.org/doc/html/latest/filesystems/ext4/index.html

9:31 AM

@StephenKitt Yes, depending on who you ask you may get (slightly) different answers :-)
However, still in both, i_size_lo and i_dir_acl each store 32 bit of the size value (taken from this kernel page https://www.kernel.org/doc/html/latest/filesystems/ext4/dynamic.html#inode-size

I find the gnu page simpler, easier to read for general reference.

@Isaac yes, I know what’s stored on disk

@StephenKitt Ooops, sorry if I disturbed you, no intent of that whatsoever. Be happy :-)

@Isaac no worries, be happy too!

12:10 PM

I used R quite a while ago, without much understanding of the language

R is a very different language to me. I am looking at its connection with other languages

Happy Wednesday, chat

Meanwhile not sure about the employability of R though

@Tim R is a sucky language, but it's very widely used.

Though since it's essentially a statistical environment, it's not so useful unless you are planning to do statistics.

From Advanced R, 2ed, 2019 by Wickman

> Because the R community mostly consists of data scientists, not computer sci-
entists, there are relatively few books that go deep in the technical underpin-
nings of R. In my personal journey to understand R, I’ve found it particularly
helpful to use resources from other programming languages. R has aspects of
both functional and object-oriented (OO) programming languages. Learning
how these concepts are expressed in R will help you leverage your existing

> knowledge of other programming languages, and will help you identify areas
where you can improve.

> To understand why R’s object systems work the way they do, I found The
Structure and Interpretation of Computer Programs 1 [Abelson et al., 1996]
(SICP) to be particularly helpful. It’s a concise but deep book, and after read-
ing it, I felt for the fi rst time that I could actually design my own object-
oriented system. The book was my fi rst introduction to the encapsulated
paradigm of object-oriented programming found in R, and it helped me un-
derstand the strengths and weaknesses of this system. SICP also teaches the

In the past, I glanced the three books he recommended but they didn't feel the way the author described.

What books or materials do you guys recommend?

I am reading about its FP and OO parts currently, because some problem that I have to solve interrupt my study of cluster managers and distributed file systems

12:30 PM

@Tim Recommend for what? And if you are trying to learn R, why?

It's not exactly recreational activity. People typicall learn/use R because they want to do things like data analysis.

Not for cultural reasons. It's a crappy language, and is something of a mess from a syntax POV.

Even one of the co-authors admitted as much.

Co-creators, sorry.

1:11 PM

Unfortunately, it is now incredibly popular in bioinformatics.

Jeff Schaller

1:26 PM

Unfortunately, bioinformatics is now incredibly popular

Recommendations on materials about R?
Employability of R besides bioinfomatics?

Disclaimer: I don't have bias against or towards working in bioinfomatics. I know it is ivory tower

@JeffSchaller Well, that one I can live with ;)

JohnnyApplesauce

2:22 PM

What is up with the meme of languages (like R, Python, and JS) that try to allow both functional programming and object-orientation?

Jeff Schaller

Fortunately, the charging one has been solved now that we've all standardized on mini-USB. Or is it micro-USB? Shit.

3:18 PM

@terdon Bioconductor. I know.

@Tim I suspect it's not possible to study everything. At least in real life, as opposed, to, say the Marvel Universe.

I'd pick something you're actually interested in, and work on that.

Stress levels rising here. Some days, I wish I drank. I hear it numbs the mind.

I am never lucky enough to stay in a field to thrive.

If I were, I would probably also chatting about days of lives and posting memes

3:58 PM

@JohnnyApplesauce Most do, don't they? I mean, if a language has OO tools, that doesn't mean you can't do functional programming, right?

Anyway, I think most of those (except JS) predate the OO idea, don't they? So the OO stuff was added later. I know that was the case for Perl.

I believe Smalltalk had an early OO implementation. Though Common Lisp might have been earlier.

the first OO language was Simula in the 1960s, and Smalltalk followed a few years later

CLOS came after that

4:21 PM

@StephenKitt Ok.

4:51 PM

@StephenKitt Well, so much for my chances of appearing knowledgeable about programming language history then. OK, but it is true that python and perl had their OO tools sort of pasted on after the fact, right?

@terdon dunno about Python, but Perl did, yes

A developer I met once went off on a long rant about how python is not actually an OO language but only has some features that let it mimic it. This was observed by another developer I know who has ~40 years experience or so and he seemed to agree.

Sadly, most of it went way over my head, but they seemed to know what they were talking about.

5:06 PM

@terdon The question is whether you view OO as a set of syntax features or a design philosophy. As a design philosophy, most languages can be used to write OO programs.

5:18 PM

@Kusalananda Wouldn't you need to have some way of allowing "objects" (presumably variables, or other data structures in non-OO languages) to have "methods"? Could you make something OO-style in bash, for example?

6:20 PM

@terdon The most common way to simulate a method is to write a function that take the object state as its first (or first few, whatever is needed) arguments, and then performs its action given that state.

6:47 PM

@Kusalananda Why would that be OO-style? Something like foo(){ [ $1 = 1 ] && echo yes || echo no; }?

7:16 PM

@terdon If the "object" is represented by an array called object in bash (which makes sense as that's the only complex datatype that bash supports), then you would call "method object other arguments here" to call a method on the object. This is exactly the same way Perl does OO. In the shell function, object may be a name reference, but that's an implementation detail.

Just handwaving now... How would one implement an object? Well, you could keep the attributes in global arrays, one array for each attribute, one element per object. The object would then be a simple index (not an array like I said in my previous comment), and all methods (shell functions) would take the index as its first argument and would immediately have access to the object state through that index.

Heck, bash even allows you to store file handles in variables.

7:41 PM

Lua has only one non-trivial type, the array.

But one can do something that mimics OO programming with it. Though it's not exactly satisfactory. But I'm not really a fan of minimal programming languages.

Philip Kirkbride

7:54 PM

Does anyone remember the news story about an old server being shut down and they found a low priority process had been waiting in queue for years?

@PhilipKirkbride Mentioned here: stackoverflow.com/a/33070088/931027 with references to the Operating System Concepts book by Abraham Silberschatz, Peter B. Galvin, Greg Gagne (with dead link).

Also found here: c-sharpcorner.com/blogs/omg-rumor-about-priority-scheduling1

... and in numerous other places on the interwebs, duckduckgo.com/…

Philip Kirkbride

@Kusalananda thanks so much!

@PhilipKirkbride No worries :-)

8:10 PM

anyone know why this command is returning literally every line even though none match?

$ printf '%s\n' *.log | parallel -j25 grep 'handled: 500'

(I know I can grep recursively but there are 100+ log files with over 50G of data total so I'm trying to speed it up a bit)

It may just be overwhelmed because testing on a smaller set of test files seems to work

@jesse_b Do you actually get every line, or just a lot of output? I'm thinking there are long lines matching...

@Kusalananda They are long lines but if I grep -r 'handled: 500' . on that directory it returns no results

Oh

The files are still being downloaded and there should only be 4 matches total

Haven't used GNU parallel in years. I might install it now and test a bit...

8:17 PM

I really like it

Oh, btw: knome.org

I took down a server at work yesterday (well just made it locked up due to load) from processing some data on it (with a poorly optimized shell script) but then downloaded the data to my mac and processed it with GNU parallel in a few minutes

ilkkachu

@FaheemMitha the table, actually. That whole idea about everything built on such a low-level primitive is one of the most annoying programming language features I know...

@jesse_b I know why it fails.

On my system it gives me an error message for each file: grep: 500: No such file or directory

that should be a hint

$ printf '%s\n' *.test | parallel grep -H 'foo'
one.test:foo
three.test:foo

8:22 PM

printf '%s\n' *.log | parallel -j25 "grep 'handled: 500'"

Note added quotes.

trying it, but it's odd that it works on the test data

No, it's not odd. It splits the arguments on spaces, and there are no spaces in your test pattern.

ahh

thanks ;)

wizard

:-)

my cpu fan is screaming right now

8:26 PM

I would report that as a possible bug, but if I was Ole Tange, I would not fix it as it's one of those typical things that would possibly break existing scripts.

that's definitely much faster than grep -r on the directory but still painfully slow

@jesse_b Probably disk-I/O-bound.

Try 4 parallel jobs instead, or 2.

I'm trying 50 now :p

more is always better

@jesse_b Well, good luck! :-)

ilkkachu

8:42 PM

@Kusalananda I think it runs through the shell on purpose, but I can't see that explicitly mentioned in the man page

@Kusalananda and umm, sure not exactly? In Perl you'd do something like $foo->method(), right? And that syntactic sugar makes it look more OO, and has the advantage of sensible namespacing for the methods.

By the way, now that I looked, the perlootut man page is a bit of a gem: "By default, Perl's built-in OO system is very minimal, leaving you to do most of the work." and "Perl's flexibility has allowed a rich ecosystem of Perl OO systems to flourish.". So that's two negatives. Reminds me of what there is in Lua, actually.

@ilkkachu I based that statement on the fact that in a Perl method, @_ will contain the object that the method was called through. $_[0] is a reference to the object.

ilkkachu

@Kusalananda yeah, you'd do something like my ($self, $arg) = @_. The implementation kinda shows through. Python has the same, you have to declare all methods with def foo(self, blah...)...

Which is kinda repetitive, but way less annoying than the Lua way where you have to make the difference on the call site, on every single call, grr...

Andras Deak

9:35 PM

It's also not universal, since for class methods the first arg is the class, and for static methods you don't have an automatic first arg at all. But that's probably beside your point.

10:43 PM

@jesse_b No, "more is always better" is not true: more stupidity is never better. ;-)