« first day (4036 days earlier)      last day (36 days later) » 

11:12 AM
In the Stevens book, he defines the following macro, and uses it all over the place.
#define MAXLINE 4096
This is (I think) intended to be the number of characters in a line. I just wonder why it's so large. In what universe are lines 4096 characters? Or is this is Unicode thing, perhaps?
 
11:26 AM
@FaheemMitha Why would you make it shorter? Isn't the idea to have it as long as possible?
 
@terdon Well, lines aren't usually 4096 characters. And "as long as possible" isn't meaningful, because integers don't have an upper bound.
 
@FaheemMitha Precisely. So a large upper bound was chosen.
Why would you want to force anyone to use short lines instead of coding to allow as large as is practical?
 
@terdon In the first place, I just wondered how that number was chosen.
 
Presumably because it is i) large and ii) a power of 2.
 
Is the power of 2 important?
 
11:45 AM
Yes. Everything related to computers tends to be in powers of two, right? 16, 32, 64, 128 etc?
 
Also a common page size, so allocating that much memory makes some sort of sense, anything larger could start causing issues. (Most of the examples are on the stack though so that’s not necessarily all that relevant.)
 
Remember that computers use binary, so base 2 arithmetic makes sense and therefore it is practical (presumably even essential in some contexts, this is way beyond my comfort zone) to use powers of 2.
 
12:01 PM
Hi guys, I was recommended to ask this question here, rather than as a Question on StackExchange, since it is off-topic. I hope someone might point me in the right direction:

I'm looking for an introductory resource on how the Linux kernel and distros work. Specifically I'd like to understand which problems are being solved by different parts of the kernel, and how it works at a level of functionality, rather than details about which commands to use to in order to manage a system. I don't have any particular goal in mind, I'd just like to understand it at a conceptual/functionality level.
 
@user56834 Linux kernel and distributions are different topics. Distributions are a more user level topic. But it would probably help if you could describe your use case, if any.
There are books about the Linux kernel, of course. At different levels.
And there are distribution specific books too.
But those are mostly for people who are actually using that distribution. I doubt they would make for a very interesting read if one isn't.
 
it's now off-topic, but had accumulated some Q & A's before that:
 
@user56834 Most of these are more useful and relevant when you are actually trying to accomplish something. My experience is if you just read something, you've forgotten it the next day, if not before. If you've actually made use of it, you have a fighting chance of actually retaining some of the information.
 
12:25 PM
@FaheemMitha I don't have any specific use case. I'm just curious about how the Linux operating system works, in the same way one might be curious about how a biological organism or biological cell works on the inside. For example I'm curious about how exactly Linux manages to boot itself (I suspect it's mostly not different from other OSes there, but I don't know how the others do it), and I'm curious about which functionalities are preformed by the kernel vs the distros.
I'm not curious about "tips & tricks" about how to achieve specific things.
 
@user56834 That's a fairly large range of stuff. I suppose the threads Jeff pointed to would have relevant books.
And there isn't such a thing as the Linux operating system. Linux is the kernel.
And no, I don't want to have a discussion/argument about it.
The kernel is itself a big topic, especially if you don't already have some familiarity with Unix kernels.
 
I'm ok with it being a big topic.
 
Hi @user56834 and thanks for coming! For context, this is the original (closed) question:
 
People do get picky about the words, once they matter. "Linux the kernel" is easily confused with the OS ecosystem around it, since they usually go together. Linux distributions end up with different startup daemons, desktop environments, services, etc.
 
0
Q: Overview of the functionality of the linux kernel & distros for someone who is just curious

user56834I don't have any particular goal in mind (e.g. not planning to contribute to the kernel, or build my own distro or anything like that). I'm just curious what the kernel actually does, and what is done by the added distros, how the kernel is booted. I'd like to understand which problems are bein...

 
12:30 PM
@JeffSchaller A Unix kernel is usually a rather small part of the OS, albeit a very important one.
 
I guess my ideal book would be something that progressively zooms in on the kernel, listing first all the modules, and what their purpose is, how they interact, then how they work inside, zooming in more and more.
 
@user56834 I'd go to a bookshop and browse. My experience of computer books (and I've purchased some) is that you mostly don't end up reading them. With some occasional exceptions. So I'd be cautious about buying them. Use a library instead.
 
I remember picking up a book, something like "Linux Kernel Internals"? that had a CD with it. Probably still have it somewhere.
 
What do you mean by a library in this case?
do you mean, use many books as reference?
 
@user56834 a place where one can borrow books rather than buy them
 
12:33 PM
oh lol, i see.
 
not to be confused with Stephen's library
 
1:01 PM
@user56834 I think both you and Faheem just aged yourselves! :P
By the way, don't expect any book to "list all the modules". There is no such thing. Modules are like drivers in the Windows world. There are loads of them and loads of user-specific ones and whatever. You can get a list of the modules available to be compiled into the kernel, presumably, but it will never be a comprehensive list of all modules.
 
Libraries still exist.
Buying books just for oneself is actually very wasteful. But a common practice.
Unless you plan to use it regularly. In my experience, that rarely happens.
 
1:17 PM
@FaheemMitha or if you’re interested in books which libraries no longer carry ;-) (old technical books for example)
of course these days archive.org has scans of many old technical books
 
@StephenKitt Libraries are more likely to carry older books than newer books.
But if you can't get it in a library, and need or want it, I suppose buying it is the only option.
Still kind of wasteful, though.
 
@FaheemMitha slightly older books, yes, but for my retro-computing interests I have a bunch of books which libraries no longer carry — in fact some of mine are former library books
 
@StephenKitt I see. For example?
Like the Apple 2e manual?
 
Oregon Trail for Dummies
 
@FaheemMitha comparing my nearest bookshelf and my local library catalog, UNIX Secrets (which is obsolete but I still find interesting) and Programming Perl (which isn’t obsolete but is too old for libraries)
and that’s without considering my DOS books ;-)
 
1:20 PM
@StephenKitt Why is "Programming Perl" too old for libraries?
 
or books which libraries don’t carry, like the Intel CPU manuals
@FaheemMitha because it was published in the 20th century
 
@StephenKitt I sounds like you have a large collection. Perhaps stored inside a handy TARDIS.
@StephenKitt Is that bad?
 
@FaheemMitha for technical books in libraries, yes
my local libraries carry technical books published between ~2005 and 2-3 years ago, so lots of PHP and Java books :-/
 
@StephenKitt So they automatically throw older stuff away?
Or sell or give it away, more likely.
 
@FaheemMitha yes, give it away or sell it (for peanuts)
libraries end up being like bookshops: they have limited shelfspace and have to make the best use of it
some libraries also have larger archives, but they’re not all that common
 
1:24 PM
A lot of the libraries I've used kept books around for ages. And the technical ones they'd purchased were typically not the best.
 
@StephenKitt The Camel Book? People are throwing out the Camel Book!?
 
Not sure where they got the space.
 
@terdon libraries are, yes :-(
 
But the 4th edition came out in 2012!
(no, I don't know that off the top of my head, I just looked it up)
 
There's a library in Fort, Bombay, called "The J.N. Petit Institute", which, at the time I was using it, had books from the 1930s. It was like a time machine.
Or maybe even earlier.
Unsurprisingly, it has a Wikipedia page.
The J. N. Petit Library (officially the J. N. Petit Institute) is a membership library in a Grade II heritage structure in Fort, Mumbai. It was founded in 1898 by a group of Parsi students studying at Elphinstone College. Membership is open to residents of Mumbai.The library is one of the finest examples of Neo-Gothic architecture in Mumbai. In 2014–15, it was restored by a team led by the conservation architect Vikas Dilawari. The restoration project won the Award of Distinction under the UNESCO Asia-Pacific Awards for Cultural Heritage Conservation in 2015. == History == The library traces its...
It was quite a strange experience being there. Though it was bang in the middle of South Bombay with traffic roaring all around, it was like being on a desert island.
They've kept the building in good shape, though. Until recently I used to pass by it quite often, though I don't think I've been inside it since the early 1990s, or possibly even before.
 
1:47 PM
@terdon I’ve got the 3rd edition from 2000. I suspect that the length of time between editions doesn’t help: by 2010, a 2000 Perl book might have triggered the “obsolete tech” flags in libraries, and then in 2012 noone would be asking for Perl books any more...
The nearest library with a decent technical section only has two Perl books: one published in 2015 which is a comparison of multiple scripting languages, and one published in 2020 which is a history of the web
 
 
3 hours later…
4:49 PM
those sound like unusual perl books
 
@AndrasDeak Isn't that the standard reference?
I don't use Perl, so I'm not sure.
When I first tried to learn C in 1997, I recall having a lot of trouble with it, and wondered why it was made so difficult. In 2021 it all seems quite reasonable to me. I guess that's what they call perspective.
 
@FaheemMitha The Camel Book (Programming Perl) is the standard reference. Andras was referring to the two books that Stephen mentioned just above which do indeed not sound like usual perl books since one is a comparison of many languages and the other is a history of the web.
 
@terdon Ah yes, I wasn't reading carefully. Sorry.
Also, half asleep.
 
 
2 hours later…
6:36 PM
As a matter of style, should one check every system or standard function call in C and return strerror(errno) if there is an error? I don't recall doing this for C programs I wrote earlier, and it certainly makes for a lot of boilerplate.
 
Most of your functions probably don't have a return signature of const char* in the first place and for the ones that do, writing all your call sites to use string manipulation functions to tell if an error occurred seems like a bad idea.
Also, not all libc functions can set errno in the first place!
But like, as to the question I think you were trying to ask -- yes, if you care at all about whether a call succeeded or not (which is probably not always, but you should definitely care if, e.g., you were able to successfully open a file you need to read/write, or execute a subprocess, or what have you), you should check if the call succeeded using whatever idiom that function employs for error checking
 
Oh, strerror is a POSIX thing, which I why I don't recall using it earlier.
 
And if it's not a recoverable error, you should probably output an error message (which, for errno-based errors, can be conveniently done with perror or strerror) and then exit
And if it's a recoverable error, but one the user should be made aware of (either because it's something they could reasonably fix for future runs or because it affects the operation of the program in some way), you should at least log it
 
@ToxicFrog Sorry, I don't follow the reference to functions with return signature of const char *.
 
> and return strerror(errno)
> man 3 strerror: char *strerror(int errnum)
Aah, you're right, it's not const
 
6:43 PM
For example, should one test for error with malloc and memcpy?
@ToxicFrog Well, I would normally just print the strerror to stderr, and exit or return, I suppose.
 
You should read the man pages for both of those.
 
@ToxicFrog Fair enough.
 
> I would normally just print the strerror to stderr, and exit or return
That's what I'd generally do, but that's not the same thing as returning the error, which is what you originally said (and in some languages is a common idiom for error handling)
 
I guess _exit is the right thing to use here. return isn't.
@ToxicFrog I wouldn't attempt to handle the error. Just exit.
I wrote:
> should one check every system or standard function call in C and return strerror(errno) if there is an error.
No handling was implied.
Well, there are certainly a lot of things that can go wrong in even a small program.
 
Yes there are!
Also, is there a particular reason you're preferring _exit() to exit()?
 
6:48 PM
Supposedly POSIX defines a UNIX standard of sorts, but a lot of it doesn't seem particularly UNIXy. Just enhancements of various sorts which could easily be in the standard library.
@ToxicFrog Well, it makes less of an effort to clean up.
 
Yes; personally I generally want my output buffers flushed and atexit handlers to run.
 
@ToxicFrog Doesn't it disguise problems, though?
 
> a lot of it doesn't seem particularly UNIXy
So, (a), a lot of the stuff in POSIX has, in the ~35 years since it was written, been adopted by a lot of non-UNIX OSes, to varying degrees of success (windows being a notable exception until very recently)
and (b) POSIX originated as an attempt to standardize APIs and user environements across the at the time very heterogenous UNIX ecosystem, specifically
 
Really, even pipes seem relatively general. Though I don't know if other OS's use them.
So, when Bash uses a pipe, it's just calling the corresponding C function, right?
 
Probably? I haven't looked at bash's internals ever.
 
6:53 PM
OK. You were saying? specifically...?
 
And, again, POSIX is almost as old as I am; a lot of stuff that looks "general" now is general because it was specified in POSIX and people copied it for their non-UNIX operating systems to make cross-platform development easier
 
@ToxicFrog I see. I meant, it doesn't appear to be tied to any very specific model of computing.
I suppose there is a lot of "everything is a file", which I realise isn't exactly a universal interface.
 
Like, pipes "seem very generaly" and indeed as a concept they predate UNIX, but as an operating system feature they were a major innovation in 1970s UNIX
 
@ToxicFrog I didn't know that.
You didn't finish what you were saying about "heterogenous UNIX ecosystem".
 
What? Yes I did.
Oh, I see the confusion
That wasn't "specifically, X, Y, Z"
 
6:57 PM
@ToxicFrog Well, you forgot the period. :-)
So "specifically" was the last word there? OK.
 
It was "posix was specifically created to standardize the state of UNIX, not as a standard for OSes more generally; the fact that many non-UNIX OSes have adopted it is a convenient accident".
 
Does Windows now use pipes, then?
 
Windows has had pipes since at least NT -- possibly earlier, I am by no means proficient with windows -- but the API for using them is completely different from POSIX pipes, and as far as I know there is no equivalent to mkfifo to create a pipe in the filesystem -- named pipes exist but have their own completely isolated namespace separate from files on disk.
 
@ToxicFrog I see. So not for use at the user level?
 
They're usable in cmd.exe in a manner conceptually similar to bash (e.g. dir | find or so), but the semantics are different in some (to me) surprising ways
(which I do not remember the details of, it's been well over a decade since I used cmd.exe for anything but the most trivial of tasks)
[_exit disguising problems] -- what sorts of problems do you expect to be disguised by this? I could perhaps see it making some problems harder to diagnose if the program has gone sufficiently wrong that an atexit() handler stomps the stack or something...
 
7:10 PM
@ToxicFrog I just thought one might lose some information. So you think exit is better, then?
 
In general I find that exit() is a better default. There are some circumstances where _exit may be preferable. You need to decide on a case by case basis.
 
@ToxicFrog OK.
 
 
2 hours later…
8:48 PM
@ToxicFrog if you're forking child processes, you might not want to flush everything or process exit handles in both the parent and the child. E.g. something like printf("going to fork now.\n"); int a = fork(); if (a == 0) { printf("now in child\n"); exit(0); } would print the first line twice if the output is redirected to a pipe.
 

« first day (4036 days earlier)      last day (36 days later) »