/dev/chat - 2021-10-19

« first day (4028 days earlier) ← previous day next day → last day (909 days later) »

12:29 PM

@FaheemMitha no, it doesn't. execve() keeps e.g. the process id and the parent-child relations, but wipes the memory of the process (replacing it with another program), plus some other things. And for file descriptors, it depends on the close-on-exec flag (FD_CLOEXEC etc.) There's a list of what it does in at least the man page on Debian: manpages.debian.org/bullseye/manpages-dev/execve.2.en.html

@ilkkachu Confusing.

In particular, calling execve() in a child process does not trigger wait() or SIGCHLD on the parent, since the child didn't terminate. Calling _exit() on the child does.

@ilkkachu Calling _exit() after the execve in the child process?

Confusing, and flexible, e.g. it allows the shell to first fork() a copy of itself, then shuffle file descriptors in the child, and only then exec() the actual command, which then sees everything set up like it should be.

after, or before, or whenever.

@ilkkachu I used to have a copy of "Unix System Programming" by someone called Stevens, perhaps. I wonder if I could locate it now.

@ilkkachu As it happens, I'm sitting here trying to make sense of the apparently insane output being produced by this C code.

Not the most fun way to spend an afternoon, perhaps.

Advanced Programming in the Unix Environment

Advanced Programming in the Unix Environment is a computer programming book by W. Richard Stevens describing the application programming interface of the UNIX family of operating systems. The book illustrates UNIX application programming in the C programming language. The first edition of the book was published by Addison-Wesley in 1992. It covered programming for the two popular families of the Unix operating system, the Berkeley Software Distribution (in particular 4.3 BSD and 386BSD) and AT&T's UNIX System V (particularly SVR4). The book covers system calls for operations on single file...

@ilkkachu I read that after calling execve the image is replaced, so the child won't do anything after that.

So what is executing the _exit?

12:36 PM

@FaheemMitha the parent waits for a child process, irrespective of what the child process is running; execve replaces the running image in the process, but the replacement can then exit, which closes the process and notifies the parent

processes and code running inside them are two different things

@StephenKitt Isn't the running image just the process at an instant in time? Does this running image still execute instructions given to the child after the call to execve?

@FaheemMitha a process is a construct within which code runs. When you fork, the new process runs the same code as the process which forked it. When you execve, the calling process runs the new code. In all cases, the parent can’t give instructions to the child after fork, unless both processes are set up to communicate...

@StephenKitt Yes, I'm assuming that the code after the execve is restricted to the child.

Perhaps an example would help.

Though is it the case that if a program starts as the parent process, all the code after the fork is run on the parent?

@FaheemMitha no, after the fork there are two processes

And they run the code simultaneously? And what about after the execve?

12:47 PM

@FaheemMitha yes, they continue running simultaneously; that’s why fork has two return values (on success). execve only affects the process which calls it.

@StephenKitt So if the child process calls execve, what happens to the code that takes place after the excecve is called? Does the execve derail the child process?

@FaheemMitha on success, execve doesn’t return

@StephenKitt So it stops executing the listed code?

Found my copy of Stevens. I guess I'll do some reading.

@FaheemMitha yes, and starts executing the code in the replacement image

@StephenKitt OK. That's true of the entire exec family, right?

But the code I was looking at looks like it kept going. I should take a closer look.

12:55 PM

@FaheemMitha no. The program calling exec() won't do anything after it, since it's replaced by another program. The process continues running by running (starting) that new program. Eventually that new program will probably call _exit() to terminate. (it's shown as exit_group() in Linux, but don't mind that. of course it might crash with a signal instead)

#include <unistd.h>

int main(void)
{
execlp("/bin/ls", "ls", NULL);
write(1, "end.", 4);
}

(I have no idea if code blocks work on chat.)

That write() will never be called. It'll run ls instead.

@ilkkachu they do if you click the "fixed font" button, or just press ctrl+k

bottom line is uniform 4-space indent on every line

or maybe put another write() before the exec() too

@AndrasDeak yeah, of course it would be too much to hope the triple-backtick would work on chat too. thanks.

Well, triple-backticks are really recent. Chat has been abandoned for a decade from what I can tell.

yes...

@ilkkachu Yes, I see. Thanks for that clarification.

1:00 PM

But it's also crap w.r.t. the main-site rules. You can't mix multiline code and non-code. Or any other kind of markdown in multi-line messages. It's a mess.

we ended up writing a code formatting guide to point python chatroom visitors to

hehe

2 hours later…

Martin Tournoij

3:09 PM

What are the advantages of bash over zsh (beyond being pre-installed on many Linux systems)? This sounds a bit like a flame-bait-y question, but I'm genuinely curious as I'm actually not really seeing any and I wonder why bash is so ubiquitous in spite of its limitations and difficulties (quite a few of which – though not all – are solved in zsh)

3:20 PM

@MartinTournoij the fact that it's preinstalled in pretty much every Linux system :D

^ that

Bash is ubiquitious; zsh is not. zsh is an excellent choice if you want something "like bash but better" (as opposed to something like fish that completely discards bash/sh compatibility) but you need to expect that it'll need installation on most systems you use.

This also makes it unsuitable for publication of shell scripts in most cases, while using bash is generally defensible, so even people who use zsh as their daily driver need some familiarity with bash if they plan to publish anything

As for "why bash and not zsh became ubiquitous", this is pure speculation on my part, but AIUI zsh did not start out as "like bash but better"; it started as a ksh/csh hybrid and gradually evolved into its current form. By the time it became a valid alternative for people who liked bash, perhaps bash had already become the de facto standard? (A position that it probably acquired by being mostly compatible with sh, the POSIX standard bourne shell)

@MartinTournoij Based on my (limited) experience, it's worth splitting the issue into "reasons for choosing a shell for personal use" and "reasons for choosing the shell to be shipped with a distribution".

Martin Tournoij

@ToxicFrog bash and zsh came around at roughly the same time (1989 for bash, 1990 for zsh), so "bash but better" probably didn't make a lot of sense in those days anyway

As for ubiquitousness, IMHO it's overrated; but that's a different argument and I'm mostly interested in the actual feature-set/behaviour

3:36 PM

Makes you wonder who it was that chose the first default shell that went into the first wide-spread distro (or distros) that gained popularity "by default". I know I learned about bash decades before zsh, as one perspective.

Martin Tournoij

My guess is that it was the default because it's part of GNU, but that's purely a guess

@MartinTournoij it was certainly part of the Linux bootstrap

> bash and zsh came around at roughly the same time (1989 for bash, 1990 for zsh), so "bash but better" probably didn't make a lot of sense in those days anyway

Right, that's my point; bash was "sh but better" and zsh was off doing its own thing. It's only much later that zsh occupied the "bash but better" niche (which, as you say, is only a niche people care about because of bash's ubiquity)

I've been led to the understanding that ksh had license issues back then, which basically was the reason for Bash being developed. And the reimplementing of a number of ksh features in Bash.

zsh has a POSIX sh mode now, but did that exist always? If they started only doing their own thing, without a compatibility mode, then that's probably very good reason it didn't become the "default" shell

Martin Tournoij

@ToxicFrog As I understand it, bash wasn't really "sh but better", but rather a Free implementation of ksh, which was Proprietary AT&T UNIIX® a the time

I'm not sure which free shells existed in the late 80s/early 90s; I think (t)csh wasn't free until the 90s either

3:50 PM

I remember ksh being VERY different from bash last time I used it, but that was in like 2002 or so

Martin Tournoij

ksh development also didn't stop of course, so things probably diverged. I never used either much (I went from tcsh to zsh), but a lot of things seem to trace back to ksh

Bash was and is the official GNU Project shell. And I think initially Linux just used the GNU tools, sort of as a set. gcc, gdb, the GNU toolchain, Bash.

That's still largely the case, 30 years later.

Linus at the time used the GNU tools on top of Minix, I think.

Then I guess he took them with him when his kernel became viable.

Were the other Unix shells at the time easily portable? GNU has always been portable, by design. Hence autotools.

Aside from considerations of freedom.

So it seems Ch 8 "Process Control" of Stevens' book "Advanced Programming in the UNIX Environment", corresponds to the topics under discussion.

My impression is that Stevens is a standard and well known text, though the material seems relatively elementary. Perhaps a more descriptive title would be "Basic Unix Systems Programming".

Does anyone have other suggestions?

4:20 PM

I remember Kernighan & Pike's The UNIX Programming Environment being good, but I also read it like 20 years ago; I have no idea how well it holds up now

These days I just rely on the man pages, but they're more reference than instructional.

@FaheemMitha The Linux Programming Interface if you’re interested in Linux specifically (although the text does explain POSIX in general as well).

@FaheemMitha it does cover all the Unix API of the time, and its sections on signal handling and terminals strike me as rather more on the advanced side; the various sections on processes should address all your earlier questions on the topic

@ToxicFrog @StephenKitt Thank you for the suggestions.

@StephenKitt There is now a 3rd edition, though not written by Stevens, who died in 1999.

@StephenKitt en.wikipedia.org/wiki/The_Linux_Programming_Interface ?

@FaheemMitha yes

@ToxicFrog That one might be a mite dated. Published in 1984, and apparently no second edition came out.

Still, might be worth a look if I can find a copy.

Sometimes it helps to read multiple treatments of the same material from different perspectives.

Hmm, the APUE source code is available online.

Martin Tournoij

@AndrasDeak Aside: in spite of its quirks, the SE chat is still one of the best webchats I know of. It's simple, clear, and Just Works™ without ridiculous input lags and such

4:34 PM

@FaheemMitha indeed, and I have it, but I prefer LPI

Tried building it, but the build failed.

@StephenKitt OK. Well, I'll take a look at that too. But maybe I should make a first pass through Ch 8 first.

@FaheemMitha apuebook.com/faqs3e.html#7 perhaps?

@StephenKitt I had libbsd-dev installed already, so not that particular one.

Anyway, I don't need it built.

1 hour later…

6:04 PM

Greetings, beautiful people. My hard drive fails. I don't know how to recover the data

@Ben If you have a second hard drive and backups, there's that. Or a professional data recovery service. Or is it failing and you need something like ddrescue?

(sorry for responding; I'm far from a beautiful person; hopefully that's OK) :)

@ilkkachu The documentation suggests that the last argument to execlp should of the form (char *)0. Perhaps that is equivalent to NULL if both of them represent a null pointer.

It's been a while, so I don't remember the details about null pointers.

6:23 PM

I don't have another backup drive for the entire 1TB failed one.

I was just imagining where you'd put the recovered data, is all

It is the hdd for Ubuntu with all the partitions . I can't boot into it now, and have pulled the hdd out and connect it via an adapter to another laptop, but it was only recognized as one /dev/sdb, no partitions shown

The failed hdd is half full, so I normally rsync the files out

When I connected the failed hdd to another laptop, It was shown by lshw as /dev/sdb, and not shown by lsblk.
After an hour, the other laptop seems to implicitly make some progress in discovering the failed hard drive. lsblk now has shown the partitions of /dev/sdb. and df has shown only the root partition of /dev/sdb, not the home partition. Two partitions have been mounted in /media/, but ls and cd on them get stuck, so does lshw.
Any one knows what is going on?

@FaheemMitha ugh, yes. it's a varargs function, so there's no type in the function prototype, so the program should make sure that the final NULL is correctly cast to char *. NULL could be just a raw zero in C.

6:47 PM

😭

@ilkkachu So (char *)0 then?

@FaheemMitha yeah

1 hour later…

Martin Tournoij

7:53 PM

@Ben If the drive is actually damaged then then the safest thing to do is copy it first since any further reads risk damaging it further. If it's merely a software issue (e.g. filesystem corruption) then there are some things you can try, however, further usage may risk damaging a drive further

It kind of depends on how important the data is to you and how much risk you want to take

Putting it in the fridge for an hour or two can help in the case of damaged drives (hard disks, not SSDs). I've had a few occasions where it would work enough to quickly copy data from it before it gave up

But in general, dd_rescue to another disk first is usually the first thing I'd recommend if at all possible. You can then do whatever and have a safe backup (or at least, as safe as possible) in case you make things worse

Oh, you can use smartctl to see if the disk has any bad sectors (not always recorded in there, but often are)

Thanks. how can I find out if the issue is the hardware or the filesystems?
I think I have to save money to invest some bigger backup harddrive. Which one is good and cheap?

@MartinTournoij

Martin Tournoij

I'd start with smartctl from smartmontools

Hypothetically, can LVM make data recovery more difficult or easy?

Martin Tournoij

And I have no idea what drive to buy; I haven't been involved in hardware in a long time

Usually more layers make things harder

> Hypothetically, can LVM make data recovery more difficult or easy?

Depends entirely on the configuration. If the LV is based on mirrored PVs, much easier, since you now have multiple copies of the data. If it's one LV spread across multiple PVs without mirroring, potentially much harder, because the filesystem is not guaranteed to be contiguous.

Martin Tournoij

8:06 PM

But yeah, if you have mirroring then you're at least protected against (some) hardware faults

what is mirrored PVs? I only have one hdd for a laptop

Martin Tournoij

It means the computer will write the same data to two or more drives; they're a "mirror image" of each other. I don't think it's very relevant in your case, especially because it's too late now anyway since you've already got a broken drive

It is, because I am also starting to reinstall Linux on another laptop, so I can start doing things again

Martin Tournoij

Oh right :-) Well, if you have space for two drives then it can help

I have only one for that laptop too

Martin Tournoij

8:09 PM

But it's not fool-proof: if you drop your laptop in a lake then you're still screwed, or if you accidentally remove the wrong file, or things like that

For backup devices, is it correct that capacity is more important than portability? My current one is portable, and only 1TB. I use it for both backup files and portable files Is that the reason that might lead me to more often free up space and less frequent backup?

Martin Tournoij

Mirroring is not really a replacement for backups; or a limited one at best

I want to save to invest on backup device some day

so wondering about what inexpensive backup device to consider

Martin Tournoij

@Ben I can't answer that question for you? I have just ~5G of data and 1TB would be overkill for me, so I'd prefer portability. Other people have more data.

A desktop 2TB hdd a good choice?

Martin Tournoij

8:13 PM

It could be if you have 2TB of data...

Or thereabouts

That you actually need to backup. Personally, I only care about my code projects and a very limited set of pictures, and everything else can just be gone and I'd be fine with it, so 2TB would be overkill and much less would be fine for me

@Ben If your data is mostly text and maybe small data files, then using a distributed version control system is very helpful.

Otherwise, deduplicating backup software is probably the state of the art.

E.g. Borg.

8:32 PM

Is LVM worth to go for its benefits?

@Ben yes

9:13 PM

@ilkkachu how so?

10:12 PM

1

Q: What is advantage of backing up a filesystem by version control software?

Version control software seems to be used for backing up project mostly in plain text files. For backing up a file system, with large amount of files or large size, regular file copying/transferring software such as rsync seems to me more proper? If not, what is the advantage and disavantage of...

backup version-control

« first day (4028 days earlier) ← previous day next day → last day (909 days later) »

Transcript for

Oct '2119

/dev/chat

General discussion for unix.stackexchange.com. If you have a q...

join this room
about this room

00:00

06:00

12:00

18:00

all times are UTC

site design / logo © 2024 Stack Exchange Inc; legal

mobile