/dev/chat

Q: How to use Linux manual for understanding how a computer works after it's turned on?

7:38 AM

13

A few years ago I recall using the terminal and reading a tutorial on the Linux manual (using man) on how a computer worked after it was turned on. It walked you through the whole process explaining the role of the BIOS, ROM, RAM and OS on this process. Has anyone seen something similar to this? ...

It's a pity that this was voted "off-topic". Yes, it asks for learning material, but specifically about one of the on-line manuals. I think this falls under "official documentation"

@Kusalananda that’s an interesting angle — I’d always thought of “official documentation” as standards (based on the POSIX example), but perhaps systemd documentation counts too...

story-identification

It is a bit borderline

"Which man page has this content?" is probably an on-topic question

@MichaelHomer I voted to reopen it and left a comment.

@Kusalananda I have edited it substantially to try to make it clearly on-topic

The last paragraph was a problem but the overall question seems sound to me

Good! Looks good now.

7:53 AM

@Kusalananda thanks for the clarifying paragraph ;-)

@StephenKitt Not only. What we don't want are general requests for howtos etc. Asking about something specific like a given man page is fine. The issue is with asking for random webpages that can easily become obsolete.

@terdon right, good to know

@StephenKitt :-) web-manuals are a bit of a pet-peeve for me. They are good to have, but may differ from what's true for a specific system.

@Kusalananda indeed, I like to include links since SE is web-based, and it also counts as “supporting evidence” in some cases, but I need to remember to tell readers to check their local manual in some cases

And just as I wondered how it reopened so fast...

7:55 AM

there was a time when local admins would maintain their own manpages but I doubt anyone still does that

@StephenKitt No worries at all.

@MichaelHomer magic. . .

:-)

@MichaelHomer yeah @terdon's the fastest gun in the west

snort

7:56 AM

he moderates before you flag

I checked the timeline and it all made sense

Now that's part of the SE UX that I think could be improved, unless I've missed something

finding the timeline I mean

Yeah, that's a bit of a hidden feature isn't it? You need to manually put in the $questionUrl/timeline address, right?

Close/reopen is actually on /revisions too, which is linked

But otherwise, yeah, you have to edit the URL

Mods get a link for it in our special mod menu. I'm sure there's a stackApp somewhere that makes this simpler.

6

Q: View Post Timeline

Inspired by this MSO question. This tiny userscript will add a link to the timeline for any post, just below the "favorite question" star button. Example image: This is the code: // ==UserScript== // @name Add Timeline Button // @namespace none // @description Adds a "View Timeline...

script timeline

8:00 AM

@terdon nice!

He even found the userscript before I got there

"reading a tutorial in the Linux manual" reads slightly oddly. Perhaps it should be "reading a tutorial in one of the Linux manual pages"?

@Kusalananda That's with reference to this question.

@FaheemMitha I’ll scratch my imaginary beard, shake my cane and retort that “the Linux manual pages are the Linux manual”

@StephenKitt Mmm. I don't think I've ever heard it called that.

So you have an imaginary beard? What color is it?

they’re pages from the manual

8:04 AM

More to the point, why don't you have a beard? What kind of a neckbeard are you?

the terminology dates back to the Unix manual

@terdon a stealth neckbeard

@StephenKitt <Shrug.> Ok, fair enough.

The only word that doesn't quite fit is "tutorial". manuals generally do not have tutorials (and shouldn't have).

I remember printing the fvwm manpage on ~100 sheets of A4 paper when I was at university

@Kusalananda Well, not generally. But sometimes they do. E.g. the rsync man page has something that looks rather like a tutorial. Which I don't think is a bad thing, necessarily.

It depends whether you consider examples to be a tutorial, for example.

Personally I think man pages should have more examples. Often they don't have any.

8:21 AM

@FaheemMitha yeah very good manpages have good examples that you can adapt to most cases

instead of having to figure out all the options yourself

@FaheemMitha Examples and tutorials are sometimes kept separate. On my system there's /usr/local/share/doc, /usr/local/share/examples as well as individual directories under /usr/local/share for many third-party applications.

@Kusalananda Isn't it \usr\local\share on your system?

@terdon Eh? What's with the backslashes?

Your system runs Windows.

:-P

8:25 AM

;)

My system is the OpenBSD that runs in my VMs. Silly.

The laptop may run z/OS for all that I care.

@Kusalananda that would be some laptop

either that or some lap

:-D

imagining Kusalananda in a plane

unfolding his water-cooled z/OS laptop

"excuse me, I need some room to work"

"oh and sorry for the noise"

Here's What Happens When an 18 Year Old Buys a Mainframe

Reminded me about this guy:

►

8:29 AM

ah yes that's a fantastic presentation

He has some very supportive parents!

8:55 AM

@Kusalananda Now that's what I call an interesting childhood.

@Kusalananda Always helpful.

12:53 PM

@Kusalananda wow! and how did I not realize that Thomas Dickey wrote Lynx? been too long since I loaded lynx without a URL, I suppose

@JeffSchaller he’s not the original author but he’s been maintaining it for at least twenty years

IIRC my very first patch was to Lynx, sometime around 1994 or 1995, but I can no longer find any trace of it

@StephenKitt ahh! thanks for the correction!

Does anyone know how to time a pipe?

Apparently, time seems to only report the time taken by the first command in the pipe, despite waiting for the whole thing to exit before printing its output.

1:08 PM

time ls | wc

on my system reports the time for ls and for wc

(on zsh)

I thought that was one of the reasons time is a shell built-in

so that it can time pipelines

@terdon I get the same behavior, despite the manpage saying it supports timelines (bash)

@StephenKitt apparently not

@terdon yeah I just saw your question there

I get the same thing on bash, no detail

so perhaps “use zsh”? ;-)

> All tests were run on an Arch system and using bash 4.4.12(1)-release. I can only use bash for the project this is a part of so even if zsh or some other powerful shell can get around it, that won't be a viable solution for me.

And yes, I can confirm zsh does this on Arch too.

Also, what gives? I find it very strange that time ( foo.sh | bar.sh ) doesn't work. It should be timing the subshell. And why does that fail when time (foo.sh; bar.sh) works?

Ah, you're talking about it here... didn't spot that.

@terdon What was the script with seq that you tried?

1:19 PM

Gimme a few minutes, I need to drive someone.

@terdon No worries.

1:34 PM

@Kusalananda here, for example:

$ cat ~/scripts/foo.sh
#!/bin/sh
echo 1
sleep 1
echo 2
sleep 1
echo 3
sleep 1
echo 4

and:

$ cat ~/scripts/bar.sh
#!/bin/sh
sleep 2
while read line; do
		echo "LL $line"
done
sleep 1

So, I would expect that to take 3 + 3 seconds, but:

$ time ( foo.sh | bar.sh )
LL 1
LL 2
LL 3
LL 4

real	0m4.010s
user	0m0.003s
sys	0m0.003s

No, the first echo won't finish until bar has slept for 2 seconds.

Then each echo follow, once every second.

And then bar sleeps for two seconds.

I will test it and tthink

This might be more informative:

$ time ( unbuffered foo.sh | unbuffered bar.sh )
1
2
LL 1
LL 2
LL 1
LL 2
3
LL 3
LL 3
4
LL 4
LL 4

real	0m4.016s
user	0m0.003s
sys	0m0.003s

It still doesn't report real time correctly though.

It does, because the pipes don't block per line

both scripts start simultaneously

bar sleeps

foo echoes 1

foo echoes 2

foo sleeps

bar reads 1 and 2

foo echoes 3, bar reads 3

foo sleeps

foo echoes 4, bar reads 4 and sleeps

foo finishes after echoing 4, after 3 secs

boo finishes after 4 secs

the total wall-clock time is 4 secs

I see. Yes. And I did time it externally and, as you say, it takes 4 seconds, not 6.

Because that's how pipes work terdon, you idiot.

And indeed, this does work as expected:

foo.sh:

#!/bin/sh
sleep 3
echo foo

bar.sh:

#!/bin/sh
while read line; do
		echo "LL $line"
done
sleep 5

$ time ( foo.sh | bar.sh )
LL foo

real 0m8.011s
user 0m0.007s
sys 0m0.003s

yes

there's also the fact that foo exits which stops bar's read loop

1:45 PM

Yes, it makes sense now. Should have figured it out myself, really. Thanks @Kusalananda and @StephenKitt.

you're welcome!

running it with zsh foo.sh | bar.sh helps since it shows the separate execution times

$ time ( env PS4='$SECONDS foo:' sh -x foo.sh | env PS4='$SECONDS bar:' sh -x bar.sh )
0 bar:sleep 2
0 foo:echo 1
0 foo:sleep 1
1 foo:echo 2
1 foo:sleep 1
2 bar:read line
2 bar:echo LL 1
LL 1
2 bar:read line
2 bar:echo LL 2
LL 2
2 bar:read line
2 foo:echo 3
2 bar:echo LL 3
LL 3
2 bar:read line
2 foo:sleep 1
3 foo:echo 4
3 bar:echo LL 4
LL 4
3 bar:read line
3 bar:sleep 1

real    0m4,12s
user    0m0,01s
sys     0m0,08s

@Kusalananda Ooh, clever! Thanks:)

2:08 PM

@terdon Oh, it was your question! :-) I didn't spot that until now.

@Kusalananda snort :)

I don't usually look at who's asking...

Probably for the best.

2:27 PM

This will help answer the various history questions that come up now and again: spinellis.gr/blog/20170510

Count on a Unix hacker to write a tool to scratch their own itch like that! Nice!

Q: Any recommendations for drawing a hierarchically nested box diagram?

I’d seen

0

Is there a package, tool, or method you would recommend for automatically drawing a large, complex, hierarchically nested box diagram, such as the sample below? I'd like to provide the input in textual format (so that I can easily perform large changes) and get the result in vector format (e.g....

tikz-pgf tables diagrams

the other day and thought that might mean something interesting was coming down the pipe

@terdon I put that in the answer and annotated it.

@Kusalananda Ah, thanks. I was going to suggest that.

3:23 PM

Hey I had this error pipe failed: too many open files in system. I recently made a shell function recently and I was wondering if it was the cause.

# competitive programming python like cpp file runner
cpprun () {
  g++ $1 -std=c++14 -o a.out && ./a.out && rm ./a.out
}

I don't do bash scripting very often so I don't know how to evaluate it so I was wondering if someone could take a look at the script and see if anything is wrong with it.

looks OK to me, apart from wanting to quote "$1"

Should I quote $1? what's the difference?

in case you wanted to run: cpprun "file name with spaces.cpp"

other than that, I don't see an error in your function; I'm guessing you just have some number files open already, and running this bumps you into the limit.

ulimit -n will tell you what your limit is

Okay thanks for taking a look at it.

It appears mine is 8192.

I was running it with a pipe by the way like this cpprun main.cpp < input.txt

find /proc -user $LOGNAME 2>/dev/null | grep -c /fd/ is one quick hack I just came up with to give you some idea for how many files you have open now

3:31 PM

It says 0. So it must have been something else on my system.

maybe LOGNAME was misleading -- try $USER ?

Still says 0. I'm on a mac by the way. I don't know whether that makes a difference for that particular command.

It probably does. Does OSX even have /proc?

@John might; I don't have OSX to play with -- I made that up on a Red Hat system

@John Does the message survive a reboot?

3:34 PM

@terdon It has, but it's nowhere near the same.

ls -l /proc/$$ would be a good quick test

It doesn't have proc

That sounds like some infinite loop going on somewhere. A reboot should fix it.

If it happens again, you'll need to track it down though.

Yeah I rebooted it seems to be working now.

you might .... oh well, have run 'ps -ef' and see if there are processes stuck out there waiting to read from a file

4:01 PM

What would the status code be if it is waiting to read from a file?

likely just (S)leeping, but the direction I was going with that was to see if you had (accidentally) created a bunch of background processes that were eating up open files

ps x may be a better view -- list all of your own processes, and just see if there are "extra" ones out there

Does Mac OS X have lsof? If so, lsof -a -u$USER -d 0-9999 might help

Hmm well thanks for your help. I'll probably worry about it more if it happens again. I definitley feel like I should learn unix better now. I have a couple books I could bought recently from oreily about unix. It was in a humblebundle.

Yeah it has lsof

If it happens again, that lsof should show what's eating all your file descriptors

@derobert nicely done!

4:15 PM

The problem is I'm such a noob I don't know what I should be looking for.

@John Well, you can rummage around with the things you do know, and come here if you fail to find good solutions ;-)

A: Is there a fix for the "Too many open files in system" error on OS X 10.7.1?

@John see also:

150

According to this helpful article (which I recommend reading): By default, the maximum number of files that Mac OS X can open is set to 12,288 and the maximum number of files a given process can open is 10,240. You can check these with: sysctl kern.maxfiles sysctl kern.maxfilesperproc...

@John That lsof is showing all the open files on your system. Not sure the columns used on Mac OS X, but on Linux its the program name, then the process ID, then the user, the the file descriptor number (first file a program opens is 0, second is 1, etc. Numbers get re-used if a file is closed. Actually 0/1/2 are normally stdin/stderr/stdout). 3rd is type of file, then the device (disk/partition its on). Size should be obvious, node can be ignored, and the file name is last

Mac OS X's lsof might have different fields, not sure.

But the command (program name), pid, and file name ought to be there.

@JeffSchaller (IMHO, it would be better to track down what's keeping file descriptors open and fix that rather than just increase the limits)

@Kusalananda agreed

4:21 PM

If Mac OS X's limit is only ~12K open, may have just run out... But yeah, definitely worth checking if its just a program going crazy (kept calling open without calling close) or it's really just a too low limit

Okay thanks guys! You've all been helpful to me. I'll have to read those books I have in order to get a better knowledge base so I can figure out how to track down these kinds of things.

@John Just thinking aloud: lsof | awk '{ print $1 }' | sort | uniq -c | sort -n This will show what running program have how many files open.

 [...]
 134 cloudd
 159 AppleSpel
 209 Slack\x20
 233 Dock
 295 com.apple
 460 Dropbox
1201 Google

So would google be the one with the most in your example?

Yep

Seems like that's mine too it's around 800

4:34 PM

"files" also include network connections etc. in this case.

... but it may be heplful for you to try to pinpoint the problematic process if you run into the same problem again.

@Kusalananda Hmmm, plain lsof probably includes things like memory mappings, etc. that may not count—might want to add in the -d 0-9999 (I really wish there were a better way to say file descriptors only)

@Kusalananda On my Linux box, that gives absurd results without the -d:

   7705 QThread
   8295 ImgDecode
   8430 WorkerPoo
   8433 thunderbi
   9543 mozStorag
   9616 GlobalQue
  14220 JS\x20Hel
  20414 dropbox
 146987 firefox

:-D

@derobert lots of WorkerPoo going on there!

@JeffSchaller Worrying. Hope there are plenty of muckers available.

    # with -d
    107 firefox
    113 ksmserver
    115 thunderbi
    160 dropbox
    164 clementin
    219 xterm
   1166 chromium

Ok, def. lower numbers with `-d 0-9999`:   38 photoanal
  65 Slack
  70 AppleSpel
  89 nsurlstor
  91 UserEvent
  98 Slack\x20
  99 cloudd
 106 com.apple
 234 Dropbox
 582 Google

4:48 PM

On Linux at least, need to run as root to get full output.

    112 ksmserver
    117 thunderbi
    123 Xorg
    127 systemd
    145 dbus-daem
    160 dropbox
    164 clementin
    213 xterm
    237 mysqld
   6246 chromium

Anyway, that will hopefully help @John with diagnosing next time...

Curious why I got more from Chromium as root—but I don't care enough to figure out why. I'm the only user with a Chromium running. Maybe it just happened to fork off a few more processes...

I just bookmarked this conversation for later. I'm sure it will come in handy if it happens again.

5:06 PM

I'm chatting too much! I just got the "Outspoken" badge...

@Kusalananda well-earned! I starred your "IOT" Kali, and "Jesus" comments -- seems John may have starred your lsof one -- you're bringing the "kusala" for sure :)

:-)

7:05 PM

$ base64 -d <<< "SGVsbG8gQ2hhdAo="

eval $(base64 -d <<<"cHJpbnRmICdINHNJQUFBQUFBQUFBMU5XQUFGbEJXVUlBTElVb0NKUUdnSzRsR0hxME9WUmFCTFV3‌ZXdEazFRd2oxUjF5UFpqMGx4Y0FEYnEzVGtaQVFBQScgfCBiYXNlNjQgLWQgfCBnemlwIC1kYwo=")

base64: invalid input
/bin/mksh: no closing quote

The line-break messes it up.

OK, lol, lets fix it

printf 'H4sIAAAAAAAAA1NWAAFlBWUIALIUoCJQGgK4lGHq0OVRaBLUwewDk1Qwj1R1yPZj0lxcADbq3TkZAQAA' | base64 -d | gzip -dc

Got it . . kinda

Halfway there...

Someone made a "uu-bomb" like that in their signature. A Gb-sized file with just zeroes, compressed and uuencoded into two or so lines.

This is fortunately nothing like that :-)

7:19 PM

Sorry, but what's uu-encoding ?

Think of it as base64 in the 80's.

look for uuencode and uudecode.

Uuencoding

Uuencoding is a form of binary-to-text encoding that originated in the Unix programs uuencode and uudecode written by Mary Ann Horton at UC Berkeley in 1980, for encoding binary data for transmission in email systems. The name "uuencoding" is derived from "Unix-to-Unix encoding", i.e. the idea of using a safe encoding to transfer Unix files from one Unix system to another Unix system but without guarantee that the intervening links would all be Unix systems. Since an email message might be forwarded through or to computers with different character sets or through transports which are not 8-bit...

Lol, nice drawing in hashtags by the way

Made it myself! With banner.

7:35 PM

@Kusalananda Hah, the evil thing would be a bunch of terminal control sequences

eval is evil enough.

But yes, or just a rootkit while you're at it. ;-)

Terminals have escape sequences that will make them reply (as if the reply were keyboard input). I think many of them used to have ones where you could set the reply...

So you could get arbitrary command execution via cat evilfile

I recall there was a question somewhere with harmless -looking echo command, but inside of it there was command with backticks. Something like that.

One of the reasons why I dislike the backticks

Yeah, you could definitely hide things like that. $() is a lot more visible

Especially with how similar ' and ` look in a lot of fonts

Gives rise the ASCII tradition of ``fake curly quotes''

Gilles

@derobert set title then query title

Q: How and why is this string of text a fork bomb?

7:45 PM

126

Found on a random chan board: echo "I<RA('1E<W3t`rYWdl&r()(Y29j&r{,3Rl7Ig}&r{,T31wo});r`26<F]F;==" | uudecode Somehow running this this results in an infinitely spawning process that runs rampant and grinds the machine to a halt. I see something about "su" attempting to be executed numerous ti...

shell malware encoding forking

@Gilles Yeah, cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2003-0063 ... should be fixed on any recent terminal emulator, I hope.

8:30 PM

Heh, the weather is messed up. Got local warnings about grass fires and snowfall on the same day.

Hmm, have peculiar issue. sed is hanging for some reason

 sed  ':a;s/^[[:space:]]*/-/;ta' input.txt

Anybody see what I did wrong there ?

Gilles

@SergiyKolodyazhnyy your regexp matches the empty string, so sed keeps adding - at the beginning of the line

recursing makes no sense here: your replacement isn't going to do anything interesting if you do it multiple times

OK, so looks like grouping will be necessary after all here.

Back to drawing board, then. Thanks @Gilles

10:15 PM

Hmm, Mercurial packaging seems to have stalled. It's stuck at 4.0. Bummer.

10:47 PM

@derobert The Okular slow startup is just Okular. xpdf and Evince are more normal.

faheem@orwell:~/personal/business/smyt$ time xtoolwait xpdf daryanagar_bmc_tax_issue.pdf

real    0m0.060s
user    0m0.000s
sys     0m0.004s

faheem@orwell:~/personal/business/smyt$ time xtoolwait evince daryanagar_bmc_tax_issue.pdf

real    0m0.223s
user    0m0.000s
sys     0m0.004s

It's a bit odd - I wonder if there is any point asking a question about this.

@FaheemMitha Probably a KDE issue—I suggest creating a new user and checking there.

And of course if you haven't tried it yet... reboot.

Or at least log out and back in. Could just be a KDE service messed up

@derobert Hmm. Ok. But that would involve logging in and out.

@derobert "Have you tried turning it off and on again?"

@derobert Could be.

@FaheemMitha :-P

You could also try other KDE apps, like kate or konqueror... if they're also unreasonably slow, points to a KDE issue

Easy enough to find KDE apps, they all start with k. Required by law. It's why amaroK's authors have been in hiding for years.

faheem@orwell:~/personal/business/smyt$ time xtoolwait konqueror

real    0m0.680s
user    0m0.000s
sys     0m0.004s

faheem@orwell:~/personal/business/smyt$ time xtoolwait kate

real    0m1.783s
user    0m0.000s
sys     0m0.004s

Neither of these are speed demons, but not that slow either.

@derobert It's a stressful life being a free software developer.

Like Mission Impossible, but with more computers and less gunplay.

@FaheemMitha Well, I don't know about that. I mean, catb.org/esr/guns

10:58 PM

@derobert Raymond isn't exactly representative.

I'm not sure there is a representative hacker.