Island of castaway thoughts

Eliah Kagan

03:23

in Raiders of the Lost Downboat, 20 hours ago, by Zanna

sorry, everything closed and I got logged out. Probably OOM

Shouldn't that just kill your browser (the program using the most memory) but leave your desktop session up?

Eliah Kagan

14:09

@Zanna I realized I needed to show that .bashrc is sourced in some non-login non-interactive shells in like half a comment, and sot it could be tested in ~1 min. Having realized that, I immediately figured out how to do it, though this doesn't cover checking if /etc/bash.bashrc is ever sourced automatically in non-login non-interactive shells, and also I will have to undertake a bit of effort to figure out how to summarize it (i.e., what I am about to say is longer than half a comment).

Zanna

sounds like good progress...

Eliah Kagan

What I did was first to look through /etc/profile and /etc/bash.basrhc to make sure neither sourced any files in the user's home directory. Then I created a test user as and modified .profile and .bashrc to report when they were running (and, in the case of .bashrc, when it was before or after the interactivity check). Then:

$ ssh [email protected] 'echo $-; shopt login_shell'
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
hBc
login_shell     off

It's not a login shell, and there is no i in $- so it's noninteractive. .profile is not sourced, .bashrc is sourced, and code in it before the check for whether or not the shell is interactive runs, but code after that check does not run.

Logging on as as normally shows all three lines (after the motd and before the prompt):

In ~/.profile, after bash check.
In ~/.bashrc, BEFORE interactivity check.
In ~/.bashrc, AFTER interactivity check.

Zanna

haha very nicely done

Eliah Kagan

Thanks. Unfortunately, this isn't quite enough for me to figure out how to fix my answer that contains wrong information in its "Where to Put Commands to Run in non-Interactive Shells" section.

Zanna

so, a noninteractive shell does source .bashrc if it knows its running over ssh (and generally won't do anything about it, since the first part of the file tells it not to do anything)

which is what the documentation says... right?

do you want tp keep the title of the section the same? Where does one put commands for a noninteractive shell that doesn't source .bashrc?

Eliah Kagan

14:19

@Zanna Likely still in .bashrc, if the noninteractive shell is a login shell and one has not also taken the very weird and generally inadvisable step of removing the code in .profile that sources .bashrc.

Zanna

what if it's not a login shell?

Eliah Kagan

You can use $BASH_ENV, but it's very rare to do so, because one does not usually want anything sources in that situation.

@Zanna Yes, except that I don't actually know quite what "Bash attempts to determine when it is being run with its standard input connected to a network connection" in the documentation means. Only the shell invoked by ssh seems to work this way. Not subsequent shells run from within the session.

Zanna

hmm. maybe there's no way for them to tell

Eliah Kagan

Well it's probably good that they do not source it. Otherwise it would get sourced again when you started a shell to run a script file or commands with -c.

Zanna

@EliahKagan yeah, as you explain in your answer...

Eliah Kagan

14:30

Well, I don't mention BASH_ENV.

Android Dev

So I decided to post an SO question about an issue that TheWanderer and I had like 3 months ago :D

I have no idea if anyone will ever have that particular issue, but shrug

It's there just in case

Zanna

I mean you explained that it was a bad idea

@AndroidDev nice!

Android Dev

thanks, but the code in the answer is super ugly...

we probably should have just used a custom toolbar

Zanna

I'm too clueless to tell how ugly your code is :D I'm thinking "hmm java is so readable"

I guess it seems like there would be a way to avoid repeating some of the things that are the same, but, I guess not

well done for solving your problem

Android Dev

@Zanna oh shoot, there is way, Zach did that later but I grabbed the code from the original commit... lemme fix that

ok I updated the code

@Zanna it's mainly ugly because you're going through so many different view hierarchy to get to it

Eliah Kagan

14:45

@Zanna Well that's one benefit of doing a lot of shell scripting. Most all other programming languages then feel intrinsically clean and readable. :)

@Zanna I think Bash must check that the standard streams are all pipes and examine what is at the other ends of them (or maybe it just looks at one of them). I was actually mistaken in my claim that in, bash -c ..., the shell doesn't source .bashrc. It does if it's in the command you tell ssh to use the shell to run.

Zanna

@EliahKagan haha yeah I do have that reaction mostly. But my brother has to write Objective C code and he sends me things saying "hey look what I made" and I'm like o.O "what is it?"

Eliah Kagan

He hasn't moved to Swift?

Zanna

mm that would be much nicer

Eliah Kagan

Should we move some of these messages (including the discussion of my post, which has become long) to the island?

Android Dev

@Zanna oh he does iOS development as well as Android?

Zanna

14:58

yeah

Eliah Kagan

Shall I do it, or do you want to?

Android Dev

I think she was replying to me.

Zanna

I'll do it :)

Eliah Kagan

@AndroidDev You're right. Sorry about that.

Zanna

@AndroidDev do you mind if I move your messages about your post to the island?

Eliah Kagan

14:59

@Zanna Ok.

Android Dev

@Zanna nope :)

Zanna

@AndroidDev I was, but my bad for being ambiguous

← 36 messages moved from Raiders of the Lost Downboat‌

← 49 messages moved from Raiders of the Lost Downboat‌

Eliah Kagan

15:18

Shouldn't there be some process related to the ssh connection that has the pipes open?

$ ssh [email protected] 'ls -l /proc/self/fd/{0..2}; lsof 2>/dev/null | grep -Ff <(readlink /proc/self/fd/{0..2} | grep -oP "\d+")'
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
lr-x------ 1 as as 64 Oct 13 11:10 /proc/self/fd/0 -> pipe:[4149853]
l-wx------ 1 as as 64 Oct 13 11:10 /proc/self/fd/1 -> pipe:[4149854]
l-wx------ 1 as as 64 Oct 13 11:10 /proc/self/fd/2 -> pipe:[4149855]
bash      27805               as    2w     FIFO               0,10      0t0    4149855 pipe
lsof      27807               as    1w     FIFO               0,10      0t0    4152916 pipe

Oh.

Those aren't the pipes I was looking for.

Zanna

haha

Eliah Kagan

Yeah, of the original pipes, lsof is only showing the one for stderr. (This is unaffected by the 2>/dev/null redirection.) I mean, the others are not shown, anyway.

$ ssh [email protected] 'ls -l /proc/self/fd/{0..2}; lsof | grep -Ff <(readlink /proc/self/fd/{0..2} | grep -oP "\d+")'
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
lr-x------ 1 as as 64 Oct 13 11:20 /proc/self/fd/0 -> pipe:[4157568]
l-wx------ 1 as as 64 Oct 13 11:20 /proc/self/fd/1 -> pipe:[4157569]
l-wx------ 1 as as 64 Oct 13 11:20 /proc/self/fd/2 -> pipe:[4157570]
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.

The problem is that, to extract and operate on the information from proc, I am causing it to report the wrong standard streams, because they are being redirected, or explicitly piped, or implicitly piped due to process substitution.

Zanna

haha uncertainty principle

Eliah Kagan

15:37

In the first terminal:

$ ssh [email protected] 'ls -l /proc/self/fd/{0..2}; echo $$; sleep 1d'
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
lr-x------ 1 as as 64 Oct 13 11:36 /proc/self/fd/0 -> pipe:[4164576]
l-wx------ 1 as as 64 Oct 13 11:36 /proc/self/fd/1 -> pipe:[4164577]
l-wx------ 1 as as 64 Oct 13 11:36 /proc/self/fd/2 -> pipe:[4164578]
28728

In the second terminal:

$ ssh [email protected]
[email protected]'s password:
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-97-generic x86_64)

 * Documentation:  help.ubuntu.com
 * Management:     landscape.canonical.com
 * Support:        ubuntu.com/advantage

0 packages can be updated.
0 updates are security updates.

New release '17.04' available.
Run 'do-release-upgrade' to upgrade to it.

*** System restart required ***
Last login: Fri Oct 13 11:38:24 2017 from 192.168.10.4
In ~/.profile, after bash check.

That sidesteps the problem of looking at the wrong pipes. And... shouldn't there be some process related to the ssh connection that has the pipes open?

Zanna

hmm...

Android Dev

uhm why did I get a notification that I was invited here?

because my message was moved maybe?

Eliah Kagan

If it was an invite, it should have said what user invited you. Did it say it was one of us?

Android Dev

Zanna

maybe it was because some of the messages that were moved here are pings to you

Android Dev

16:07

hey @Zanna remember the Nvidia bug question? So I think I may have just uncovered that it isn't 100% fixed

look at that --^

doesn't it look like the same kind of video memory corruption or whatever it was?

Eliah Kagan

What question was that? (If you don't mind my asking.)

Android Dev

@EliahKagan you mean you don't remember THE Nvidia bug question that exploded a few months ago? Just kidding :D Here you go: askubuntu.com/q/896221/518562

It also happens to be a question that I asked and answered

Zanna

@AndroidDev uh oh

there was even a meta question about it because someone wanted to make it community wiki

Android Dev

someone who is the only person allowed to awooooo

in Ask Ubuntu General Room, May 30 at 18:18, by Kaz Wolfe

(7) Nathan Osman may not awoo. He is not a wolf.

12

Q: Should the NVIDIA bug question be made a community wiki?

NVIDIA screwed up and pushed some buggy drivers, affecting many users who are coming to AU with the same issue: Strange artifacts along window borders after waking computer from sleep mode The question has a list of affected cards at the end: Someone has proposed that this question be made a...

Eliah Kagan

@AndroidDev Thanks.

I do remember now. I had voted on it and its answers, actually.

Android Dev

16:20

that Q got me the 10k views badge :)

Eliah Kagan

@Zanna Well, here's an unmangled way to show what is confusing me (it's better to use $$ than self since the shell expands $$):

$ ssh [email protected] 'ls -l /proc/$$/fd/{0..2}; lsof 2>/dev/null | grep -f <(readlink /proc/$$/fd/{0..2} | grep -oP "\d+")'
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
lr-x------ 1 as as 64 Oct 13 12:23 /proc/29647/fd/0 -> pipe:[4176399]
l-wx------ 1 as as 64 Oct 13 12:23 /proc/29647/fd/1 -> pipe:[4176400]
l-wx------ 1 as as 64 Oct 13 12:23 /proc/29647/fd/2 -> pipe:[4176401]
bash      29647               as    0r     FIFO               0,10      0t0    4176399 pipe
bash      29647               as    1w     FIFO               0,10      0t0    4176400 pipe

I figured it out.

Actually, partially never mind.

I went back and did it that way again, but on the second terminal I logged in as as a user in the sudo group and ran lsof as root. It showed the sshd process on the other ends of the pipes. But this was a fork of sshd running as as (to drop privileges for security, like you'd expect from OpenSSH), so I don't know why I had needed to run lsof as root to see it.

Was I wrong in this answer too when I said you don't usually need to run lsof as root to see your own processes?

ek@Io:~$ sudo lsof | grep -E '^COMMAND|417702[678]'
[sudo] password for ek:
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
COMMAND     PID   TID       USER   FD      TYPE             DEVICE    SIZE/OFF       NODE NAME
sshd      29779               as   10w     FIFO               0,10         0t0    4177026 pipe
sshd      29779               as   11r     FIFO               0,10         0t0    4177027 pipe
sshd      29779               as   13r     FIFO               0,10         0t0    4177028 pipe

But the as user can't see its own sshd with lsof!

@Zanna Apparently that is the reason for the invitation. @AndroidDev

10

Q: Don't invite people when moving their messages

Every time we move people's messages to some recycle bin/trash can room, they're invited to this room, thus leading to sarcastic comments such as this one. Should these invitations be disabled? I haven't seen any useful situation for those invitations when moving a message. If you think there a...

discussion feature-request status-declined chat

Android Dev

16:58

That's what Gradle does to your CPU --^

Zanna

@AndroidDev lol

Android Dev

and that's a 3rd gen i7 mind you

Zanna

17:35

I got an OOME when I was trying to use it

but it ran with browsers closed haha

@EliahKagan aah thanks!

@EliahKagan seems strange

I wonder if that's why the trash can is deleted

Eliah Kagan

Well, I think I don't need to figure that out in order to fix my answer about Bash startup files. Though I will want to look into it in case I also need to fix my answer about checking what's writing to a particular terminal. Perhaps "Island of castaway thoughts" should be renamed to "Are any of Eliah's answers actually correct?"

So on the one hand, this is vague:

> Bash attempts to determine when it is being run with its standard input connected to a network connection, as when executed by the remote shell daemon, usually rshd, or the secure shell daemon sshd.

On the other hand, if I actually say what happens, will that end up being specific to just some versions of Bash? That is, if something isn't included there, doesn't that suggest that it's especially subject to change (even without the change being listed in the changelog summary associated with the Bash release)?

The run_startup_files function in shell.c shows what happens.

  /* get the rshd/sshd case out of the way first. */
  if (interactive_shell == 0 && no_rc == 0 && login_shell == 0 &&
      act_like_sh == 0 && command_execution_string)
    {
#ifdef SSH_SOURCE_BASHRC
      run_by_ssh = (find_variable ("SSH_CLIENT") != (SHELL_VAR *)0) ||
           (find_variable ("SSH2_CLIENT") != (SHELL_VAR *)0);
#else
      run_by_ssh = 0;
#endif

      /* If we were run by sshd or we think we were run by rshd, execute
     ~/.bashrc if we are a top-level shell. */
      if ((run_by_ssh || isnetconn (fileno (stdin))) && shell_level < 2)

Zanna

aah that's illuminating

Eliah Kagan

Yes.

And SSH_SOURCE_BASHRC is defined in config-top.h. That's the upstream source code (I'm only linking to it on Launchpad because it has syntax highlighting and Savannah doesn't) but when I use apt source bash and examine config-top.h in the patched downstream version, the #define directive for it is uncommented.

$ ssh [email protected] 'echo $SHLVL'
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
1
$ ssh [email protected] 'bash -c '\''echo $SHLVL'\'''
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
2
$ ssh [email protected] 'SHLVL=0 bash -c '\''echo $SHLVL'\'''
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
In ~/.bashrc, BEFORE interactivity check.
1

And (in my testing) it doesn't do it without SSH_CLIENT:

$ ssh [email protected] 'env SHLVL=0 bash -c '\''echo $SHLVL'\'''
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
In ~/.bashrc, BEFORE interactivity check.
1
$ ssh [email protected] 'env -u SSH_CLIENT SHLVL=0 bash -c '\''echo $SHLVL'\'''
[email protected]'s password:
In ~/.bashrc, BEFORE interactivity check.
1

Eliah Kagan

18:15

Checking the shell level is apparently necessary because, otherwise, .bashrc will get sourced again when bash -c command (or, presumably, bash scriptname) is used during a running SSH session. This is while logged in as as via SSH:

as@Io:~$ bash -c 'echo $SHLVL'
2
as@Io:~$ SHLVL=0 bash -c 'echo $SHLVL'
In ~/.bashrc, BEFORE interactivity check.
1
as@Io:~$ env -u SSH_CLIENT SHLVL=0 bash -c 'echo $SHLVL'
1

SSH_CLIENT is not just a necessary condition (in the way I'm testing this), but also a sufficient condition to make a non-login noninteractive "initial" shell source .bashrc automatically. This is from inside a (locally running) terminal window on the machine:

ek@Io:~$ su - as
Password:
In ~/.profile, after bash check.
In ~/.bashrc, BEFORE interactivity check.
In ~/.bashrc, AFTER interactivity check.
as@Io:~$ SHLVL=0 bash -c 'echo $SHLVL'
1
as@Io:~$ SSH_CLIENT= SHLVL=0 bash -c 'echo $SHLVL'
In ~/.bashrc, BEFORE interactivity check.
1
as@Io:~$ exit
logout

Android Dev

I just discovered that Java 9 has scripting support! docs.oracle.com/javase/9/jshell/…

It's called JShell

Eliah Kagan

@AndroidDev Yeah, I had tried that out with the Java 9 beta. It's awesome!

Android Dev

drools

Eliah Kagan

Another thing I like about Java 9 is that it adds some much needed methods to the stream interfaces (e.g., Stream, IntStream) that had probably been omitted before because of the heavy emphasis on parallelism, like TakeWhile.

@EliahKagan The remaining question is, what does the isnetconn function do, and why doesn't it know there's an SSH connection even when SSH_CLIENT is unset? Why is it sufficient only for RSH (or at least, why is it insufficient for SSH in my tests)?

Android Dev

18:30

pinging yourself? lol

Eliah Kagan

@AndroidDev It's a reply-ping, so it indicates what previous message it is related to.

I find it useful sometimes.

@EliahKagan I meant takeWhile, of course, not TakeWhile. :)

Android Dev

18:55

hahaha :)

Eliah Kagan

19:46

@Zanna I think sshd and .bashrc provides the other half the illumination. It addresses why Bash doesn't know it's an SSH session without SSH_CLIENT.

Zanna

haha lots of digging required!

Eliah Kagan

Well that came up pretty high in Google search results when I included isnetconn. (I didn't spend all this time looking for it. Rather, I was doing other stuff and only recently returned to looking into it.)

So now I have to figure out if /etc/bash.bashrc is also sourced under these conditions. This is not as simple to figure out because at the moment I don't have the ability to quickly and easily use a virtual machine and I don't want to actually modify that file. (I could use debootstrap and chroot but I'd have to set that up.) The relevant part in shell.c is:

      if ((run_by_ssh || isnetconn (fileno (stdin))) && shell_level < 2)
	{
#ifdef SYS_BASHRC
#  if defined (__OPENNT)
	  maybe_execute_file (_prefixInstallPath(SYS_BASHRC, NULL, 0), 1);
#  else
	  maybe_execute_file (SYS_BASHRC, 1);
#  endif
#endif
	  maybe_execute_file (bashrc_file, 1);
	  return;
	}
    }

Zanna

maybe_execute_file :)

Eliah Kagan

The answer appears to be yes.

:)

Zanna

hahaha

Eliah Kagan

19:56

In config-top.h:

/* System-wide .bashrc file for interactive shells. */
/* #define SYS_BASHRC "/etc/bash.bashrc" */

That #define is uncommented in the files downloaded, unpacked, and patched via apt source bash on my 16.04 system.

Let's see if I can actually test this (safely).

I can't.

I mean, not as safely as I want. I really don't want to mess with /etc/bash.bashrc on a system I'm using for a whole bunch of other stuff, and the first line that isn't blank or a comment in that file checks if the shell is interactive and quits if it isn't.

Zanna

well I don't mind messing with mine

Eliah Kagan

Actually I just figured out a way to test is safely.

@Zanna I spoke too soon! I can do it.

Zanna

@EliahKagan I expected that

Eliah Kagan

In an SSH session:

ek@Io:~$ strace bash -c '' |& grep bashrc
ek@Io:~$ SHLVL=0 strace bash -c '' |& grep bashrc
open("/etc/bash.bashrc", O_RDONLY)      = 3
read(3, "# System-wide .bashrc file for i"..., 2188) = 2188
open("/home/ek/.bashrc", O_RDONLY)      = 3
read(3, "# ~/.bashrc: executed by bash(1)"..., 3760) = 3760
ek@Io:~$ env -u SSH_CLIENT strace bash -c '' |& grep bashrc
ek@Io:~$ env -u SSH_CLIENT SHLVL=0 strace bash -c '' |& grep bashrc

In a non-SSH (locally running) terminal window:

ek@Io:~$ strace bash -c '' |& grep bashrc
ek@Io:~$ SHLVL=0 strace bash -c '' |& grep bashrc
ek@Io:~$ SSH_CLIENT= strace bash -c '' |& grep bashrc
ek@Io:~$ SSH_CLIENT= SHLVL=0 strace bash -c '' |& grep bashrc
open("/etc/bash.bashrc", O_RDONLY)      = 3
read(3, "# System-wide .bashrc file for i"..., 2188) = 2188
open("/home/ek/.bashrc", O_RDONLY)      = 3
read(3, "# ~/.bashrc: executed by bash(1)"..., 3760) = 3760

Zanna

20:12

oh smart :)

Eliah Kagan

Given the rest of what we know about how those environment variables change Bash's behavior with respect to .bashrc and also what the source code says, I think that's sufficient confirmation that it sources /etc/bash.bashrc as well as ~/.bashrc for non-login non-interactive shells that it thinks are initial shells of an SSH session.

Zanna

:)

Eliah Kagan

That means I'm half done figuring this out!

The other half is, what should my answer actually say?

Zanna

yeah :S

Eliah Kagan

20:29

I might make a Q&A about this. But I want to edit that answer into something that's 100% correct, not misleading, still useful, and that I am happy with, before I do that. If I make a self-answered question about this, I would want that to be "further reading," not something people have to click on and read to avoid being confused in the first place .

Zanna

summary of stuff... (not suggesting, just what I understood so far)
noninteractive shells that are login shells source rc files because profile files source them (unless they don't)
usually noninteractive nonlogin shells don't source rc files except in {circumstances} but since the first uncommented part of those files checks that the shell is interactive, they usually don't affect it. However, you could put some commands before those tests, if you really want to.
If this kind of shell isn't going to source any rc files, it will try to expand $BASH_ENV, so you can use that if you really rea…

Eliah Kagan

21:02

@Zanna When is $BASH_ENV actually used? Is it used just when "rc" files are not sourced? Or is the file named by that variable, if any, also sourced in some situations where "rc" files are sources, such as in an initial non-login non-interactive shell in an SSH session?

Zanna

21:14

Hmm

Eliah Kagan

22:11

s/sources,/sourced,/

Transcript for

Island of castaway thoughts