The last system call before the SIGSEGV is write(1, "\n", 1) = 1.

Can you tell which executable? Sulogin or a shell?

It would also be useful to see which files were accessed just before it died. Maybe a PAM module can't open a file.

It's /sbin/sulogin. I wish I could paste the entire log here, but I'm limited to a few hundred characters and cannot insert newlines. The last two system calls were ioctl(0, ...) and write(1, ...).

I added an ltrace command to my answer. Can you try that? It will log the library calls.

At boot time, I see: The disk drive for /var/opt/companyname/productname/data is not ready yet or not present. keys:Continue to wait, or Press S to skip mounting or M for manual recovery I then press M for manual recovery.

I've run ltrace and the last two library calls are: 837 crypt("password", "x") = nil 837 strcmp(nil, "x" <no return ...> 837 --- SIGSEGV (Segmentation fault) ---. This looks like a bug in sulogin to me.

I looked at the source. strcmp(crypt(...),"encryptedpassword"). Not as robust as it could be, but the problem is that crypt() is returning NULL, which is very uncommon.

The man page for crypt() suggests that it returns NULL if an error is encountered. It seems there should be a test for a NULL return from crypt(). Should I file a problem report?

Was the 2nd argument to crypt and strcmp really "x", or something else?

I assume it was "x", that ltrace logged it correctly.

It ought to have gotten the encrypted password from /etc/shadow, not "x". Can you see if root has a sensible encrypted password in /etc/shadow ?

The command sudo getent shadow root returns root:x:16273:0:99999:7:::.

So you're successfully logged in on the problem system now? Does it use something other than shadow for passwords? NIS or LDAP?

I am able to log in if I replace the original data disk or format the new one. We do not use LDAP or NIS. I access the problem system by logging in to the admin account over SSH.

I believe we have disabled login as root but do not recall how.

I can reproduce the segmentation fault if /etc/shadow has "x" as the encrypted password for root. But it shouldn't be "x". It is "!" by default, which means sulogin won't ask for a password for the maintenance shell. Maybe this got changed awhile back and you never had to use the single-user root login program since then?

You are correct. It did get changed and I have never had to use the single-user root login program since then.

Why do you say it shouldn't be "x"? The superuser can change the root password to any arbitrary text string.

I still think that there should be a test to determine whether or not the call to crypt() succeeded. It just seems good programming practice to me.

I'm filing a problem report against sysvinit-utils.

The password can be anything, but the encrypted password will be a string of whatever length the configured encryption method requires (13 chars for DES, somewhat more for md5 or sha). You should be able to put an asterisk or "x" or "xyzzy" there to disable an account, and it's a bug if the login software breaks.

Agreed.

Thanks so much for helping me sort this out.

Sure.