« first day (245 days earlier)      last day (1066 days later) » 
17:00 - 23:0023:00 - 00:00

5:50 PM
uh... that's an XY problem but... what did they do?!?
6:18 PM
they actually mv'd libc... what instructions would say that? trolling instructions...
@Zanna Busybox should still work because it is statically linked.
awesome if we could fix that problem...
Nowadays, busybox even has a dpkg command.
Maybe that will help.
if they can reinstall it...
Didn't they just rename libc.so.6?
If so, they can use /bin/busybox mv to rename it back.
Or they can run /bin/busybox to get a shell and run mv from there.
But they can also just boot a live USB and rename it.
They don't have to chroot for that, just rename the file.
Am I missing something?
6:24 PM
no not at all
So, if they have a shell, then they should do it the busybox way.
of course you can always fix it from live USB, but that's boring :)
So... is that just the answer?
@EliahKagan that would be great
No, that won't work.
The other dimension of the problem is that they need to rename it back as root.
6:25 PM
For some reason I thought their shell was a root shell but of course it's not and they're trying to use sudo.
which is an external program that can't finds its libs
7:11 PM
@Zanna They're expecting that LD_PRELOAD would enable them to make sudo run with an alternate libc. Obviously that's prohibited for security reasons--the UID and EUID have to be the same for LD_PRELOAD to have an effect, prohibiting setuid executables like sudo from having their behavior modified when they are run as non-owners. I've posted an answer.
@EliahKagan I was hoping you would do that :)
I actually had the tab hopefully open and kept peeking at it
I don't think my answer is very good. I think it's correct, but I'm not sure it's well presented.
@Zanna Hoping I'd post an answer in general, or hoping I'd post an answer about LD_PRELOAD and setuid executables specifically?
OP read your answer really fast...
an answer that I would find illuminating
Regarding the LD_PRELOAD question, they've commented. I guess I should edit my answer to make more explicit that the same holds for su -- I think they understand that from my answer, but some readers might not. When I do, I might also add something about how they have the option of booting with init=/bin/busybox as a kernel option.
Sorry, I mean with init=/bin/busybox sh.
I hope this comment makes sense. I'll be afk for a few minutes shortly, then for a while longer.
If I understand them right, they already fixed it and that forthcoming edit of mine isn't urgent.
7:42 PM
I'm not sure
About which?
Their comment? My comment? Something else?
Or is this about the question that was close POB?
@EliahKagan haha sorry, I mean, I'm not sure that they fixed it, from their comment which sounds like they did fix it, but might actually be expressing an intention to fix it
I think they gave up on trying to fix it without rebooting and booted from a live DVD before seeing my answer (or perhaps they accessed the site from the live environment). That seems like the most likely explanation for their very quick reply and accept. However, I admit this isn't the only possibility.
yeah, I also thought they had probably already started doing that, having realised nothing was working
7:58 PM
I'm actually not sure how to do this and I don't have the ability to easily test it right now.
Kernel boot options in GRUB can be specified with double quotes; that's no problem. But does init="/bin/busybox sh" cause /bin/busybox to be used as the init process, with sh passed as its first command-line argument (which is what I want), or does it attempt to use a (presumably nonexistent file) whose full path is /bin/busybox sh as the init process?
Btw, thanks for the edit!
do you want me to try it? :D
most welcome, thanks for the answer...
@Zanna because I can definitely test that much
I am even willing to rename libc... right after I check that I still have some live USBs that boot on this system
Should this be an addition to my answer? Or should it be a separate answer. You had pointed out in comments (since deleted) that regular recovery mode won't work.
If you think it shouldn't be a separate answer then I am totally willing to add it to mine.
It's just... not only have I not tested it, I haven't written the instructions yet and I'd have to check how to do it. I haven't messed with boot options in GRUB for over a year now. It's terrible! :)
Me not having done that lately, I mean.
perhaps I should not have deleted those comments
they were orphaned
(I have a live system that boots, so I don't have to do anything time-consuming to test breaking stuff)
8:13 PM
The interface for changing boot options in GRUB for a live system is different from that for an installed system, right?
it seems like it would be at home in your answer
That may not be a problem, though. The syntax should be the same.
Okay, I can definitely put it in my answer.
@EliahKagan I don't think I've done that... but why would I need to change boot options in the live system? I thought we did that by pressing F6... but it was a GUI-based thing
@Zanna Yeah, F6. That doesn't work for GRUB on an installed system, though, does it?
On an installed system you hold Shift (if necessary) to get a list of kernels. Then select one and press the key to edit its config temporarily. I think that key is Escape. Then add the boot option. I think that's right.
you have to press e on the GRUB menu
maybe what you are suggesting also works... hmm... which kernel boots when I do it the usual way?
tests oh, you select the kernel by highlighting, then you press e
8:23 PM
@Zanna That's in the GRUB menu when GRUB is installed on the hard disk?
however, I just tried adding init="/bin/busybox sh" to the boot parameters, and my system booted absolutely normally
@EliahKagan yes
@Zanna Darn.
I always get the GRUB menu (by force) because shift doesn't work for me
nothing in fact works for me except forcing the menu. So I just force it to come up every time, because I very often have to boot a different kernel for some stupid testing or tinkering
Forcing it... how?
8:27 PM
@Zanna I know who might know...
if I put init=/bin/busybox can't I just type sh when I get there?
@Videonauth Do you know if it's possible to make GRUB boot the kernel with busybox sh as an alternate init? This is like the old init=/bin/sh trick, but with /bin/busybox sh instead for systems where the reason you have to do it is to fix libc (which /bin/sh depends on and /bin/busybox doesn't).
@Zanna Doesn't busybox print its list of commands and exit when run with no arguments?
sadly not
Do you mean that it's not possible, or that you don't know if it is possible?
Also, hi @Videonauth! :)
@EliahKagan no clue if that is possible
never tried
guess best way to fix pibc is to boot from a stick and then chroot
8:33 PM
OP only renamed the file, so no need to chroot
well then rename it back
@EliahKagan well what happened when I tried doing what I said I was going to do was in fact a kernel panic
from live image boot and then mount rw and rename it back
never played around with busybox and the like
yes, we are trying to find a more interesting method though
ah ok
8:35 PM
Yeah, I'm hoping to add an alternative method to:
A: How can I fix a missing/renamed libc.so.6?

Eliah KaganYou should just boot from a live USB (or CD/DVD) and rename libc.so.6 back. LD_PRELOAD doesn't work with setuid executables. But, if you want, you can use /bin/busybox to do last-minute backups (that don't require running commands as root) before rebooting. Currently you are attempting to make ...

maybe I want to quote the whole parameter, not just the command...
You could try that. You could also try omitting quotes.
I got a clean normal boot again when I tried quoting the whole thing
@EliahKagan but then init=/bin/busybox and sh will be interpreted as separate boot parameters, no?
I don't know how an unquoted word that doesn't contain = is treated.
what about quiet splash etc?
8:40 PM
Do those come before the ones with = or after?
But yeah you're probably right.
Yeah you're definitely right.
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-4.13.0-39-generic.efi.signed root=UUID=c30dfc4b-772d-4f9c-8d04-193f9e35f4ed ro "init=/bin/busybox sh" initcall_debug no_console_suspend intel_idle.max_cstate=1
still, I can totally try it that way...
You can. I doubt it would work. You're right it shouldn't.
I need to brush up on this. @Videonauth had you been saying in AUGR that cross-compiling a kernel for ARM is hard? I should attempt that immediately. :)
I've compiled kernels a number of times but never cross-compiled one for anything.
@Zanna So, first, what happens when you do init=/bin/sh or init=/bin/bash? Does that give you a shell instead of starting up normally?
If not, then the problem may not be the way in which busybox sh is being expressed.
@EliahKagan IT WORKED!
8:47 PM
Huh. Okay. What about without the sh, then? Maybe busybox behaves differently when its PID is 1.
without the sh I just get a kernel panic, which is also what happened when I typed exit just now when it did work
so if busybox exits, I get a kernel panic
@EliahKagan shall I try it?
@Zanna You can reboot cleanly (with a reasonably proper shutdown) with Ctrl+Alt+Delete from within that shell, though, right?
I'll try that next time haha
@EliahKagan at least tricky
what else shall I do?
8:50 PM
Or maybe: exec /sbin/init
^^^ To attempt to boot normally. Idk about that.
that is a good idea :)
@Zanna That shouldn't be necessary, considering that init=/bin/busybox sh worked.
(sorry for shouting earlier)
So is init=... at the end of the kernel boot options in the GRUB line?
I put it at the beginning (because it's tedious moving the cursor)
8:53 PM
How does it know to pass sh as a command-line argument if it's not passing whatever else comes after it as a command-line argument? Is there a -- somewhere or something? Does the kernel treat that specially. I recall something about it doing so. (It's not obvious that it would.)
...Why did I think it might work without the quotes?
I mean it did, but why did I think it did?
I'm going to pretend say that I remembered something about that.
maybe the rest of them got passed as command line arguments too for good measure, but none of them meant anything
Well you can check that.
there would be some errors
there were some errors, but I was so shocked it had actually worked I didn't read them
so what should I do now?
It should try to use them as scripts to run. But if for some reason it didn't, it would have them in "$@".
8:59 PM
Does it also work if you put it at the end?
I'll try that, but anything else to try while I'm there?
I'm doing it on the same device I am using to chat right now...
Let's see...
Did you try rebooting with Ctrl+Alt+Delete?
Also, did you try:
exec /sbin/init
I'll get my phone haha
Actually never mind all that.
busybox has a reboot builtin. I can just say to use that in my answer.
I mean, you can try it. If you do I might mention if it works, if I have room. But it's not at all important. They and other readers would probably just want to run reboot after restoring the backup.
@EliahKagan doesn't work for me
exec /sbin/init returns Couldn't find an alternative tellinit implementation to spawn
reboot returns nothing and nothing happens
Ctrl Alt Del returns me tidily to the GRUB menu
However, I was able to run readlink on /sbin/init to check the path, and then exec /lib/systemd/systemd did work
9:12 PM
Oh. That must assume you have a compliant init or that you are using busybox's init. How about Ctrl+Alt+Del?
returned me to the GRUB menu ^^^^
init /bin/busybox sh worked at the end of boot parameters too
That sounds reasonable, but is there a filesystem syncing issue?
Without an =?
I tried it at the beginning again and found nothing in $@
Yeah, if it gives you an interactive shell, then it's not likely receiving the other boot options as positional parameters for the shell.
@EliahKagan sorry, that was a typo I made just now! I did not omit the = sign
amazing that it somehow works...
9:16 PM
I think I do remember something about that working. I don't know why though.
that Linux kernel is one smart cookie, that's all I can say
That or GRUB. I'm not sure which. Is the GRUB line passed verbatim to the kernel?
Can you see if you can make changes to the filesystem?
(And if the changes persist.)
shrug idk. But I once had to compile GRUB, and it was a shedload of code
sure, should be fine...
@EliahKagan read only...
Okay, no problem. Can you remount it readwrite?
What's the command again?
I'll look on the site XD
9:26 PM
For remounting it?
What partition are you remounting?
I found it, no worries...
I exec'd systemd again and here I am
I wrote echo foo > /usr/local/bar
the file is still there
Exec'd systemd?
to boot properly out of the busybox shell
@Zanna Ah, right. Sorry I missed that.
sorry, I'm writing too many messages
shall I try mv'ing libc?
9:32 PM
I think so long as you move it back before your exec systemd then you're good.
You could try.
why do you think reboot doesn't work? that's the kind of thing that makes writing answers difficult "er, this should work, but it didn't work for me and I don't know why..."
I wonder if it's better to explicitly sync the filesystem and go back to GRUB with Ctrl+Alt+Delete though.
@EliahKagan I mean, mv it first, then boot this way and mv it back, like OP
@Zanna reboot with no options tells init to reboot gently. If no appropriate init is running the it won't work. It should work with -f.
ek@Io:~$ busybox reboot --help
BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) multi-call binary.

Usage: reboot [-d DELAY] [-n] [-f]

Reboot the system

	-d SEC	Delay interval
	-n	Do not sync
	-f	Force (don't go through init)
@EliahKagan how can I do that?
command to remount rw in busybox:
mount -o remount, rw /
A: How do I remount a filesystem as read/write?

coderofsalvationfor busybox/android users: you need to add a space (in contrast to normal usage) between 'remount' and 'rw': mount -o remount, rw / otherwise it won't work.

9:36 PM
The sync command syncs cached writes. busybox has it as a builtin.
Ooh, does busybox require the space after , even for the build in Debian and Ubuntu?
idk, I can test next time... it worked that way (with a space)
@EliahKagan what will that do?
going afk for 5-10 minutes
@Zanna Most advice on Ask Ubuntu has it without the space, which works with the regular mount command provided by the mount package. That's why I'm wondering.
@Zanna Normally you sync or unmount a disk to ensure any writes that have not actually gone to the disk go to the disk. (This actually only deals with caching performed by the kernel, not by hardware. It also is not related to buffering by programs, but buffers are flushed when a program terminates normally.)
The S and U in REISUB are "sync" and "umount," respectively. The S achieves the same thing as the sync command; I believe the U remounts filesystems readonly.
sorry, I don't get it. you said "I wonder if it's better to explicitly sync the filesystem and go back to GRUB with Ctrl+Alt+Delete though." but I don't know what you are suggesting that would be better than
but I wouldn't know whether it would be better anyway, so, I don't need to get it, sorry
9:57 PM
@Zanna Continuing into systemd.
ah :)
I can try that too then
So I'm thinking...
mount -o remount,ro /   # or without the space, if allowed
reboot -f
I'm talking about remounting it back readonly for safety before rebooting.
10:02 PM
Similarly, the sync that precedes that would be after all the writes to the filesystem have been made.
shut up Z! see I am overexcited
but I really need to go to sleep
I can test the final progression of commands if you're too tired, otherwise unable, or just prefer not to do so. That should not be a problem. I have a machine available for this now. It is slow and hard to use, but for what is left, that should not cause any hassle.
I can do it, and my machine is fast
Okay cool.
That's actually quite convenient, because the host with the VM I was just talking about just completely locked up and had to be hard reset.
do you know what you want me to test?
10:06 PM
Can you rename libc.so.6 on this test system?
(As the OP did.)
If this is the system you're typing on now... then I feel like maybe the answer should be no.
If not that's still okay.
it's ok
but my libc is in a different place
    $ ldd /bin/bash
	linux-vdso.so.1 =>  (0x00007ffef5af9000)
	libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fa7e8377000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa7e8173000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa7e7d93000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa7e85a0000)
anyway, there it is for reference. in case I forget
what shall I do now?
Oh, that's no problem. It can be in a different place.
well, now it's in the wrong place
10:13 PM
Due to you renaming it? :)
So, if you're willing to, you could break it like the OP did. Or perhaps you've done that.
Ah OK.
if I try to undo my terrible action, I get that error
The error the OP got?
sudo: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory
10:14 PM
For ease of reading I'm posting the steps as separate messages.
Then reboot.
Get the GRUB menu.
Press e.
Add init=/bin/busybox sh to the end of the line.
Press Enter to boot with it.
In the busybox shell, remount the root filesystem readwrite with:
mount -o remount,rw /
If that gives an error then do it with the space as you had done before.
Rename the file back, using a command such as:
mv /lib/x86_64-linux-gnu/libc.so.6.bak /lib/x86_64-linux-gnu/libc.so.6
reboot command requires libc. System is hung on Reached target Shutdown
Where is that happening?
Unmounting /run/user/1000 failed
Is that in busybox or just when you try to reboot / shut down?
Where are these errors occurring?
@EliahKagan I used the GUI dialog to "restart"
But system won't shutdown fully
10:18 PM
Oh, yeah. I don't expect that to work.
Ah well
To reboot the broken system, use Alt+SysRq+REISUB.
That does the syncing and unmounting and stuff that I'm planning to suggest they use commands for in busybox later. I should really add mention of REISUB into the post!
...Continuing on with the instructions for what to do in busybox (but please do mention if there are further problems getting into it):
I should have tried it. I just held down the power button :(
That's all right. I don't think Alt+SysRq+REISUB requires testing.
I mean, it would perhaps have been better for your system had you used it...
So, in the busybox shell, after renaming the file, sync, remount readonly, and reboot:
mount -o remount,ro /
reboot -f
Use a space after the , in the mount command if one was needed for the first one.
The -f flag is needed because otherwise reboot requires a working init (and it has to be running -- so even if libc6 is fixed, there's still nothing with the expected functionality of the init daemon running).
@EliahKagan I don't think it was going to work at that stage? GUI long gone...
In busybox now
@EliahKagan you press F10 to boot, or Ctrl+x
I took a bad photo of the edit screen haha
10:24 PM
@Zanna Maybe, but Alt+SysRq+REISUB has nothing to do with the GUI.
@Zanna Oh, sorry. Thanks!
@EliahKagan I should test it. My power button is so abused it's actually cracked
I should mention I need to wait for a while for the shell to produce something like
So are you able to mount the filesystem readwrite? (I assume you can, since I think that's one of the things you did before.)
@Zanna Like this?
BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

~ $
[ 173.921448] random: crng init done
So... do you have a shell?
This pops up, and might come in the middle of a command and break it
10:27 PM
I'm not too worried about that.
Yes, everything is OK, I'm just mentioning anything that seems worthy of mention
Thanks, I appreciate it!!
The space in the mount command was not needed
Excellent. I will not show it then.
Are you able to rename libc.so.6?
Btw don't worry I'll make sure to move this conversation to the island.
complete success
10:35 PM
Awesome. I'll let you know when I've edited the post so you can look it over, but you need not do so immediately (nor at all, if you don't want to). In particular, you need not stay awake until then. I plan to do it soon but I can't be sure when I'll get it done; new difficulties always arise! :)
have renamed the file, sync'd, remounted ro, rebooted with -f (back to GRUB menu), booted into my system, and nothing is broken
just for clarity, at the GRUB menu, you choose Advanced options for Ubuntu, move the cursor to the kernel you want, but instead of pressing enter, press e, then you get the edit screen
I'll want to link to something on that with a beginner-friendly explanation and/or pictures. Do you know of something?
I took a crappy photo of the edit screen for you
the problem is... no screenshots!
I hope you're not offended if I say that I am hoping not to include any screenshots because it would make that section of the answer too long.
so there are a couple of decent pictures of the GRUB menu around, but they are old... I can never find a good set of photos to illustrate this procedure
not at all
10:39 PM
Anyway, given all the work you have just done... I think it would probably have made sense if you'd posted a new answer. I've done a significant amount of the edit to mine, though, and I know you had preferred not to post a separate one.
I only took the photo just in case it might be useful for your reference
I only did a little monkey work :D
Besides doing the testing, you contributed considerably to the actual procedure. There was stuff I got wrong, stuff I didn't know/remember how to do, and so forth.
Well, I'll never get the edit done if I find all the ideal resources first. I can always add links. What do you think of this (not the whole edit, of course, just the wording about the GRUB menu):
> Hold down Shift so the GRUB boot menu appears. Select Advanced options for Ubuntu and press Enter. Select any kernel and press e to edit its boot options temporarily. Add init=/bin/busybox sh to the end and press F10 to boot it.
We had edit suggestions that claimed Shift never works anymore for anybody, remember? They didn't cite the claim and they made additional changes so they were easy to reject. But is that true? I never do this these days...
you can say if shift doesn't work try esc, but shift doesn't work for me and esc brings the boot menu instead XD
I read it has to be left shift
you might want to mention not to press enter on any of the kernels
because if you press enter it just boots
it's sort of intuitive that you select something by pressing enter I think
just now while I was testing a lot, I accidentally booted the system twice by pressing enter absentmindedly when I wanted to "select" a kernel to press e on!
10:49 PM
So Esc does work for you?
no, I get a minimal UEFI boot menu
I can select Windows boot manager, ubuntu, ubuntu, and if there's a bootable USB, I can select it. I need to press ESC to boot from USB
(only the Windows Boot Manager option boots the installed system. And it's not dual boot. It's just a terrible mess)
(which I'm going to claim is not my fault. 32-bit only UEFI is... not very well supported)
Oh. Right. I was not thinking about your strange machine. :)
yeah... it's not the best for testing!
10:54 PM
Wait, does your system have the 32-bit version of Ubuntu on it?
32-bit UEFI, 32-bit GRUB, 64-bit everything else
Oh, right.
the 32-bit version would never work at all whatsoever
I mean, I actually don't understand the distinction between 64-bit UEFI and 32-bit UEFI. But I remember that distinction exists and that you've mentioned it in connection with your machine.
Anyway, your libc.so.6 clearly is 64-bit.
yeah I don't understand either. Maybe everyone has described this problem misleadingly. In any case it was a fun intro to Linux having to go from 100% clueless to figuring out how to boot the installed system from the GRUB command line and compile GRUB to get anything to work apart from the live system. And when I say fun, I really mean it
17:00 - 23:0023:00 - 00:00

« first day (245 days earlier)      last day (1066 days later) »