Setting up a RAID 1 in Funtoo

derobert

Jul 19, 2013 12:36

@Gilles is it just me, or does is unix.stackexchange.com/questions/83457/… already in the 12th inning...

Gilles

@derobert at least the guy has a genuine problem and is following up on requests in comments

but it is the sort of thing where it helps a lot to be in front of the machine

derobert

Yes. Its just not a case where the SE software works so well for. Tempted to try to get him into chat, but I think we're all in much different time zones.

I am most definitely not going to confuse the issue by pointing out that you can get at data by using a loop mount with an offset.

Tarun

Jul 19, 2013 13:33

1

Q: Gnome not starting in FreeBSD

I tried to install gnome2 in FreeBSD using the command pkg_add -v -r gnome2. I also edited my file in /etc/rc.conf and added the lines gdm_enable="YES" and gnome_enable="YES" and also added the lines in /etc/fstab proc /proc procfs rw 0 0 . Still I typed startx in my prompt i...

any help??

Nikos Alexandris

Jul 19, 2013 15:42

@derobert I am just learning. If I am "abusing" the platform, let me please know. I can summarise, shorten things. Thanks, Nikos

@derobert Hi! It certainly would be better to chat rather than to fill up the comments. I am available (almost) anytime requested. Thanks.

derobert

Jul 19, 2013 16:26

@NikosAlexandris No, you're not abusing it, it's just a weakness in the platform...

Nikos Alexandris

Jul 19, 2013 17:22

@derobert Thanks. I consider the Q&A as of the most advanced platform out there. So, re-syncing completed. I can't find a clear description (from what I read and understand of course) of what exactly --update=resync does. I guess it does the right thing, though, I might have misfired again: I did not zeroed the superblocks before re-assembling.

derobert

If it did a full sync, it should be fine

Nikos Alexandris

@derobert I am trying to identify the erroneous number the Gilles spoted, so as to compare with the current status.

derobert

Not sure what's going on with the different block count, Funtoo must have done something weird.

But the array itself seems fine, since mdadm will assemble it after boot.

I'm trying to install a Funtoo VM here to quickly test, but, well, installing Funtoo is not quick.

Even on a fast i7.

its still working on "emerge debian-sources", and has been for a while

Nikos Alexandris

:D No, Funtoo is real Fun (though)!

derobert

I mean, it's been doing that for longer than the entire Debian install on the host machine took, but... :-P

Nikos Alexandris

Jul 19, 2013 17:29

So, it seems ok now: mdadm -E /dev/sde1 reports "Array Size : 1953382208 (1862.89 GiB 2000.26 GB)" and cat /proc/mdstat reads "md0 : active raid1 sde1[3] sdc1[2]" and, next line "1953382208 blocks super 1.2 [2/2] [UU]".

derobert

My only experiences with Gentoo and derivatives before has been helping a coworker fix his Gentoo install each time it broke. We did find a permanent fix for it eventually, we installed Debian.

@NikosAlexandris have you given that update-initramfs a try?

Nikos Alexandris

@derobert :D Oh, no. I am not abandoning Funtoo. Not that easy. I waited 2 days to learn and get a working system. But it is so solid...

I am doing it now...

Ehm.. heck, I might need to emerge it (or the package that includes it) or it is called something else in Gentoo/Funtoo!

derobert

no, if you don't have it, you don't need it

Nikos Alexandris

A-ha.

derobert

do you know if you're using an initial ramfs?

Nikos Alexandris

Jul 19, 2013 17:32

What other double-checks can I perform?

It sure looks healthy from the mdadm perspective.

@derobert I think yes, I've seen a couple of references on the web and in the funtoo.forums. As well, as I can see stuff related to mdadm in the file /usr/share/genkernel/gen_initramfs.sh

But I don't know it (exactly)!

derobert

you can check your bootloader config...

See if it has initrd lines (if you're using grub)

Nikos Alexandris

hmm... I can see there is bootlogd -- useful for after (re-)booting I guess.

derobert

you can check if you have a /var/log/boot

Nikos Alexandris

I am not familiar with the boot-loader in Funtoo. I need to go step by step.

No there are not.

I did already!

derobert

Grub config is probably in /boot/grub/grub.cfg

Nikos Alexandris

Jul 19, 2013 17:37

Hem... ok, there is /etc/boot.conf

But I can't locate any grub.conf except for the sample file

derobert

Did you install by following this guide? funtoo.org/wiki/Funtoo_Linux_Installation

Nikos Alexandris

Yes.

boot-update? Irrelevant, right?

derobert

that just got spit out from my vm... I wonder what that domdadm argument does.

Nikos Alexandris

Me too!

Didn't see any reference to it elsewhere in the Funtoo wiki.

Here <http://forums.gentoo.org/viewtopic-t-775025-start-0.html> in the kernel line

derobert

That answer looks suspicious. "line raid autodetect" isn't recommended (check the mdadm manpage, for example)

Nikos Alexandris

Jul 19, 2013 17:47

Here as well <http://forums.gentoo.org/viewtopic-t-775025-start-0.html>

Oh man, a bit complicated!

"Note: With latest genkernel, you should pass domdadm option to kernel. In case if busybox can't find /etc/mdadm.conf, it will try to autodetect arrays." from <http://en.gentoo-wiki.com/wiki/Software_RAID_Install>

derobert

Worth a try, I guess.

I haven't been able to figure out how you tell it to regenerate the initramfs

my vm is now emerging boot-update

Nikos Alexandris

Ok, I found /boot/grub/grub.cfg

locate didn't look into /boot I guess (?)

So, I might only need to add it there? i.e. as per (yet another Gentoo-Wiki page) <http://en.gentoo-wiki.com/wiki/RAID/Software>:

GRUB_CMDLINE_LINUX_DEFAULT="dolvm domdadm"

Well, I only need the latter.

derobert

you can just hit edit in grub to test it

Nikos Alexandris

ok, I am going for it.

derobert

install finished

Nikos Alexandris

Jul 19, 2013 17:54

WoW! That was a quick installation!

derobert

it didn't boot, though

apparently it built an initramfs w/o virtio...

Nikos Alexandris

Hmm...

I am not yet sure as to where I should add the line

there is no simple 'grub' command, if you mean that.

derobert

when you boot up, you get a grub menu, which lists the kernels you can boot

press e instead of enter

Nikos Alexandris

aaaa, right!

ok

The I am rebooting!

derobert

It goes on the linux line, just put it at the end

Nikos Alexandris

Jul 19, 2013 17:59

O-K. Will be back... I did thank you for your time, didn't I? Well, if there is another way than inexpensive thank yous, let me know what I can do.

It doesn't play nice!

Again, some disk missing.

derobert

did you spot anything in the boot messages?

anything mentioning md or mdadm or raid?

Nikos Alexandris

No, too fast. The only error is that some local filesystem(s) didn't mount

And now, manually... gets to "Assembly, now, doesn't find both disks."

I can access the data of course.

Hmmm. I have in /etc/fstab:

/dev/md0 geo xfs defaults 0 2

The system said, however, before assembling manually that there is no md0

derobert

hmmm, what if you change /dev/md/0 to /dev/md0 in your /etc/mdadm/mdadm.conf ?

Nikos Alexandris

That was my idea -- but I thought it's simply silly.

derobert

(still trying to get my VM to boot. Apparently the option labeled "include all modules" didn't actually mean all modules...)

Nikos Alexandris

Jul 19, 2013 18:12

Sorry that I can't be of any help.

Should I restart the md0 after altering mdadm.conf?

derobert

you can stop and start it, I'm pretty sure it'll still work.

real question is if that makes it work on bootup

Nikos Alexandris

Right, I am just slightly worried because this time...

mdadm --assemble -v --scan

ends up with

mdadm: /dev/md/0 has been started with 1 drive (out of 2).

derobert

did it say why it didn't add the other drive?

Nikos Alexandris

mdadm: /dev/sdc1 is busy - skipping

derobert

did you do mdadm --stop /dev/md0 first?

Nikos Alexandris

Jul 19, 2013 18:17

Yes, I stopped it.

derobert

or whatever raid device showed in /proc/mdstat

Nikos Alexandris

Let me check the history.

derobert

busy means its part of a currently-assembled raid array

Nikos Alexandris

1st thing I did before anything else after boot:

cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1] [raid0] [raid10] [linear] [multipath]
md127 : active raid1 sdc1[2]
1953382208 blocks super 1.2 [2/1] [U_]

derobert

oh, 127? It's confused itself

yeah, changing /dev/md/0 to /dev/md0 should help with that.

Nikos Alexandris

Jul 19, 2013 18:18

Perfect! :-)

I changed it.

derobert

do a mdadm --stop /dev/md127

Nikos Alexandris

a-ha...

mdadm --stop /dev/md127
mdadm: stopped /dev/md127

...

mdadm: /dev/sde1 is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdc1 is identified as a member of /dev/md0, slot 0.
mdadm: added /dev/sdc1 to /dev/md0 as 0 (possibly out of date)
mdadm: added /dev/sde1 to /dev/md0 as 1
mdadm: /dev/md0 has been started with 1 drive (out of 2).

derobert

what does /proc/mdstat show now?

Nikos Alexandris

cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1] [raid0] [raid10] [linear] [multipath]
unused devices: <none>

Ehm, I stopped it.

mdadm --examine --scan -v
ARRAY /dev/md/0 level=raid1 metadata=1.2 num-devices=2 UUID=73bf29ca:89bff887:79a26531:b9733d7a name=Resilience:0

derobert

assemble it again, leave it assembled. We're going to have to add that disk back.

it should assemble with 1 drive out of the 2.

Nikos Alexandris

Jul 19, 2013 18:22

mdadm --assemble -v --scan
mdadm: looking for devices for /dev/md0

...

derobert

yes, now does it show up in /proc/mdstat?

Nikos Alexandris

yes

Personalities : [raid6] [raid5] [raid4] [raid1] [raid0] [raid10] [linear] [multipath]
md0 : active raid1 sde1[3]
1953382208 blocks super 1.2 [2/1] [_U]

derobert

ok, and it shows with only sde1?

Nikos Alexandris

yes

derobert

ok, try mdadm -a /dev/md0 /dev/sdc1

Nikos Alexandris

Jul 19, 2013 18:23

ehmm

it's recovering...

are we sure it's opl

are we sure it's ok?

derobert

When the array got assembled with only one disk a minute ago, that one disk became more "recent" than the other

Nikos Alexandris

Shouldn't the sdc1 be the source?

derobert

So it's rebuilding to fix that.

Nikos Alexandris

That's not very good.

derobert

Well, they contain the same data. It doesn't matter which is the source.

Nikos Alexandris

Jul 19, 2013 18:25

I am not sure that sde1 is the "latest". For testing I did add some files in sdc1, mounted of course ad /md/0

Bad thing?

derobert

but you let it rebuild after that, right?

Nikos Alexandris

Yes, I did I think the complete --update=resync. It took 3 hours+

I am still not sure though... just worried.

derobert

Yeah. That copied all the data from one disk to the other.

You can go ahead and mount it—you don't have to wait for the rebuild

Nikos Alexandris

Can't we reverse this?

I mean the "date"?

derobert

Well, it's already started copying...

Nikos Alexandris

Jul 19, 2013 18:28

As far as I can remember, all is there.

derobert

if you want to pause it, echo 0 > /proc/sys/dev/raid/speed_limit_max

Nikos Alexandris

I do have a lot of stuff there of course...

derobert

if you pause it, you can look around

Nikos Alexandris

paused

looking...

derobert

echo 200000 to that file to resume the sync

(well, any number you want, 200000 is the default, I believe)

Nikos Alexandris

Jul 19, 2013 18:29

ok

All seems fine.

Another sync? Well, recovery...

I need a break...

I hope someone will reply in the funto.forums as well.

I will come back after the recovery... !

derobert

Make sure you unpaused it, and check /proc/mdstat to confirm its going again

Nikos Alexandris

Just another question: does copying data when md is active, apply to both sdc1 and sde1 at the same time?

Yes, it works

I am watch -d cat /proc/mdstat

  [>....................]  recovery =  4.2% (83603200/1953382208) finish=168.1min speed=185341K/sec

derobert

@NikosAlexandris Yes, as long as you do it through /dev/md0

Nikos Alexandris

ok, no need to worry then.

Phew!

derobert

Once /proc/mdstat says the sync is done, the disks will have the same data. Even if you modify it during the sync

Nikos Alexandris

Jul 19, 2013 18:36

Right.

derobert

You can use the array while its syncing.

Nikos Alexandris

Hem... well let's see. Interesting story...

I am going out of desk for a while. Catch some fresh air.

derobert

You are pretty close to that 200k number, echoing a higher number (say, 400000) to the file might speed it up some

I'll try to pop back into chat later.

Nikos Alexandris

thanks.

derobert

Jul 19, 2013 19:00

I give up on this VM :-(

Anyway, BBL.

Nikos Alexandris

Jul 19, 2013 21:43

@derobert Sorry :-/

Nikos Alexandris

Jul 19, 2013 22:47

The recover process completed. Successfully I guess. To re-boot or not... is the question :-)

Nikos Alexandris

Jul 19, 2013 23:42

I see a short set of in structions over at <http://www.funtoo.org/wiki/Rootfs_over_encrypted_lvm_over_raid-1_on_GPT#Encrypting_the_raid-1>

echo "Activating RAID device."
if [ ! -e '/etc/mdadm.conf' ]
then
echo "DEVICE /dev/sda[0-9] /dev/sdb[0-9] /dev/md[0-9]" > /etc/mdadm.conf
mdadm --examine --scan --config=/etc/mdadm.conf >> /etc/mdadm.conf
mdadm --assemble --scan
fi

No offense, but where should this set of instructions be added for enabling the activation of RAID during the boot process?

The Wiki-page is, imho, not clean.

/dev/chat

Participants