Hi folks
I've got an old F9 server which has been running forever without any problems has just failed to reboot.
I have 2 x RAID1 setup
sda and sdb are the first raid device sdc and add are the second device.
Also, to make life even more fun I only have 4 SATA ports so I have to remove one HDD to get the DVD to work, and the m/b doesn't support USB DVD
it looks like sda has failed - it doesn't even appear in the BIOS list.
I am currently cloning sdb before doing anything, but once that's done, I want to replace sda and rebuild the array. From memory, I do not think that the boot partitons could be RAID'ed back then so I'm not sure what state the boot partition is in on sdb.
Does anyone have any instructions on how to do this based on the age of the system? So far, Google only seems to want to give me more recent solutions.
On Wed, 2019-02-27 at 14:43 +0000, Gary Stainburn wrote:
Hi folks
I've got an old F9 server which has been running forever without any problems has just failed to reboot.
I have 2 x RAID1 setup
sda and sdb are the first raid device sdc and add are the second device.
Also, to make life even more fun I only have 4 SATA ports so I have to remove one HDD to get the DVD to work, and the m/b doesn't support USB DVD
it looks like sda has failed - it doesn't even appear in the BIOS list.
I am currently cloning sdb before doing anything, but once that's done, I want to replace sda and rebuild the array. From memory, I do not think that the boot partitons could be RAID'ed back then so I'm not sure what state the boot partition is in on sdb.
Does anyone have any instructions on how to do this based on the age of the system? So far, Google only seems to want to give me more recent solutions.
Given that it's RAID1 (i.e. mirroring) can't you just use dd to copy all of sdb to sda?
poc
On Wednesday 27 February 2019 15:14:33 Patrick O'Callaghan wrote:
Given that it's RAID1 (i.e. mirroring) can't you just use dd to copy all of sdb to sda?
poc
I am already doing that, but using clonezilla instead of DD. I should end up with the same result. However, I did that so I had a safe copy.
It's taken 4 1/2 hours so far, and hasn't finished yet. If I do the same again to have a working copy it'll be tomorrow before I get it working.
The other thing I wondered is whether that would actually work. Doesn't mdadm store unit ID's to prevent drives changing sequence when booting up? Wouldn't it just see the same drive twice?
I was hoping I could just get the system so I could boot from sdb, and then get it to rebuild the RAID while the server is up and available.
On Wed, 2019-02-27 at 15:23 +0000, Gary Stainburn wrote:
On Wednesday 27 February 2019 15:14:33 Patrick O'Callaghan wrote:
Given that it's RAID1 (i.e. mirroring) can't you just use dd to copy all of sdb to sda?
poc
I am already doing that, but using clonezilla instead of DD. I should end up with the same result. However, I did that so I had a safe copy.
It's taken 4 1/2 hours so far, and hasn't finished yet. If I do the same again to have a working copy it'll be tomorrow before I get it working.
The other thing I wondered is whether that would actually work. Doesn't mdadm store unit ID's to prevent drives changing sequence when booting up? Wouldn't it just see the same drive twice?
I was hoping I could just get the system so I could boot from sdb, and then get it to rebuild the RAID while the server is up and available.
Sorry, my only experience is with a NAS with RAID1 which supports hot- plugging. I've had to replace drives twice and it has Just Worked (i.e. plug in a fresh drive and it configures itself). The NAS is running an old copy of Debian and does use mdadm. I know it has an nvram for configuration settings so maybe the unit IDs are stored there, but beyond that I couldn't say.
poc
On Wed, 2019-02-27 at 14:43 +0000, Gary Stainburn wrote:
Hi folks
I've got an old F9 server which has been running forever without any problems has just failed to reboot.
I have 2 x RAID1 setup
sda and sdb are the first raid device sdc and add are the second device.
Also, to make life even more fun I only have 4 SATA ports so I have to remove one HDD to get the DVD to work, and the m/b doesn't support USB DVD
it looks like sda has failed - it doesn't even appear in the BIOS list.
I am currently cloning sdb before doing anything, but once that's done, I want to replace sda and rebuild the array. From memory, I do not think that the boot partitons could be RAID'ed back then so I'm not sure what state the boot partition is in on sdb.
I'm pretty sure you could raid, but you needed grub trickery on the MBR to get the second disk to boot.
Does anyone have any instructions on how to do this based on the age of the system? So far, Google only seems to want to give me more recent solutions.
Err, from memory, so please measure twice, cut once.
If you do get a bootable kernel and grub (or lilo): - write down serial numbers so you know sda/sdb, ... they might swap. - sfdisk -d /dev/sdb | sfdisk /dev/sda # copy partitions - find /dev/md0 or /dev/md127 (some kernels that era start at 0, some start at 127) - mdadm /dev/md0 --fail /dev/sda1 # unnecessary? - mdadm /dev/md0 --remove /dev/sda1 # remove broken - mdadm /dev/md0 --add /dev/sda1 # add new - watch "cat /proc/mdstat" to see when it's finished.
I'm sure there's a better rebuild command than remove/add; but you did ask about old installs.
On Wednesday 27 February 2019 14:43:30 Gary Stainburn wrote:
Hi folks
I've got an old F9 server which has been running forever without any problems has just failed to reboot.
I have 2 x RAID1 setup
sda and sdb are the first raid device sdc and add are the second device.
Also, to make life even more fun I only have 4 SATA ports so I have to remove one HDD to get the DVD to work, and the m/b doesn't support USB DVD
it looks like sda has failed - it doesn't even appear in the BIOS list.
I am currently cloning sdb before doing anything, but once that's done, I want to replace sda and rebuild the array. From memory, I do not think that the boot partitons could be RAID'ed back then so I'm not sure what state the boot partition is in on sdb.
Does anyone have any instructions on how to do this based on the age of the system? So far, Google only seems to want to give me more recent solutions.
I've now cloned the good drive, and the copy has been put safe. I'm now trying to get it bootable with just this one drive. I've booted using a Fedora 16 DVD in rescue mode, and using the instructions on
https://www.thegeekdiary.com/centos-rhel-how-to-assemble-a-software-raid-in-...
I've managed to get the RAID devices running (on 1 our of 2 drives) and have access to all four devices.
/dev/md/boot /dev/md/root /dev/md/swap /dev/md/var_bacula
I have mounted /dev/md/root on /mnt/sysimage and then run
chroot /mnt/sysimage
to go to the true root. However, when I then try to mount /boot it says no such device /dev/md/boot as this is on the pre-chrooted tree.
How can I get access to /boot so that I can then run grub2-install?
On Wednesday 27 February 2019 19:15:22 Gary Stainburn wrote:
I've now cloned the good drive, and the copy has been put safe. I'm now trying to get it bootable with just this one drive. I've booted using a Fedora 16 DVD in rescue mode, and using the instructions on
https://www.thegeekdiary.com/centos-rhel-how-to-assemble-a-software-raid-in -rescue-mode/
I've managed to get the RAID devices running (on 1 our of 2 drives) and have access to all four devices.
/dev/md/boot /dev/md/root /dev/md/swap /dev/md/var_bacula
I have mounted /dev/md/root on /mnt/sysimage and then run
chroot /mnt/sysimage
to go to the true root. However, when I then try to mount /boot it says no such device /dev/md/boot as this is on the pre-chrooted tree.
How can I get access to /boot so that I can then run grub2-install?
This is a Fedora 19 box, not a Fedora 9 box, sorry everyone
On 2/27/19 8:39 PM, Gary Stainburn wrote:
On Wednesday 27 February 2019 19:33:03 Gary Stainburn wrote:
This is a Fedora 19 box, not a Fedora 9 box, sorry everyone
I have had a look at /boot and it is completely empty. Am I right in thinking I'm stuffed?
Why is it empty? Wasn't that also a RAID? Was you boot partition /dev/md/boot or not?
You have your root partition running, so you can check how /boot was mounted in /etc/fstab: were you really using the md or just pointing to sdaX?
Regards.
On Wednesday 27 February 2019 21:11:12 Roberto Ragusa wrote:
On 2/27/19 8:39 PM, Gary Stainburn wrote:
On Wednesday 27 February 2019 19:33:03 Gary Stainburn wrote:
This is a Fedora 19 box, not a Fedora 9 box, sorry everyone
I have had a look at /boot and it is completely empty. Am I right in thinking I'm stuffed?
Why is it empty? Wasn't that also a RAID?
Yes it was part of the RAID, yes it was activated when I ran mdadm. It was listed as /dev/md/boot and when I mounted it it did so without errors.
However, when I looked inside, there was nothing listed.
Was you boot partition /dev/md/boot or not?
Yes
You have your root partition running, so you can check how /boot was mounted in /etc/fstab: were you really using the md or just pointing to sdaX?
I do have /dev/md/root mounted on /mnt/sysimage and I have looked in there.
/etc/mdadm.conf matches exactly what was generated in the rescue environment when I ran
mdadm --examine --scan >/etc/mdadm.conf
On 2/27/19 10:27 PM, Gary Stainburn wrote:
You have your root partition running, so you can check how /boot was mounted in /etc/fstab: were you really using the md or just pointing to sdaX?
I do have /dev/md/root mounted on /mnt/sysimage and I have looked in there.
/etc/mdadm.conf matches exactly what was generated in the rescue environment when I ran
mdadm --examine --scan >/etc/mdadm.conf
My question was about /etc/fstab. If you actually have had /dev/md/boot as /boot in /etc/fstab, how is it possible that the /dev/md/boot partition is now empty just because the RAID has switched to degraded mode? My suspicion is that you were not using the RAID for /boot.
Anyway, even if your /boot content is indeed lost, you can still manage to repair the system.
-
Berend De Schouwer -
Gary Stainburn -
Patrick O'Callaghan -
Roberto Ragusa