Hi, I'm trying to install fedora40 server on a server with 3x4TB disks and I don't understand how the RAID options work. I'm familiar with RAID and how it works, but it's been quite some time since I've used the fedora installer, and it appears to have changed.
How do I create a RAID5 array from the three disks? It appears like it's not possible to create the three partitions on each device separately?
I'd also like to create a 3GB swap partition on each disk, but it also appears to not be possible with this installer?
Am 01.06.2024 um 04:12 schrieb Alex mysqlstudent@gmail.com:
Hi, I'm trying to install fedora40 server on a server with 3x4TB disks and I don't understand how the RAID options work. I'm familiar with RAID and how it works, but it's been quite some time since I've used the fedora installer, and it appears to have changed.
How do I create a RAID5 array from the three disks? It appears like it's not possible to create the three partitions on each device separately?
We have some documentation about RAID installation.
- Fedora Server Installation Guide https://docs.fedoraproject.org/en-US/fedora-server/installation/
- Fedora Server interactive local installation -> Raid configuration https://docs.fedoraproject.org/en-US/fedora-server/installation/interactive-...
- Fedora on Rented Server – The Example of Hetzner Data Center https://docs.stg.fedoraproject.org/en-US/fedora-server/tutorials/hetzner-dc/
Unfortunately due to a technical issue I lost most of the text parts in the latter. But the remaining screenshots may offer enough guidance. Additionally, I can send you my original documentation that I wrote for Hetzner (for F33, but it basically didn’t change much) by e-mail if you like.
Basically You use the „Destination“ step in the GUI and select all 3 disks. Then you have 2 options:
(a) Select custom installation
Here you don’t work with partitioning but with mount points and file systems. Anaconda does the partitioning for you. E.g. you add a root file system on a LVM logical volume and select RAID 3 as basis. Anaconda creates the three partition and the raid, creates the Volume Group and the logical volume.
You can find a detailed description of the work steps in Issue #89 Software Raid on a UEFI system using GPT partitioning scheme https://pagure.io/fedora-server/issue/89 Issue #87 Software Raid on a BiosBoot system using GPT partitioning scheme https://pagure.io/fedora-server/issue/87
This option is quite comfortable. One disadvantage is that Anaconda mixes up the order of the partitions on the disk during installation. Everything works, but this may make it difficult to adjust the partitioning later.
(b) Select Advanced custom (blivet GUI)
Here you do the partitioning, creation of Raid, creation of LVM VG and creation of LVs in a GUI all by yourself. It is quite low level and you have to do every single step.
My Hetzner documentation describes this path. The advantage is that you can define the partitioning exactly, but it is quite detailed and quite tedious.
I'd also like to create a 3GB swap partition on each disk, but it also appears to not be possible with this installer?
With both method you can add a swap partion. In Custom you find swap as a „special mount point“. In Advanced custom you can add a partition or logical volume and „format“ it as swap.
But mind you, in these days and the default memory equipment, a swap partition is no longer necessary nor useful in most cases.
-- Peter Boy https://fedoraproject.org/wiki/User:Pboy PBoy@fedoraproject.org
Timezone: CET (UTC+1) / CEST (UTC+2)
Fedora Server Edition Working Group member Fedora Docs team contributor and board member Java developer and enthusiast
Am 01.06.2024 um 07:20 schrieb Peter Boy pboy@uni-bremen.de: ... Additionally, I can send you my original documentation that I wrote for Hetzner (for F33, but it basically didn’t change much) by e-mail if you like.
Or you may download it from https://pboy.fedorapeople.org/InstGuideF33-Hetzner-en.pdf
-- Peter Boy https://fedoraproject.org/wiki/User:Pboy PBoy@fedoraproject.org
Timezone: CET (UTC+1) / CEST (UTC+2)
Fedora Server Edition Working Group member Fedora Docs team contributor and board member Java developer and enthusiast
We have some documentation about RAID installation.
Fedora Server Installation Guide https://docs.fedoraproject.org/en-US/fedora-server/installation/
Fedora Server interactive local installation -> Raid configuration
https://docs.fedoraproject.org/en-US/fedora-server/installation/interactive-...
Thanks so much for your help. A few general questions:
- Can (should?) I put /boot/efi on RAID5? It lets me choose the option, but then complains and wants it on RAID1. Does it set up RAID1 with a failover?
- I screwed up the first time and had to switch to the console on ALT-2 and use dd to overwrite the MBR because the third disk wasn't recognized. In the past, this would crash the installer, so I'm glad to see that's fixed, but it would be nice if all disks were recognized or the existing disk layout was ignored.
- Both times I tried to install, the systemd-resolved daemon wouldn't resolve hostnames, as the "software selection" menu could not identify a local mirror. I had to remove the /etc/resolv.conf symlink and create it using 8.8.8.8.
Thanks, Alex
On Sat, Jun 1, 2024, at 12:33 PM, Alex wrote:
We have some documentation about RAID installation.
Fedora Server Installation Guide https://docs.fedoraproject.org/en-US/fedora-server/installation/
Fedora Server interactive local installation -> Raid configuration https://docs.fedoraproject.org/en-US/fedora-server/installation/interactive-...
Thanks so much for your help. A few general questions:
- Can (should?) I put /boot/efi on RAID5? It lets me choose the option,
but then complains and wants it on RAID1. Does it set up RAID1 with a failover?
I was thinking about this thread and just started a test. My thinking was that /boot/efi is small and thus raid 1 made the most sense. It let me select 3 disks and the install (under VirtualBox) is going on right now. Note that raid 1 is not limited to just two disks/partitions, it can make multi disk mirror sets. Most people don't really think about that since you don't get the size advantages of raid 5.
On Jun 1, 2024, at 17:17, Doug Herr fedoraproject.org@wombatz.com wrote:
On Sat, Jun 1, 2024, at 12:33 PM, Alex wrote:
We have some documentation about RAID installation.
- Fedora Server Installation Guide
https://docs.fedoraproject.org/en-US/fedora-server/installation/
- Fedora Server interactive local installation -> Raid configuration
https://docs.fedoraproject.org/en-US/fedora-server/installation/interactive-...
Thanks so much for your help. A few general questions:
- Can (should?) I put /boot/efi on RAID5? It lets me choose the option,
but then complains and wants it on RAID1. Does it set up RAID1 with a failover?
I was thinking about this thread and just started a test. My thinking was that /boot/efi is small and thus raid 1 made the most sense. It let me select 3 disks and the install (under VirtualBox) is going on right now. Note that raid 1 is not limited to just two disks/partitions, it can make multi disk mirror sets. Most people don't really think about that since you don't get the size advantages of raid 5.
I’m not sure it matters. I’m not aware of any hardware that would be capable of detecting a failed drive in a software RAID when the EFI firmware launches the bootloader.
Most likely setting up RAID1 and then adding a separate EFI boot entry for each part of the array would get you the closest to redundancy. But it would still only be looking at the fat32 filesystem on each device.
Hi,
- Can (should?) I put /boot/efi on RAID5? It lets me choose the option,
but then complains and wants it on RAID1. Does it set up RAID1 with a failover?
I was thinking about this thread and just started a test. My thinking
was that /boot/efi is small and thus raid 1 made the most sense. It let me select 3 disks and the install (under VirtualBox) is going on right now. Note that raid 1 is not limited to just two disks/partitions, it can make multi disk mirror sets. Most people don't really think about that since you don't get the size advantages of raid 5.
I’m not sure it matters. I’m not aware of any hardware that would be capable of detecting a failed drive in a software RAID when the EFI firmware launches the bootloader.
Most likely setting up RAID1 and then adding a separate EFI boot entry for each part of the array would get you the closest to redundancy. But it would still only be looking at the fat32 filesystem on each device.
Most importantly, will it utilize the other boot records if the first one is unavailable?
I'm old enough to remember when we had to install manually in order to do this.
I ultimately was able to get fedora40 installed. The "Preparing transaction from installation source" message was apparently due to the RAID5 being rebuilt and consuming all available I/O. I waited it out and looked at /proc/mdstat and saw it was rebuilding at 200MB/sec, so it just took a while, thankfully.
There's definitely something wonky going on with the OVH IPMI, java, and even their HTML5 interface. I would hate to be in a position where I need to rebuild/recover this system quickly. Even upload speed was abysmal.
Thanks, Alex
Am 01.06.2024 um 21:33 schrieb Alex mysqlstudent@gmail.com:
...
Thanks so much for your help. A few general questions:
- Can (should?) I put /boot/efi on RAID5? It lets me choose the option, but then complains and wants it on RAID1. Does it set up RAID1 with a failover?
You definitely have to use RAID 1. When the disk is first read in the boot process, there is no mdm available to put the pieces of the raid5 disks together. It is just reading the file system from the first available single disk.
Later in the boot process it gets mounted at /boot/efi and just then the raid software is available.
- I screwed up the first time and had to switch to the console on ALT-2 and use dd to overwrite the MBR because the third disk wasn't recognized. In the past, this would crash the installer, so I'm glad to see that's fixed, but it would be nice if all disks were recognized or the existing disk layout was ignored.
OK, I think that’s worth a bug report against Anaconda! Unfortunately I don’t have a 3 disk test equipment here at the moment.
Would be helpful to know it someone can reproduce the issue or is it specific to a special property of the hardware you are using. I suppose, the disk are used and already had some data on it. Ist there something special? E.g. had they different partitioning schemes?
- Both times I tried to install, the systemd-resolved daemon wouldn't resolve hostnames, as the "software selection" menu could not identify a local mirror. I had to remove the /etc/resolv.conf symlink and create it using 8.8.8.8.
Obviously you didn’t used the distributed installation media to boot the server. How did you create the boot medium?
-- Peter Boy https://fedoraproject.org/wiki/User:Pboy PBoy@fedoraproject.org
Timezone: CET (UTC+1) / CEST (UTC+2)
Fedora Server Edition Working Group member Fedora Docs team contributor and board member Java developer and enthusiast
Hi,
- I screwed up the first time and had to switch to the console on ALT-2
and use dd to overwrite the MBR because the third disk wasn't recognized. In the past, this would crash the installer, so I'm glad to see that's fixed, but it would be nice if all disks were recognized or the existing disk layout was ignored.
OK, I think that’s worth a bug report against Anaconda! Unfortunately I don’t have a 3 disk test equipment here at the moment.
Would be helpful to know it someone can reproduce the issue or is it specific to a special property of the hardware you are using. I suppose, the disk are used and already had some data on it. Ist there something special? E.g. had they different partitioning schemes?
Nope, they were all just type fd (Linux RAID), created by anaconda.
- Both times I tried to install, the systemd-resolved daemon wouldn't
resolve hostnames, as the "software selection" menu could not identify a local mirror. I had to remove the /etc/resolv.conf symlink and create it using 8.8.8.8.
Obviously you didn’t used the distributed installation media to boot the server. How did you create the boot medium?
This was from the fedora40 server netinstall ISO here. https://fedoraproject.org/server/download
Hi,
Just when I thought my fedora40 server installation via IPMI/VNC was going okay, it stalls at the "Preparing transaction from installation source" step. What happens here that might cause this?
When I first started, I could go to console 2 and ping an IP outside the network, so I know routing was set up properly. However, name resolution didn't work, so I removed the /etc/resolv.conf symlink and created a new /etc/resolv.conf with 8.8.8.8. I could now resolve hosts to their IPs from the command line, and the "software selection" section of the installer was able to find a local mirror. It succeeded in downloading all ~645MB of packages and created my filesystems, but now it appears to be stalled.
Maybe it has to build the arrays before it begins? It's still building the arrays according to /proc/mdstat. That hasn't been my experience in the past, but maybe it's changed?
I'm concerned the IPMI session will time out, as it's happened before :-(
I don't see anything else on any of the other consoles that would indicate a problem.
During install, there were repeated messages like:
kernel: EDID has corrupt header
but this thread appears to indicate it was from the wrong firmware or something? https://forum.proxmox.com/threads/pve8-kernel-edid-has-corrupt-header.129409...
Any ideas greatly appreciated. I've spent like 10 hours on this so far :-( Thanks, Alex
On Sat, Jun 1, 2024 at 1:21 AM Peter Boy pboy@uni-bremen.de wrote:
Am 01.06.2024 um 04:12 schrieb Alex mysqlstudent@gmail.com:
Hi, I'm trying to install fedora40 server on a server with 3x4TB disks and I
don't understand how the RAID options work. I'm familiar with RAID and how it works, but it's been quite some time since I've used the fedora installer, and it appears to have changed.
How do I create a RAID5 array from the three disks? It appears like it's
not possible to create the three partitions on each device separately?
We have some documentation about RAID installation.
Fedora Server Installation Guide https://docs.fedoraproject.org/en-US/fedora-server/installation/
Fedora Server interactive local installation -> Raid configuration
https://docs.fedoraproject.org/en-US/fedora-server/installation/interactive-...
- Fedora on Rented Server – The Example of Hetzner Data Center
https://docs.stg.fedoraproject.org/en-US/fedora-server/tutorials/hetzner-dc/
Unfortunately due to a technical issue I lost most of the text parts in the latter. But the remaining screenshots may offer enough guidance. Additionally, I can send you my original documentation that I wrote for Hetzner (for F33, but it basically didn’t change much) by e-mail if you like.
Basically You use the „Destination“ step in the GUI and select all 3 disks. Then you have 2 options:
(a) Select custom installation
Here you don’t work with partitioning but with mount points and file systems. Anaconda does the partitioning for you. E.g. you add a root file system on a LVM logical volume and select RAID 3 as basis. Anaconda creates the three partition and the raid, creates the Volume Group and the logical volume.
You can find a detailed description of the work steps in Issue #89 Software Raid on a UEFI system using GPT partitioning scheme https://pagure.io/fedora-server/issue/89 Issue #87 Software Raid on a BiosBoot system using GPT partitioning scheme https://pagure.io/fedora-server/issue/87
This option is quite comfortable. One disadvantage is that Anaconda mixes up the order of the partitions on the disk during installation. Everything works, but this may make it difficult to adjust the partitioning later.
(b) Select Advanced custom (blivet GUI)
Here you do the partitioning, creation of Raid, creation of LVM VG and creation of LVs in a GUI all by yourself. It is quite low level and you have to do every single step.
My Hetzner documentation describes this path. The advantage is that you can define the partitioning exactly, but it is quite detailed and quite tedious.
I'd also like to create a 3GB swap partition on each disk, but it also
appears to not be possible with this installer?
With both method you can add a swap partion. In Custom you find swap as a „special mount point“. In Advanced custom you can add a partition or logical volume and „format“ it as swap.
But mind you, in these days and the default memory equipment, a swap partition is no longer necessary nor useful in most cases.
-- Peter Boy https://fedoraproject.org/wiki/User:Pboy PBoy@fedoraproject.org
Timezone: CET (UTC+1) / CEST (UTC+2)
Fedora Server Edition Working Group member Fedora Docs team contributor and board member Java developer and enthusiast
-- _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Sat, Jun 1, 2024 at 12:45 PM Alex mysqlstudent@gmail.com wrote:
I don't see anything else on any of the other consoles that would indicate a problem.
Perhaps the installer's log files will be more revealing: https://docs.fedoraproject.org/en-US/quick-docs/anaconda-logging/
During install, there were repeated messages like:
kernel: EDID has corrupt header
EDID is monitor metadata like what resolution it supports, that's just the virtual IPMI VGA adapter being wacky, shouldn't be blocking install.
-
Alex -
Doug Herr -
Jonathan Billings -
Peter Boy -
T.C. Hollingsworth