I upgraded a 32-bit Fedora 27 system to Fedora 28 using the normal dnf upgrade procedure. The upgrade finished but the system is still running a Fedora 27 kernel.
There don't appear to be any kernel-PAE* packages in the repos and I don't see this mentioned anywhere in the docs I've found. Are they no longer supported? A quick attempt at using the normal kernel didn't work - the system went to a black screen with just a cursor and hung. I did a control-alt-del and it hung with a message from systemd about starting something to do with the root device. I ran out of time testing - I'll look at it later.
Is there anythng special about going back to the regular kernel?
Thanks,
Jim
On Wed, 2 May 2018 06:26:23 -0400 Jim Simmons simmonsjw-fedora@sws1.ornl.gov wrote:
I upgraded a 32-bit Fedora 27 system to Fedora 28 using the normal dnf upgrade procedure. The upgrade finished but the system is still running a Fedora 27 kernel.
There don't appear to be any kernel-PAE* packages in the repos and I don't see this mentioned anywhere in the docs I've found. Are they no longer supported? A quick attempt at using the normal kernel didn't work - the system went to a black screen with just a cursor and hung. I did a control-alt-del and it hung with a message from systemd about starting something to do with the root device. I ran out of time testing - I'll look at it later.
Is there anythng special about going back to the regular kernel?
I don't think PAE kernels are supported anymore. So, unless you download the src.rpm and build it yourself, you will not get one. There is no maintenance for 32 bit anymore, so probably not a good idea. https://koji.fedoraproject.org/koji/packageinfo?packageID=8 Or, you could continue to run F27 PAE kernels on F28 until F27 goes end of life, or a dependency problem arises.
https://www.phoronix.com/scan.php?page=news_item&px=Fedora-27-Might-Drop...
The suggestion there seems to be to buy new hardware, or move to a distribution that continues to support PAE.
On 05/02/2018 03:26 AM, Jim Simmons wrote:
There don't appear to be any kernel-PAE* packages in the repos and I don't see this mentioned anywhere in the docs I've found. Are they no longer supported? A quick attempt at using the normal kernel didn't work - the system went to a black screen with just a cursor and hung. I did a control-alt-del and it hung with a message from systemd about starting something to do with the root device. I ran out of time testing - I'll look at it later.
Is there anythng special about going back to the regular kernel?
There shouldn't be anything other than possibly less RAM available. Are you using hardware that isn't 64-bit capable? If it is, you can run a 64-bit kernel with a 32-bit userspace.
On Wed, May 02, 2018 at 02:15:35PM -0700, Samuel Sieb wrote:
On 05/02/2018 03:26 AM, Jim Simmons wrote:
...
Is there anythng special about going back to the regular kernel?
There shouldn't be anything other than possibly less RAM available. Are you using hardware that isn't 64-bit capable? If it is, you can run a 64-bit kernel with a 32-bit userspace.
I can install the regular kernel but it won't boot for some reason. I have two mirrored (softraid) disks and the Fedora 28 boot seems to be seeing the disks separately and believing it sees two different copys of the save volume group and logical volumes on the disk - the boot eventually times out with some messages about dracut. I've tried using dracut to rebuild initramfs but it doesn't help.
This isn't critical and this system will be shutdown eventually anyway, I just haven't been able to figure out what is going on yet. I'll do some more experimenting and try to capture some logs - I was just surprised that kernel-PAE was gone and I couldn't find any info on why. Usually Fedora's releases "just work" for me.
Thanks,
Jim
On 05/02/2018 05:57 PM, Jim Simmons wrote:
I can install the regular kernel but it won't boot for some reason. I have two mirrored (softraid) disks and the Fedora 28 boot seems to be seeing the disks separately and believing it sees two different copys of the save volume group and logical volumes on the disk - the boot eventually times out with some messages about dracut. I've tried using dracut to rebuild initramfs but it doesn't help.
And that happens between the two different types of kernel? Or is it something with F28 regardless of which kernel?
This isn't critical and this system will be shutdown eventually anyway, I just haven't been able to figure out what is going on yet. I'll do some more experimenting and try to capture some logs - I was just surprised that kernel-PAE was gone and I couldn't find any info on why. Usually Fedora's releases "just work" for me.
As of F28, the PAE kernels have been discontinued.
On Wed, May 02, 2018 at 10:47:02PM -0700, Samuel Sieb wrote:
On 05/02/2018 05:57 PM, Jim Simmons wrote:
I can install the regular kernel but it won't boot for some reason. I have two mirrored (softraid) disks and the Fedora 28 boot seems to be seeing the disks separately and believing it sees two different copys of the save volume group and logical volumes on the disk - the boot eventually times out with some messages about dracut. I've tried using dracut to rebuild initramfs but it doesn't help.
And that happens between the two different types of kernel? Or is it something with F28 regardless of which kernel?
...
As of F28, the PAE kernels have been discontinued.
F28 only has the normal kernel - it doesn't seem to recognize /dev/vg0/root correctly on a softraid mirrored volume for some strange reason. The F27 PAE kernel does. I'm going to try the F27 "normal" kernel and see if that works. If so, I'll update to get the F28 one and see if it makes a difference. It may also be something strange related to dracut and/or grubby on F28 with this machine - it is ancient (orignally Red Hat 5.2, I think) but has been working for at least 10 years. I may just try to reinstall with F28 or F27, too (I'm using xfce, I think that is possible still). The system has a 2.2G Pentium 4 - it isn't 64-bit but does doe PAE.
I really ought to just get rid of it, which is probably the best thing to do but it has still been working and I've kept it for sentimental reasons, more than anything else.
Jim
On 05/03/2018 02:43 AM, Jim Simmons wrote:
F28 only has the normal kernel - it doesn't seem to recognize /dev/vg0/root correctly on a softraid mirrored volume for some strange reason. The F27 PAE kernel does. I'm going to try the F27 "normal" kernel and see if that works. If so, I'll update to get the F28 one and see if it makes a difference. It may also be something strange related to dracut and/or grubby on F28 with this machine - it is ancient (orignally Red Hat 5.2, I think) but has been working for at least 10 years. I may just try to reinstall with F28 or F27, too (I'm using xfce, I think that is possible still). The system has a 2.2G Pentium 4 - it isn't 64-bit but does doe PAE.
Ok, so using the F27 installed kernel works on F28. It's more likely to be a dracut problem than a kernel problem. Try removing the rescue initramfs and reinstall the F28 kernel.
Are you sure that P4 isn't 64-bit? I have a bunch of them that are, although they are 3.4GHz ones. Also, I don't think I can even put 4GB of RAM in them, so in my case I didn't need the PAE kernel anyway. If I put 4 1GB DIMMs, the BIOS only shows just over 3GB of usable RAM. A while back, I reinstalled them all as 64-bit.
On 05/03/2018 01:07 PM, Joe Zeff wrote:
On 05/03/2018 12:46 PM, Samuel Sieb wrote:
Ok, so using the F27 installed kernel works on F28.
What else would you expect? The kernel doesn't care what version of Fedora you're running. If it did, the upgrade would have removed the F27 kernels.
That was just a lead in to the issue being with dracut. I reconsidered how to write that part a few times and I think in the end I lost part of what I was thinking and wanted to say. :-)
On Thu, May 03, 2018 at 01:36:43PM -0700, Samuel Sieb wrote:
On 05/03/2018 01:07 PM, Joe Zeff wrote:
On 05/03/2018 12:46 PM, Samuel Sieb wrote:
Ok, so using the F27 installed kernel works on F28.
What else would you expect? The kernel doesn't care what version of Fedora you're running. If it did, the upgrade would have removed the F27 kernels.
That was just a lead in to the issue being with dracut. I reconsidered how to write that part a few times and I think in the end I lost part of what I was thinking and wanted to say. :-)
I'm pretty sure it is dracut. Installing the normal kernel from Fedora 27 and it won't boot either. I saw dracut complaining about not finding busybox and biosdevname so I installed them and reinstalled the kernel - same results. I also get a couple of grep errors referring to /usr/lib/plymouth/plymouth-populate-initrd but I think they're harmless and dnf whatprovides can't find that file.
I'm going to try booting a live cd (xfce f28 i386) and if that works I'll just reinstall. Otherwise I'll leave it like it is until I can get the machine cleaned out and shutdown for good.
If I can figure out what is different in the initrd, I'll reply with the info.
Thanks for everything,
Jim
On 05/03/2018 04:36 PM, Jim Simmons wrote:
I'm pretty sure it is dracut. Installing the normal kernel from Fedora 27 and it won't boot either. I saw dracut complaining about not finding busybox and biosdevname so I installed them and reinstalled the kernel - same results. I also get a couple of grep errors referring to /usr/lib/plymouth/plymouth-populate-initrd but I think they're harmless and dnf whatprovides can't find that file.
Try installing "dracut-config-generic" and then reinstall the F28 kernel. Make sure that the initramfs timestamp is updated. The file should also be larger after. If it still doesn't work, then removing "quiet" and "rhgb" from the kernel command line might provide some more info.
I'm going to try booting a live cd (xfce f28 i386) and if that works I'll just reinstall. Otherwise I'll leave it like it is until I can get the machine cleaned out and shutdown for good.
That's a good test as well.
On Thu, May 03, 2018 at 04:45:03PM -0700, Samuel Sieb wrote: ...
Try installing "dracut-config-generic" and then reinstall the F28 kernel. Make sure that the initramfs timestamp is updated. The file should also be larger after. If it still doesn't work, then removing "quiet" and "rhgb" from the kernel command line might provide some more info.
The image was more than twice as big, but it did the same thing on boot
With quiet and rhgb removed, it got to the point where it said:
Reached target System Initialization. Reached target Basic System.
Then it stopped saying anything.
At 150 seconds after boot, it said:
dracut-initqueue[434]: Warning: dracut-initqueue timeout - start timeout scripts
It sat there until 287 seconds after boot and started giving the same message once or twice a second until at 357 seconds it said:
dracut-initqueue[434]: Warning: Could not boot.
There are a couple of audit messages, then:
Warning: /dev/mapper/vg0-root does not exist Warning: /dev/vg0/root does not exist Warning: /dev/vg0/swap does not exist
Generationg "/run/initramfs/rdsosreport.txt"
Then it enters emergency mode.
In journalctl, I see this and I think it is the problem:
WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 on /dev/sdb8 was already found on sda8. WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 on /dev/md127 was already found on sda8. WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 prefers device /dev/sda8 because device was seen first. WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 prefers device /dev/md127 because device size is correct.
Those repeat severeal times, then it says:
Inactive '/dev/vg0/root' [13.20 GiB] inherit Inactive '/dev/vg0/swap' [2.00 GiB] inherit Inactive '/dev/vg0/home' [15.00 GiB] inherit Inactive '/dev/vg0/local' [1.00 GiB] inherit Inactive '/dev/vg0/big2' [29.00 GiB] inherit PARTIAL MODE. Incomplete logical volumes will be processed.
I'm going to try booting a live cd (xfce f28 i386) and if that works I'll just reinstall. Otherwise I'll leave it like it is until I can get the machine cleaned out and shutdown for good.
That's a good test as well.
I'll try the live cd later.
Thanks,
Jim
On 05/03/2018 06:35 PM, Jim Simmons wrote:
In journalctl, I see this and I think it is the problem:
WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 on /dev/sdb8 was already found on sda8. WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 on /dev/md127 was already found on sda8. WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 prefers device /dev/sda8 because device was seen first. WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 prefers device /dev/md127 because device size is correct.
Yes, this is the problem. Someone was having a very similar problem at work recently with multipath. The problem is that LVM is checking for volumes before raid has finished checking. The raid device is getting created, but it's too late, LVM has already found the PV on the raw disk. I think if the raid version was different, LVM wouldn't be able to see the PV, but it would be difficult to change that now. If you only have LVM volumes on the raid, then add the following line to your /etc/lvm/lvm.conf in the devices section: filter = [ "r/sd.*/" ]
Then you will need to regenerate the initramfs by running: dracut -f --kver kernel-version "kernel-version" is the full name of the kernel. e.g. The latest kernel package I have installed is "kernel-4.15.7-300.fc27.x86_64", so I would run: dracut -f --kver 4.15.7-300.fc27.x86_64 You can remove the dracut-config-generic package to go back to smaller files if you want.
On Tue, May 08, 2018 at 12:04:58PM -0700, Samuel Sieb wrote:
On 05/03/2018 06:35 PM, Jim Simmons wrote:
In journalctl, I see this and I think it is the problem:
WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 on /dev/sdb8 was already found on sda8. WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 on /dev/md127 was already found on sda8. WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 prefers device /dev/sda8 because device was seen first. WARNING: PV W0e1Cx-ByMA-iRX6-OGxr-KfWC-zaUP-YXWgW5 prefers device /dev/md127 because device size is correct.
Yes, this is the problem. Someone was having a very similar problem at work recently with multipath. The problem is that LVM is checking for volumes before raid has finished checking. The raid device is getting created, but it's too late, LVM has already found the PV on the raw disk. I think if the raid version was different, LVM wouldn't be able to see the PV, but it would be difficult to change that now. If you only have LVM volumes on the raid, then add the following line to your /etc/lvm/lvm.conf in the devices section: filter = [ "r/sd.*/" ]
Then you will need to regenerate the initramfs by running: dracut -f --kver kernel-version "kernel-version" is the full name of the kernel. e.g. The latest kernel package I have installed is "kernel-4.15.7-300.fc27.x86_64", so I would run: dracut -f --kver 4.15.7-300.fc27.x86_64 You can remove the dracut-config-generic package to go back to smaller files if you want.
I tried multiple settings for filter - just enable /dev/md.* and disable everything else and others. Nothing helped. I also experimented with various lvm commands from the emergency shell - passing the filter and/or global_filter options on the command line - nothing helped.
I could boot from the Fedora 28 i386 xfce live image, though, and it saw raid volumes and volume group without problems.
In checking things, this system was originally installed with Red Hat 7.3 in 2012 and the md device was created then. It had 0.90 version metadata. I forced it to 1.0 using the live image but that didn't work, either.
I used the F28 live image to install to disk again, telling it to reformat /, /boot, etc. It installed ok but still wouldn't boot - same exact symptoms.
Finally I backed /home and the other filesystems in the volume group, went into the live cd and manually deleted the volume group and both md mirrors, then recreated them by hand, rebooted the live image and did another install to disk.
This time it worked and it has been happy since. I still don't know what the issue was but hopefully this info may be useful for someone else if they happen to have a 16 year old system they're trying to keep running :-)
I'm guessing it was related to the age of the mirror - I have several other systems (64-bit, though) using a similar configuration and they upgraded and run fine.
Thanks,
Jim
Hi,
On 04-05-18 01:36, Jim Simmons wrote:
On Thu, May 03, 2018 at 01:36:43PM -0700, Samuel Sieb wrote:
On 05/03/2018 01:07 PM, Joe Zeff wrote:
On 05/03/2018 12:46 PM, Samuel Sieb wrote:
Ok, so using the F27 installed kernel works on F28.
What else would you expect? The kernel doesn't care what version of Fedora you're running. If it did, the upgrade would have removed the F27 kernels.
That was just a lead in to the issue being with dracut. I reconsidered how to write that part a few times and I think in the end I lost part of what I was thinking and wanted to say. :-)
I'm pretty sure it is dracut. Installing the normal kernel from Fedora 27 and it won't boot either. I saw dracut complaining about not finding busybox and biosdevname so I installed them and reinstalled the kernel - same results. I also get a couple of grep errors referring to /usr/lib/plymouth/plymouth-populate-initrd but I think they're harmless and dnf whatprovides can't find that file.
I'm going to try booting a live cd (xfce f28 i386) and if that works I'll just reinstall. Otherwise I'll leave it like it is until I can get the machine cleaned out and shutdown for good.
If I can figure out what is different in the initrd, I'll reply with the info.
If you're using BIOS RAID on a non Intel controller (so you've a RAID set which the BIOS can see so it can boot from it) then it might very well be a problem with dmraid, which is the tool which recognizes the RAID headers.
Try running dmraid -r that will show a list of recognized RAID sets, if that does not see any sets try downgrading dmraid to the version from F27 and see if it then does recognize your raidset.
Regards,
Hans
-
Hans de Goede -
Jim Simmons -
Joe Zeff -
Samuel Sieb -
stan