A huge thanks to Thomas Schmitt for posting xorrisofs arguments :)
Here is a lorax PR switching to grub2 for BIOS and changing the layout of the iso as described in his post:
https://github.com/weldr/lorax/pull/1226
And a Fedora 36 iso:
https://bcl.fedorapeople.org/boot-grub2-f36.iso
I've tested this with:
- qemu bios -cdrom - qemu uefi -cdrom - qemu bios -hda - qemu uefi -hda - USB stick on uefi PC hardware with SB off - USB stick on UEFI PC hardware with SB on - USB stick on Apple hardware UEFI 2010 Macbook Air and 2012 Macbook Pro - Media test works on all of the above
I have not tested it on CD or DVD physical media. I have a stack of blank discs but apparently have unplugged all my drives to use their SATA ports for SSDs :)
Brian
On Wed, Apr 13, 2022 at 7:52 PM Brian C. Lane bcl@redhat.com wrote:
A huge thanks to Thomas Schmitt for posting xorrisofs arguments :)
Here is a lorax PR switching to grub2 for BIOS and changing the layout of the iso as described in his post:
https://github.com/weldr/lorax/pull/1226
And a Fedora 36 iso:
https://bcl.fedorapeople.org/boot-grub2-f36.iso
I've tested this with:
- qemu bios -cdrom
- qemu uefi -cdrom
- qemu bios -hda
- qemu uefi -hda
- USB stick on uefi PC hardware with SB off
- USB stick on UEFI PC hardware with SB on
- USB stick on Apple hardware UEFI 2010 Macbook Air and 2012 Macbook Pro
- Media test works on all of the above
I have not tested it on CD or DVD physical media. I have a stack of blank discs but apparently have unplugged all my drives to use their SATA ports for SSDs :)
This looks great! I've given some initial feedback and I'll do some testing. :)
Thanks for doing this!
Hi,
Brian C. Lane wrote:
https://bcl.fedorapeople.org/boot-grub2-f36.iso [...] I have not tested it on CD or DVD physical media.
I can confirm that it boots to a GRUB 2.06 menu via real iron EFI from real plastic DVD+RW. The menu offers me to install Fedora 36. So the boot lures for EFI from optical media continue to work after the layout change.
The medium type should not matter. Every readable optical medium is supposed to work the same at boot time.
Regrettably my old test machine with only legacy BIOS is currently lacking components, which serve at other places. So i cannot yet test that boot path from DVD. The report of Brian C. Lane that it works with "qemu bios -cdrom" gives hope that it will work with real iron and plastic, too. Other than OVMF SeaBIOS is not known of not distinguishing CD-ROM and HDD.
Have a nice day :)
Thomas
Hi Brian,
On 4/14/22 01:52, Brian C. Lane wrote:
A huge thanks to Thomas Schmitt for posting xorrisofs arguments :)
Here is a lorax PR switching to grub2 for BIOS and changing the layout of the iso as described in his post:
https://github.com/weldr/lorax/pull/1226
And a Fedora 36 iso:
https://bcl.fedorapeople.org/boot-grub2-f36.iso
I've tested this with:
- qemu bios -cdrom
- qemu uefi -cdrom
- qemu bios -hda
- qemu uefi -hda
- USB stick on uefi PC hardware with SB off
- USB stick on UEFI PC hardware with SB on
- USB stick on Apple hardware UEFI 2010 Macbook Air and 2012 Macbook Pro
- Media test works on all of the above
I have not tested it on CD or DVD physical media. I have a stack of blank discs but apparently have unplugged all my drives to use their SATA ports for SSDs :)
Thank you so much for working on this.
I've dd-ed the boot-grub2-f36.iso to an USB stick and successfully tested it on the following machines:
1. Siemens PC with an i5-2400 with 8G RAM 2. Fujitsu-Siemens PC with a Core 2 Duo E6600 2.4 Ghz dual core with 4G RAM 3. Dell Latitude E6400 (core 2 duo) with 4G RAM 4. Acer One AO532 netbook with a N450 + 2G RAM
The image booted fine on all 4 machines, so from a machine compatibility pov this looks good to me.
There were 2 minor issues with machine 2:
1. After the "Welcome to GRUB" text the 3.5" floppy drive was searched repeatedly before showing the grub bootmenu. It also took like 5-10 seconds to show the menu. I guess that grub is using a minimal grub.cfg which is looking for the actual grub.cfg based on the partition uuid or some such and it is also checking the floppy drive. I wonder if we can tweak grub.cfg on the bootmedia to not do this?
2. Booting the kernel once making the selection in the menu takes forever (I walked away after 1 minute). But it does boot eventually and I just tried with a Fedora 36 nightly, thus using syslinux and the same happens. I believe that the USB stack of the BIOS on this machine simply is terribly slow and reading the big kernel + ramdisk files one sector at a time. So there is nothing we can do here. Since this also happens with syslinux this is not something to worry about.
Regards,
Hans
On Thu, Apr 14, 2022 at 08:59:28PM +0200, Hans de Goede wrote:
Hi Brian,
On 4/14/22 01:52, Brian C. Lane wrote:
A huge thanks to Thomas Schmitt for posting xorrisofs arguments :)
Here is a lorax PR switching to grub2 for BIOS and changing the layout of the iso as described in his post:
https://github.com/weldr/lorax/pull/1226
And a Fedora 36 iso:
https://bcl.fedorapeople.org/boot-grub2-f36.iso
I've tested this with:
- qemu bios -cdrom
- qemu uefi -cdrom
- qemu bios -hda
- qemu uefi -hda
- USB stick on uefi PC hardware with SB off
- USB stick on UEFI PC hardware with SB on
- USB stick on Apple hardware UEFI 2010 Macbook Air and 2012 Macbook Pro
- Media test works on all of the above
I have not tested it on CD or DVD physical media. I have a stack of blank discs but apparently have unplugged all my drives to use their SATA ports for SSDs :)
Thank you so much for working on this.
I've dd-ed the boot-grub2-f36.iso to an USB stick and successfully tested it on the following machines:
- Siemens PC with an i5-2400 with 8G RAM
- Fujitsu-Siemens PC with a Core 2 Duo E6600 2.4 Ghz dual core with 4G RAM
- Dell Latitude E6400 (core 2 duo) with 4G RAM
- Acer One AO532 netbook with a N450 + 2G RAM
The image booted fine on all 4 machines, so from a machine compatibility pov this looks good to me.
Thanks for testing!
There were 2 minor issues with machine 2:
- After the "Welcome to GRUB" text the 3.5" floppy drive
was searched repeatedly before showing the grub bootmenu. It also took like 5-10 seconds to show the menu. I guess that grub is using a minimal grub.cfg which is looking for the actual grub.cfg based on the partition uuid or some such and it is also checking the floppy drive. I wonder if we can tweak grub.cfg on the bootmedia to not do this?
Interesting, it's probably because the eltoritio.img doesn't include a config. I'd need to figure out what to include that would exclude floppies but keep it working everywhere else. I'll ask around and see if there is a generic solution, but I'd rather err on the side of unplugging the floppy drive at this point ;)
- Booting the kernel once making the selection in the menu
takes forever (I walked away after 1 minute). But it does boot eventually and I just tried with a Fedora 36 nightly, thus using syslinux and the same happens. I believe that the USB stack of the BIOS on this machine simply is terribly slow and reading the big kernel + ramdisk files one sector at a time. So there is nothing we can do here. Since this also happens with syslinux this is not something to worry about.
I remember a bug about slow booting a while back, where someone complained that CHS settings in parted were making their boot slow. It ends up grub2 uses INT13 to get the disk geometry, and uses that info for the buffer size. So if the BIOS is reporting something small it might take longer to boot.
https://bugzilla.redhat.com/show_bug.cgi?id=2024261
Brian
FWIW I've now added a commit that makes the same changes for livemedia-creator created isos and tested it with QEMU.
Brian
I installed successfully using physical CD-R and physical DVD+R on American Megatrends Inc version 1613 BIOS of 12/03/2008 running on Intel Core2 Duo (E8400, 3GHz, 8GB). The USB reade was LG Slim Portable DVD Writer GP50NB40 (August 2014). The media check took 5 minutes on CD and 2.5 minutes on DVD. On the installed system, "dd if=/dev/sr0 bs=32k of=/dev/null" was 48 seconds faster than the media check for CD, and 24 seconds faster than the media check for DVD.
On 4/13/22 6:52 PM, Brian C. Lane wrote:
A huge thanks to Thomas Schmitt for posting xorrisofs arguments :)
Here is a lorax PR switching to grub2 for BIOS and changing the layout of the iso as described in his post:
https://github.com/weldr/lorax/pull/1226
And a Fedora 36 iso:
https://bcl.fedorapeople.org/boot-grub2-f36.iso
I've tested this with:
- qemu bios -cdrom
- qemu uefi -cdrom
- qemu bios -hda
- qemu uefi -hda
- USB stick on uefi PC hardware with SB off
- USB stick on UEFI PC hardware with SB on
- USB stick on Apple hardware UEFI 2010 Macbook Air and 2012 Macbook Pro
- Media test works on all of the above
I have not tested it on CD or DVD physical media. I have a stack of blank discs but apparently have unplugged all my drives to use their SATA ports for SSDs :)
Brian
Initial test with a CD-R and an HP Compaq 8510w are mixed.
It boots to GRUB, but it spins a long time blinking the HDD light displaying nothing but "Welcome to GRUB". It eventually spits out "failure reading sector 0x4f838 from 'hd31'." and "you need to load the kernel first." before finally presenting me the GRUB menu.
Any attempt to launch the installer from the GRUB menu results in the same error messages and return to GRUB menu.
The system dual boots Windows 10, if that's meaningful in some way.
Hi,
Initial test with a CD-R and an HP Compaq 8510w are mixed. It boots to GRUB, but it spins a long time blinking the HDD light displaying nothing but "Welcome to GRUB". It eventually spits out "failure reading sector 0x4f838 from 'hd31'."
'hd31' looks strange for HDD as well as for CD-ROM. Do have that many ?
If it's about the CD drive: 0x4f838 = 325688 would be in 2048-byte-block range of boot-grub2-f36.iso which has 333555 such blocks. The ISO 9660 filesystem data range ends at 328317. The rest is the appended EFI partition. If GRUB reports in units of 512-byte blocks instead, then the failure would be at CD-ROM LBA 81422.
You could checkread the CD-R (i assume at /dev/sr0) by
xorriso -indev /dev/sr0 -check_media --
It is supposed to report something like this run with a DVD+RW:
... Volume id : 'Fedora-36-x86_64' xorriso : UPDATE : 32 blocks read in 3 seconds , 0.0xD xorriso : UPDATE : 2912 blocks read in 4 seconds , 4.3xD xorriso : UPDATE : 5824 blocks read in 5 seconds , 4.3xD ... xorriso : UPDATE : 329440 blocks read in 103 seconds , 5.5xD xorriso : UPDATE : 333216 blocks read in 104 seconds , 5.6xD xorriso : UPDATE : 333555 blocks read in 104 seconds = 4.7xD Media checks : lba , size , quality Media region : 0 , 333555 , + good Media region : 333555 , 1961549 , 0 untested
Read errors would be reported with their SCSI error codes and block address. If suspicious periods of low reading speed are noticed, then "Media region" lines might appear with "quality" value "slow" or even "unreadable".
If you deem the full output interesting enough for an examination, send it to me in private. I will then post here what i believe to see in it.
Have a nice day :)
Thomas
On 4/15/22 5:06 PM, Thomas Schmitt wrote:
Hi,
Initial test with a CD-R and an HP Compaq 8510w are mixed. It boots to GRUB, but it spins a long time blinking the HDD light displaying nothing but "Welcome to GRUB". It eventually spits out "failure reading sector 0x4f838 from 'hd31'."
'hd31' looks strange for HDD as well as for CD-ROM. Do have that many ?
Looked strange to me as well, I don't think I do...
Anyway, a reburn fixed it. I can now confirm a CD-R and the aforementioned HP Compaq 8510w boot fine with the test ISO.
Hello, Brian.
On Thursday, 14 April 2022 at 01:52, Brian C. Lane wrote:
A huge thanks to Thomas Schmitt for posting xorrisofs arguments :)
Here is a lorax PR switching to grub2 for BIOS and changing the layout of the iso as described in his post:
https://github.com/weldr/lorax/pull/1226
And a Fedora 36 iso:
Big thanks to you for working on this!
I've tested this with:
- qemu bios -cdrom
- qemu uefi -cdrom
- qemu bios -hda
- qemu uefi -hda
- USB stick on uefi PC hardware with SB off
- USB stick on UEFI PC hardware with SB on
- USB stick on Apple hardware UEFI 2010 Macbook Air and 2012 Macbook Pro
- Media test works on all of the above
I have not tested it on CD or DVD physical media. I have a stack of blank discs but apparently have unplugged all my drives to use their SATA ports for SSDs :)
I've tested your ISO with:
- Dell XPS 15 L502X 8GB RAM (BIOS only) CD-R physical media
Booted and started the installer reasonably fast
- same machine USB media (written to whole device using dd)
Boot fails with "Operation system not found" error. Same USB stick boots fine on another machine (UEFI+SecureBoot), so the medium is fine.
Regards, Dominik
- Dell XPS 15 L502X 8GB RAM (BIOS only) CD-R physical media
It would be very nice to run /sbin/dmidecode, which is on the .iso, and report the "BIOS Information" section, such as: ===== # dmidecode 3.3 Getting SMBIOS data from sysfs. SMBIOS 2.5 present. 70 structures occupying 2511 bytes. Table at 0x000F0730.
Handle 0x0000, DMI type 0, 24 bytes BIOS Information Vendor: American Megatrends Inc. Version: 1613 Release Date: 12/03/2008 Address: 0xF0000 Runtime Size: 64 kB ROM Size: 1 MB Characteristics: ISA is supported PCI is supported PNP is supported APM is supported BIOS is upgradeable BIOS shadowing is allowed ESCD support is available Boot from CD is supported Selectable boot is supported BIOS ROM is socketed EDD is supported 5.25"/1.2 MB floppy services are supported (int 13h) 3.5"/720 kB floppy services are supported (int 13h) 3.5"/2.88 MB floppy services are supported (int 13h) Print screen service is supported (int 5h) 8042 keyboard services are supported (int 9h) Serial services are supported (int 14h) Printer services are supported (int 17h) CGA/mono video services are supported (int 10h) ACPI is supported USB legacy is supported LS-120 boot is supported ATAPI Zip drive boot is supported BIOS boot specification is supported Targeted content distribution is supported BIOS Revision: 16.13 ===== of which the Vendor, Version, and Release Date are the most important. In the installer, type Ctrl+Alt+F2 to get a TTY, re-direct the output of dmidecode into a file, and copy the file to a machine with a more- hospitable environment (using wired ethernet, USB flash memory, etc.)
If there are any errors or unusual behavior, then it also is interesting to know if the interface for the Floppy disk drive is enabled; but often this requires searching through BIOS configuration screens. Sometimes a CD or DVD can be Read under Floppy emulation, and it is easy for the BIOS to try this regardless of settings. Try disabling the Floppy in the BIOS settings, then re-booting.
People,
On 2022-04-18 12:41, John Reiser wrote:
- Dell XPS 15 L502X 8GB RAM (BIOS only) CD-R physical media
It would be very nice to run /sbin/dmidecode, which is on the .iso, and report the "BIOS Information" section, such as:
Is the attached any use?
Phil.
===== # dmidecode 3.3 Getting SMBIOS data from sysfs. SMBIOS 2.5 present. 70 structures occupying 2511 bytes. Table at 0x000F0730.
Handle 0x0000, DMI type 0, 24 bytes BIOS Information Vendor: American Megatrends Inc. Version: 1613 Release Date: 12/03/2008 Address: 0xF0000 Runtime Size: 64 kB ROM Size: 1 MB Characteristics: ISA is supported PCI is supported PNP is supported APM is supported BIOS is upgradeable BIOS shadowing is allowed ESCD support is available Boot from CD is supported Selectable boot is supported BIOS ROM is socketed EDD is supported 5.25"/1.2 MB floppy services are supported (int 13h) 3.5"/720 kB floppy services are supported (int 13h) 3.5"/2.88 MB floppy services are supported (int 13h) Print screen service is supported (int 5h) 8042 keyboard services are supported (int 9h) Serial services are supported (int 14h) Printer services are supported (int 17h) CGA/mono video services are supported (int 10h) ACPI is supported USB legacy is supported LS-120 boot is supported ATAPI Zip drive boot is supported BIOS boot specification is supported Targeted content distribution is supported BIOS Revision: 16.13 ===== of which the Vendor, Version, and Release Date are the most important. In the installer, type Ctrl+Alt+F2 to get a TTY, re-direct the output of dmidecode into a file, and copy the file to a machine with a more- hospitable environment (using wired ethernet, USB flash memory, etc.)
If there are any errors or unusual behavior, then it also is interesting to know if the interface for the Floppy disk drive is enabled; but often this requires searching through BIOS configuration screens. Sometimes a CD or DVD can be Read under Floppy emulation, and it is easy for the BIOS to try this regardless of settings. Try disabling the Floppy in the BIOS settings, then re-booting. _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
It would be very nice to run /sbin/dmidecode, which is on the .iso, and report the "BIOS Information" section, such as:
Is the attached any use?
In the attachment I see these interesting lines (snipped): ===== SMBIOS 3.0.0 present.
BIOS Information Vendor: American Megatrends Inc. Version: 0801 ## first release (01) for product line 08 Release Date: 01/25/2017 ## only 5 years old ROM Size: 16 MB ## probably has lots of VGA-gui "eye candy" Boot from CD is supported Selectable boot is supported USB legacy is supported BIOS boot specification is supported Targeted content distribution is supported UEFI is supported ## claims to be *not* "BIOS-only" BIOS Revision: 5.12 ## the low-level BIOS code
Base Board Information Manufacturer: ASUSTeK COMPUTER INC. Product Name: STRIX Z270H GAMING ## only 3 hardware generations old; current is Z5xx Version: Rev 1.xx =====
On Monday, 18 April 2022 at 04:41, John Reiser wrote:
- Dell XPS 15 L502X 8GB RAM (BIOS only) CD-R physical media
It would be very nice to run /sbin/dmidecode, which is on the .iso, and report the "BIOS Information" section, such as:
Here's mine:
Handle 0x0000, DMI type 0, 24 bytes BIOS Information Vendor: Dell Inc. Version: A12 Release Date: 09/07/2012 Address: 0xE0000 Runtime Size: 128 kB ROM Size: 2560 kB Characteristics: PCI is supported PNP is supported BIOS is upgradeable BIOS shadowing is allowed ESCD support is available Boot from CD is supported Selectable boot is supported BIOS ROM is socketed EDD is supported 5.25"/360 kB floppy services are supported (int 13h) 5.25"/1.2 MB floppy services are supported (int 13h) 3.5"/720 kB floppy services are supported (int 13h) Print screen service is supported (int 5h) 8042 keyboard services are supported (int 9h) Serial services are supported (int 14h) Printer services are supported (int 17h) CGA/mono video services are supported (int 10h) ACPI is supported USB legacy is supported LS-120 boot is supported Smart battery is supported BIOS boot specification is supported Function key-initiated network boot is supported Targeted content distribution is supported BIOS Revision: 0.1
Regards, Dominik
Hi,
Dominik 'Rathann' Mierzejewski wrote:
- same machine USB media (written to whole device using dd)
Boot fails with "Operation system not found" error.
(Looks like my optimism was somewhat premature.)
I wonder from where this message comes. Was it really "Operation system" and not "Operating system" ? In the cleartext strings of boot-grub2-f36.iso i find no matches for the two word string. Searching "peration" yields mostly lines with the word "Microsoft" which i assume stem from the EFI secure boot equipment. Searching for "perating" i find a few matches. None looks like a possible component of the quoted message.
So either i looked for the wrong text snippets, or the message comes from the compressed files in the ISO, or it comes from the firmware.
If it comes from the compressed files, then i assume that GRUB was found and started. Do you see any message that stems from GRUB ?
Same USB stick boots fine on another machine (UEFI+SecureBoot), so the medium is fine.
Just to be clear (and just because i cannot get a grip on the problem):
Is this the same USB stick with same ISO image on it ?
More far fetched:
Can you build a grub-mkrescue ISO from a GRUB development installation for legacy BIOS and EFI ? I don't mean a boot installation of GRUB but rather installed packages equivalent to what Debian offers as: grub-common grub-pc-bin grub-efi-amd64-bin (Sorry, i failed to find the Fedora equivalent of https://tracker.debian.org/pkg/grub2 where i could learn about the equivalent package names.)
With the necesseray GRUB packages installed, do
mkdir minimal touch minimal/empty-file.txt grub-mkrescue -o output.iso minimal
If such an "output.iso" can be built, then it would be interesting to know whether it boots to a "grub> " prompt (more it cannot do without any payload).
Have a nice day :)
Thomas
On Monday, 18 April 2022 at 10:13, Thomas Schmitt wrote:
Hi,
Dominik 'Rathann' Mierzejewski wrote:
- same machine USB media (written to whole device using dd)
Boot fails with "Operation system not found" error.
(Looks like my optimism was somewhat premature.)
I wonder from where this message comes. Was it really "Operation system" and not "Operating system" ?
Yes, that's the exact message. It might be coming from BIOS.
In the cleartext strings of boot-grub2-f36.iso i find no matches for the two word string. Searching "peration" yields mostly lines with the word "Microsoft" which i assume stem from the EFI secure boot equipment. Searching for "perating" i find a few matches. None looks like a possible component of the quoted message.
So either i looked for the wrong text snippets, or the message comes from the compressed files in the ISO, or it comes from the firmware.
If it comes from the compressed files, then i assume that GRUB was found and started. Do you see any message that stems from GRUB ?
None that I can catch. Something might be getting printed to the screen and then cleared before I can see it.
Same USB stick boots fine on another machine (UEFI+SecureBoot), so the medium is fine.
Just to be clear (and just because i cannot get a grip on the problem):
Is this the same USB stick with same ISO image on it ?
Yes, the exact same USB stick works on another machine in UEFI mode. I'll try to dig up my other BIOS-only machine and try it there.
More far fetched:
Can you build a grub-mkrescue ISO from a GRUB development installation for legacy BIOS and EFI ? I don't mean a boot installation of GRUB but rather installed packages equivalent to what Debian offers as: grub-common grub-pc-bin grub-efi-amd64-bin (Sorry, i failed to find the Fedora equivalent of https://tracker.debian.org/pkg/grub2 where i could learn about the equivalent package names.)
https://src.fedoraproject.org/rpms/grub2
With the necesseray GRUB packages installed, do
mkdir minimal touch minimal/empty-file.txt grub-mkrescue -o output.iso minimal
If such an "output.iso" can be built, then it would be interesting to know whether it boots to a "grub> " prompt (more it cannot do without any payload).
Thanks. I'll try that and report back.
Regards, Dominik
Hi,
Dominik 'Rathann' Mierzejewski wrote:
Dell XPS 15 L502X 8GB RAM (BIOS only)
BIOS-only on a laptop might mean the firmware quirk which demands to see the "boot/active" flag at some MBR partition table slot.
See a thread of grub-bug (and later grub-devel) from 2015/16 https://marc.info/?l=grub-bug&m=145052506901159&w=2 where Alexander E. Patrakov wrote: "Subject: [bug #46716] Protective MBR partition is not marked as bootable [...] Changing the byte at offset 0x1be to 0x80 in the resulting iso makes it bootable on such boards."
This led to the creation of xorrisofs option --mbr-force-bootable in xorriso-1.4.3.
Does the following bash hack "repair" the USB stick so that it boots on the affected machine ?
echo $'\x80' | dd of=/dev/sdX conv=notrunc bs=1 seek=446 count=1
(Of course with outmost care when choosing the real sdX name. Users who have a safe copy-image-to-USB-stick program should consider to patch the image file instead and then copy it to the stick. Therefore i added the "conv=notrunc" option to this proposal.)
The difference will not show up in partition editor listings, except possibly as complaint that this boot/active flag with the 0xEE partition is illegal. EFI systems might refuse to boot from this stick for the same reason.
If it turns out that we here face this BIOS bug, then a decision is needed whether those firmwares shall stay supported. If yes, then the mildest known remedy would be to add xorrisofs option
--mbr-force-bootable
which will cause a second MBR partition of type 0x00 from LBA 0 with size 1 and boot/active flag. Something like:
MBR partition table: N Status Type Start Blocks MBR partition : 1 0x00 0xee 1 1334255 MBR partition : 2 0x80 0x00 0 1
EFI implementations are supposed to ignore the partition of type 0x00. Buggy BIOS should be happy with its boot/active flag (Status 0x80). Normal BIOS should not care for that flag, which is rather a hint for a class of x86 MBR code on which PBMR to hop. GRUB's MBR is not of that class.
xorrisofs option --mbr-force-bootable may be added to the grub-mkrescue arguments. If we indeed face this bug, then a grub-mrescue ISO with EFI equipment should not boot on the affected machine, unless this extra option was given at production time.
Have a nice day :)
Thomas
On Mon, Apr 18, 2022 at 5:17 AM Thomas Schmitt scdbackup@gmx.net wrote:
Hi,
Dominik 'Rathann' Mierzejewski wrote:
Dell XPS 15 L502X 8GB RAM (BIOS only)
BIOS-only on a laptop might mean the firmware quirk which demands to see the "boot/active" flag at some MBR partition table slot.
See a thread of grub-bug (and later grub-devel) from 2015/16 https://marc.info/?l=grub-bug&m=145052506901159&w=2 where Alexander E. Patrakov wrote: "Subject: [bug #46716] Protective MBR partition is not marked as bootable [...] Changing the byte at offset 0x1be to 0x80 in the resulting iso makes it bootable on such boards."
This led to the creation of xorrisofs option --mbr-force-bootable in xorriso-1.4.3.
This sounds equivalent to the pbmr_boot option in parted. If so, it's a catch-22 because without the bit set it means some buggy BIOS systems won't boot; but with the bit set it means older versions of Tianocore or anything based on Tianocore up until maybe 6 months ago, won't boot.
Hi,
i wrote:
xorrisofs option --mbr-force-bootable
Chris Murphy wrote:
with the bit set it means older versions of Tianocore or anything based on Tianocore up until maybe 6 months ago, won't boot.
Is this maybe about the code snippet of Tianocore which Alexander E. Patrakov showed in the course of the grub-bug thread of 2015/16 ?
https://marc.info/?l=grub-bug&m=145052969801875&w=2 " OK, Tiano Core validates the protective partition as follows:
// // Verify that the Protective MBR is valid // for (Index = 0; Index < MAX_MBR_PARTITIONS; Index++) { if (ProtectiveMbr->Partition[Index].BootIndicator == 0x00 && ProtectiveMbr->Partition[Index].OSIndicator == PMBR_GPT_PARTITION && UNPACK_UINT32 (ProtectiveMbr->Partition[Index].StartingLBA) == 1 ) { break; } } if (Index == MAX_MBR_PARTITIONS) { goto Done; // i.e. not valid }
So here is an alternative suggestion: don't mark the protective partition, create another dummy MBR partition of type 0x00, mark it as bootable. "
That's what --mbr-force-bootable does. The shown code will find type 0xEE == PMBR_GPT_PARTITION in partition 1 with boot/active flag == 0x00 and be happy.
Others which are more picky will hopefully ignore partitions of type 0x00. UEFI 2.8 , 5.2.1 Legacy Master Boot Record (MBR) says: "The following test must be performed to determine if a legacy MBR is valid: * The Signature must be 0xaa55. * A Partition Record that contains an OSType value of zero or a SizeInLBA value of zero may be ignored."
For now, "may be ignored" seems to be enough of protection against trouble.
An alternative to --mbr-force-bootable could be to offer a 16-byte file for download together with the instructions to patch it into a pure and specs conformant ISO with GPT. Of course, only if a 10 year old laptop refuses to recognize the original ISO on USB stick.
The bytes would have to be
{ 128, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0 }
Assumed the file is named "dummy_mbr_part2.img" the patch instruction would be
dd if=dummy_mbr_part2.img of=boot-grub2-f36.iso \ conv=notrunc bs=1 seek=462 count=16
-------------------------------------------------------------------------
Ubuntu is testing this for you since about a year. :))
Afaik the first official ISO of this layout was ubuntu-21.04-desktop-amd64.iso and they did _not_ revoke it half a year later with ubuntu-21.10-desktop-amd64.iso
$ xorriso -indev ubuntu-21.10-desktop-amd64.iso -report_system_area plain ... Volume id : 'Ubuntu 21.10 amd64' System area options: 0x00004201 System area summary: MBR protective-msdos-label grub2-mbr cyl-align-off GPT ISO image size/512 : 6086880 Partition offset : 16 MBR heads per cyl : 0 MBR secs per head : 0 MBR partition table: N Status Type Start Blocks MBR partition : 1 0x00 0xee 1 6086879 MBR partition : 2 0x80 0x00 0 1 GPT : N Info ...
$ /sbin/fdisk -l ubuntu-21.10-desktop-amd64.iso
Disk ubuntu-21.10-desktop-amd64.iso: 2.9 GiB, 3116482560 bytes, 6086880 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: EF83665C-2F29-48D6-8C3F-80F5A69CFEB7
Device Start End Sectors Size Type ubuntu-21.10-desktop-amd64.iso1 64 6077751 6077688 2.9G Microsoft basic da ubuntu-21.10-desktop-amd64.iso2 6077752 6086215 8464 4.1M EFI System ubuntu-21.10-desktop-amd64.iso3 6086216 6086815 600 300K Microsoft basic da
-------------------------------------------------------------------------
Whatever, we do not yet know for sure whether the boot failure with the Dell XPS 15 L502X is caused by the lack of a boot/active flag.
We also don't know yet whether the failure with "CD-R and an HP Compaq 8510w" that was reported by Brandon Nielsen is due to drive-media problems with the CD-R or due to problems with the ISO.
Have a nice day :)
Thomas
Tested this iso with an USB stick on this machine here and it passed the tests and Anaconda started correctly.
Machine: Type: Laptop System: Hewlett-Packard product: HP G42 Notebook PC Mobo: Hewlett-Packard model: 1484 v: 77.19 BIOS: Hewlett-Packard v: F.17 date: 07/06/2010 CPU Dual core Pentium T4500 2.3GHz RAM 6GB
Geraldo Simião
@geraldosimiao:matrix.org
Em seg, 18 de abr de 2022 14:44, Thomas Schmitt scdbackup@gmx.net escreveu:
Hi,
i wrote:
xorrisofs option --mbr-force-bootable
Chris Murphy wrote:
with the bit set it means older versions of Tianocore or anything based on Tianocore up until maybe 6 months ago, won't boot.
Is this maybe about the code snippet of Tianocore which Alexander E. Patrakov showed in the course of the grub-bug thread of 2015/16 ?
https://marc.info/?l=grub-bug&m=145052969801875&w=2 " OK, Tiano Core validates the protective partition as follows:
// // Verify that the Protective MBR is valid // for (Index = 0; Index < MAX_MBR_PARTITIONS; Index++) { if (ProtectiveMbr->Partition[Index].BootIndicator == 0x00 && ProtectiveMbr->Partition[Index].OSIndicator == PMBR_GPT_PARTITION && UNPACK_UINT32 (ProtectiveMbr->Partition[Index].StartingLBA) == 1 ) { break; } } if (Index == MAX_MBR_PARTITIONS) { goto Done; // i.e. not valid }
So here is an alternative suggestion: don't mark the protective partition, create another dummy MBR partition of type 0x00, mark it as bootable. "
That's what --mbr-force-bootable does. The shown code will find type 0xEE == PMBR_GPT_PARTITION in partition 1 with boot/active flag == 0x00 and be happy.
Others which are more picky will hopefully ignore partitions of type 0x00. UEFI 2.8 , 5.2.1 Legacy Master Boot Record (MBR) says: "The following test must be performed to determine if a legacy MBR is valid:
- The Signature must be 0xaa55.
- A Partition Record that contains an OSType value of zero or a SizeInLBA value of zero may be ignored."
For now, "may be ignored" seems to be enough of protection against trouble.
An alternative to --mbr-force-bootable could be to offer a 16-byte file for download together with the instructions to patch it into a pure and specs conformant ISO with GPT. Of course, only if a 10 year old laptop refuses to recognize the original ISO on USB stick.
The bytes would have to be
{ 128, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0 }
Assumed the file is named "dummy_mbr_part2.img" the patch instruction would be
dd if=dummy_mbr_part2.img of=boot-grub2-f36.iso \ conv=notrunc bs=1 seek=462 count=16
Ubuntu is testing this for you since about a year. :))
Afaik the first official ISO of this layout was ubuntu-21.04-desktop-amd64.iso and they did _not_ revoke it half a year later with ubuntu-21.10-desktop-amd64.iso
$ xorriso -indev ubuntu-21.10-desktop-amd64.iso -report_system_area plain ... Volume id : 'Ubuntu 21.10 amd64' System area options: 0x00004201 System area summary: MBR protective-msdos-label grub2-mbr cyl-align-off GPT ISO image size/512 : 6086880 Partition offset : 16 MBR heads per cyl : 0 MBR secs per head : 0 MBR partition table: N Status Type Start Blocks MBR partition : 1 0x00 0xee 1 6086879 MBR partition : 2 0x80 0x00 0 1 GPT : N Info ...
$ /sbin/fdisk -l ubuntu-21.10-desktop-amd64.iso
Disk ubuntu-21.10-desktop-amd64.iso: 2.9 GiB, 3116482560 bytes, 6086880 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: EF83665C-2F29-48D6-8C3F-80F5A69CFEB7
Device Start End Sectors Size Type ubuntu-21.10-desktop-amd64.iso1 64 6077751 6077688 2.9G Microsoft basic da ubuntu-21.10-desktop-amd64.iso2 6077752 6086215 8464 4.1M EFI System ubuntu-21.10-desktop-amd64.iso3 6086216 6086815 600 300K Microsoft basic da
Whatever, we do not yet know for sure whether the boot failure with the Dell XPS 15 L502X is caused by the lack of a boot/active flag.
We also don't know yet whether the failure with "CD-R and an HP Compaq 8510w" that was reported by Brandon Nielsen is due to drive-media problems with the CD-R or due to problems with the ISO.
Have a nice day :)
Thomas _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Monday, 18 April 2022 at 12:17, Thomas Schmitt wrote:
Hi,
Dominik 'Rathann' Mierzejewski wrote:
Dell XPS 15 L502X 8GB RAM (BIOS only)
BIOS-only on a laptop might mean the firmware quirk which demands to see the "boot/active" flag at some MBR partition table slot.
See a thread of grub-bug (and later grub-devel) from 2015/16 https://marc.info/?l=grub-bug&m=145052506901159&w=2 where Alexander E. Patrakov wrote: "Subject: [bug #46716] Protective MBR partition is not marked as bootable [...] Changing the byte at offset 0x1be to 0x80 in the resulting iso makes it bootable on such boards."
This led to the creation of xorrisofs option --mbr-force-bootable in xorriso-1.4.3.
Does the following bash hack "repair" the USB stick so that it boots on the affected machine ?
echo $'\x80' | dd of=/dev/sdX conv=notrunc bs=1 seek=446 count=1
No, it doesn't. Still getting "Operation System not found". FWIW, I can see the following messages in dmesg when inserting the stick into another machine: [18881.575576] GPT:Primary header thinks Alt. header is not at the end of the disk. [18881.575580] GPT:1334219 != 15974399 [18881.575582] GPT:Alternate GPT header not at the end of the disk. [18881.575583] GPT:1334219 != 15974399 [18881.575584] GPT: Use GNU Parted to correct GPT errors. [18881.575593] sdc: sdc1 sdc2 sdc3 [18881.598493] sd 9:0:0:0: [sdc] Attached SCSI removable disk [18881.943729] ISO 9660 Extensions: Microsoft Joliet Level 3 [18881.945322] ISO 9660 Extensions: Microsoft Joliet Level 3 [18881.946584] ISO 9660 Extensions: RRIP_1991A
[...]
xorrisofs option --mbr-force-bootable may be added to the grub-mkrescue arguments. If we indeed face this bug, then a grub-mrescue ISO with EFI equipment should not boot on the affected machine, unless this extra option was given at production time.
I'll try the grub2-mkrescue ISO next.
Regards, Dominik
On Tuesday, 19 April 2022 at 08:46, Dominik 'Rathann' Mierzejewski wrote:
On Monday, 18 April 2022 at 12:17, Thomas Schmitt wrote:
[...]
xorrisofs option --mbr-force-bootable may be added to the grub-mkrescue arguments. If we indeed face this bug, then a grub-mrescue ISO with EFI equipment should not boot on the affected machine, unless this extra option was given at production time.
I'll try the grub2-mkrescue ISO next.
And indeed, it doesn't boot. Same "Operation System not found" message.
Regards, Dominik
On Tuesday, 19 April 2022 at 09:08, Dominik 'Rathann' Mierzejewski wrote:
On Tuesday, 19 April 2022 at 08:46, Dominik 'Rathann' Mierzejewski wrote:
On Monday, 18 April 2022 at 12:17, Thomas Schmitt wrote:
[...]
xorrisofs option --mbr-force-bootable may be added to the grub-mkrescue arguments. If we indeed face this bug, then a grub-mrescue ISO with EFI equipment should not boot on the affected machine, unless this extra option was given at production time.
I'll try the grub2-mkrescue ISO next.
And indeed, it doesn't boot. Same "Operation System not found" message.
Looks like it's a common issue with USB stick booting on these old Dell XPS 15s: https://askubuntu.com/questions/611135/operation-system-not-found-during-usb... https://askubuntu.com/questions/451931/operation-system-not-found-dell-xps-1... https://www.dell.com/community/Laptops-General-Read-Only/Operation-System-No... https://www.dell.com/community/SupportAssist-Enterprise/Trying-to-boot-from-... https://bbs.archlinux.org/viewtopic.php?id=226003
Regards, Dominik
Hi,
Dominik 'Rathann' Mierzejewski wrote:
Looks like it's a common issue with USB stick booting on these old Dell XPS 15s: https://askubuntu.com/questions/611135/operation-system-not-found-during-usb... https://askubuntu.com/questions/451931/operation-system-not-found-dell-xps-1... https://www.dell.com/community/Laptops-General-Read-Only/Operation-System-No... https://www.dell.com/community/SupportAssist-Enterprise/Trying-to-boot-from-...
These look like the problem we face. But no remedy to see.
This was probably a problem with the MBR on hard disk which wanted to hop on a PMBR but was directed to the wrong partition by the boot/active flag.
Whatever, from the boot success of the old Fedora ISO we know that the problem can be avoided. The riddle is to find out what does the trick.
Building something nearer to the old ISO layout:
new_iso=test_mjgappended.iso
# Extract the EFI partition dd if=boot-grub2-f36.iso bs=512 skip=1313268 count=20288 of=test_efi.img
# Extract the MBR code dd if=boot-grub2-f36.iso bs=1 count=446 of=test_mbr.img
# Pad MBR up to 32768 bytes, so that it overwrites the loaded system area dd if=/dev/zero bs=1 count=32322 >>test_mbr.img
# xorriso in its native mode does not truncate -outdev test -e "$new_iso" && rm "$new_iso"
# The boot lures need to be thoroughly re-arranged. # To avoid the need for being superuser let xorriso load the file tree # of the original ISO. (Not only mounting but also reading the file # -rwx------ /mountpoint/EFI/BOOT/BOOTIA32.EFI would need root rights.) xorriso \ -indev boot-grub2-f36.iso \ -outdev "$new_iso" \ -as mkisofs \ -iso_mbr_part_type 0x83 \ --grub2-mbr test_mbr.img \ -append_partition 2 0xef test_efi.img \ -part_like_isohybrid \ --boot-catalog-hide \ -b /images/eltorito.img \ -no-emul-boot -boot-load-size 4 -boot-info-table \ --grub2-boot-info \ -eltorito-alt-boot \ -e '--interval:appended_partition_2:all::' \ -no-emul-boot -isohybrid-gpt-basdat \ -R -J \ -- \ -padding included \ -changes_pending yes \ -stdio_sync off
The resulting ISO is MBR partitioned:
$ /sbin/fdisk -l test_mjgappended.iso ... Disklabel type: dos ... Device Boot Start End Sectors Size Id Type test_mjgappended.iso1 * 0 1313135 1313136 641.2M 83 Linux test_mjgappended.iso2 1313136 1333423 20288 9.9M ef EFI (FAT-12/16/32)
but has the inofficial smell of GPT by bearing a GPT header and a GPT partition slot which marks the EFI partition (not by the EFI type GUUID). Quite like the current Fedora ISOs do.
----------------------------------------------------------------------
If above boots on the Dell XPS 15 L502X, we should confirm that the MBR boot/active trick is really, really not the decisive difference to boot-grub2-f36.iso. I.e. whether the resulting ISO from this repack run does _not_ work:
new_iso=test_bootactive.iso
# xorriso in its native mode does not truncate -outdev test -e "$new_iso" && rm "$new_iso"
# The original boot layout can be reused with one little adjustment xorriso -indev boot-grub2-f36.iso \ -outdev "$new_iso" \ -boot_image any replay \ -boot_image any mbr_force_bootable=on \ -padding included \ -changes_pending yes \ -stdio_sync off
----------------------------------------------------------------------
Have a nice day :)
Thomas
On Tuesday, 19 April 2022 at 17:34, Thomas Schmitt wrote:
Hi,
Dominik 'Rathann' Mierzejewski wrote:
Looks like it's a common issue with USB stick booting on these old Dell XPS 15s: https://askubuntu.com/questions/611135/operation-system-not-found-during-usb... https://askubuntu.com/questions/451931/operation-system-not-found-dell-xps-1... https://www.dell.com/community/Laptops-General-Read-Only/Operation-System-No... https://www.dell.com/community/SupportAssist-Enterprise/Trying-to-boot-from-...
These look like the problem we face. But no remedy to see.
[...]
Whatever, from the boot success of the old Fedora ISO we know that the problem can be avoided. The riddle is to find out what does the trick.
Building something nearer to the old ISO layout:
[...]
Unfortunately, that one doesn't boot.
If above boots on the Dell XPS 15 L502X, we should confirm that the MBR boot/active trick is really, really not the decisive difference to boot-grub2-f36.iso. I.e. whether the resulting ISO from this repack run does _not_ work:
[...]
Indeed, this one doesn't boot, either.
Regards, Dominik
Hi,
i wrote:
Does it boot with ubuntu-21.10-desktop-amd64.iso ?
Dominik 'Rathann' Mierzejewski wrote:
It doesn't. Same error. [...] $ grub2-mkrescue -o output-mbr.iso --mbr-force-bootable minimal gives me the "Operation System not found" BIOS error again.
Building something nearer to the old ISO layout: new_iso=test_mjgappended.iso
Unfortunately, that one doesn't boot.
So it is really a new BIOS quirk. It is astounding how much the old jackalope ISO snuggles itself into such firmwares.
I'll try modifying the ISO from the beginning of this thread according to your instructions from another subthread.
If it is about the variant with options -part_like_isohybrid and -isohybrid-gpt-basdat then i have to warn of a bug in xorriso which i found while preparing the next experiment. The bug prevents the production of the MBR partition table in this case and thus makes the resulting ISO unbootable on EFI from USB stick.
BIOS booting should be ok. But the wrong result is not a candidate for a generally bootable ISO. Maybe it is not even significant for a test.
I am currently testing a remedy, but this will last a few days because i have to be quite paranoid with changing the boot equipment code in libisofs. When it's ready, i will ask you to download and compile a GNU xorriso 1.5.5 development snapshot. (The compilation has no special dependencies.)
------------------------------------------------------------------------
Whatever, if you want to try already now with the buggy result, begin like with the last experiment. You can re-use the files from the last time:
# Extract the EFI partition dd if=boot-grub2-f36.iso bs=512 skip=1313268 count=20288 of=test_efi.img
# Extract the MBR code dd if=boot-grub2-f36.iso bs=1 count=446 of=test_mbr.img
# Pad MBR up to 32768 bytes, so that it overwrites the loaded system area dd if=/dev/zero bs=1 count=32322 >>test_mbr.img
The xorriso run shall bring the ISO as near to the old layout as possible:
new_iso=test_oldlayout.iso
# xorriso in its native mode does not truncate -outdev test -e "$new_iso" && rm "$new_iso"
xorriso \ -indev boot-grub2-f36.iso \ -outdev "$new_iso" \ -map test_efi.img /images/efiboot.img \ -as mkisofs \ -part_like_isohybrid \ --grub2-mbr test_mbr.img \ --boot-catalog-hide \ -b /images/eltorito.img \ -no-emul-boot -boot-load-size 4 -boot-info-table \ --grub2-boot-info \ -eltorito-alt-boot \ -e images/efiboot.img \ -no-emul-boot -isohybrid-gpt-basdat \ -R -J \ -- \ -padding included \ -compliance no_emul_toc \ -stdio_sync off
After bug correction the result will look like this:
$ xorriso -indev "$new_iso" -report_system_area plain -report_el_torito plain [...] System area options: 0x00004a00 System area summary: MBR grub2-mbr cyl-align-off GPT ISO image size/512 : 1334000 Partition offset : 0 MBR heads per cyl : 64 MBR secs per head : 32 MBR partition table: N Status Type Start Blocks MBR partition : 1 0x80 0x17 0 1334000 MBR partition : 2 0x00 0xef 1313032 20288 MBR partition path : 2 /images/efiboot.img GPT : N Info GPT disk GUID : 63f762129e2da8499aee97f68461f45f GPT entry array : 2 248 separated GPT lba range : 64 1333936 1333999 GPT partition name : 1 490053004f004800790062007200690064003100 GPT partname local : 1 ISOHybrid1 GPT partition GUID : 1 63f762129e2da8499aef97f68461f45f GPT type GUID : 1 a2a0d0ebe5b9334487c068b6b72699c7 GPT partition flags: 1 0x1000000000000001 GPT start and size : 1 1313032 20288 GPT partition path : 1 /images/efiboot.img El Torito catalog : 77 1 El Torito images : N Pltf B Emul Ld_seg Hdpt Ldsiz LBA El Torito boot img : 1 BIOS y none 0x0000 0x00 4 78 El Torito boot img : 2 UEFI y none 0x0000 0x00 20288 328258 El Torito img path : 1 /images/eltorito.img El Torito img opts : 1 boot-info-table grub2-boot-info El Torito img path : 2 /images/efiboot.img
$ /sbin/fdisk -l "$new_iso" [...] Disklabel type: dos [...] Disk identifier: 0x00000000
Device Boot Start End Sectors Size Id Type test_oldlayout.iso1 * 0 1333959 1333960 651.4M 17 Hidden HPFS/NTFS test_oldlayout.iso2 1313008 1333295 20288 9.9M ef EFI (FAT-12/16/32)
With the bug still in xorriso, the lines which begin by "MBR" will be missing. fdisk will report no partitions. (Normal BIOS would not care about these defects.)
Have a nice day :)
Thomas
Hi,
i wrote:
Whatever, if you want to try already now with the buggy result,
It comes to me that the MBR signature will be missing too. So it won't be recognized by BIOS and its x86 code won't start.
The signature could be inserted by
echo $'\x55'$'\xAA' | dd of="$new_iso" conv=notrunc bs=1 seek=510 count=2
Have a nice day :)
Thomas
Hi,
i uploaded a new GNU xorriso development tarball for the next experiment with the Dell XPS 15 L502X of Dominik 'Rathann' Mierzejewski.
The program will not interfere with the libraries or the /usr/bin/xorriso binary which gets installed by Fedora from the xorriso package. There are no special hard dependencies. But i assume that the Fedora xorriso gets built with these optional dependencies: - libreadline development package - zlib development package - libacl development package None of them should be needed for the experiment.
Download:
https://www.gnu.org/software/xorriso/xorriso-1.5.5.tar.gz
MD5: e622b2547202529fbee6c9186fe39c40 (This might change with future uploads of the xorriso-1.5.5 snapshot. The next stable release will be 1.5.6.)
Build:
cd /...some.../...playground.../...directory... tar xzf /...where_it_was_downloaded.../xorriso-1.5.5.tar.gz cd xorriso-1.5.5 ./configure --prefix=/usr make
This will produce a binary named ./xorriso/xorriso
Installation is not necessary. Just execute the path that is given by ls $(pwd)/xorriso/xorriso
Check whether it is willing to run:
/...path.../xorriso/xorriso -version
It should report:
GNU xorriso 1.5.5 : RockRidge filesystem manipulator, libburnia project.
GNU xorriso 1.5.5 ISO 9660 Rock Ridge filesystem manipulator and CD/DVD/BD burn program Copyright (C) 2021, Thomas Schmitt scdbackup@gmx.net, libburnia project. xorriso version : 1.5.5 Version timestamp : 2022.04.22.113324 Build timestamp : -none-given- libisofs in use : 1.5.5 (min. 1.5.5) libjte in use : 2.0.0 (min. 2.0.0) libburn in use : 1.5.5 (min. 1.5.5) libburn OS adapter: internal GNU/Linux SG_IO adapter sg-linux libisoburn in use : 1.5.5 (min. 1.5.5) License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
The "libjte" line will only appear if zlib development was present at compile time. All reported libraries are part of the tarball and linked statically.
-------------------------------------------------------------------------
Now for the partition layout with GRUB which comes nearest to the layout that was produced with ISOLINUX (quite like the result of isoybrid --uefi). Begin like with the last experiment:
# Extract the EFI partition dd if=boot-grub2-f36.iso bs=512 skip=1313268 count=20288 of=test_efi.img
# Extract the MBR code dd if=boot-grub2-f36.iso bs=1 count=446 of=test_mbr.img
# Pad MBR up to 32768 bytes, so that it overwrites the loaded system area dd if=/dev/zero bs=1 count=32322 >>test_mbr.img
The xorriso run shall bring the ISO as near to the old layout as possible:
xorriso=/...path.../xorriso/xorriso
new_iso=test_oldlayout.iso
# xorriso in its native mode does not truncate -outdev test -e "$new_iso" && rm "$new_iso"
"$xorriso" \ -indev boot-grub2-f36.iso \ -outdev "$new_iso" \ -map test_efi.img /images/efiboot.img \ -as mkisofs \ -part_like_isohybrid \ --grub2-mbr test_mbr.img \ --boot-catalog-hide \ -b /images/eltorito.img \ -no-emul-boot -boot-load-size 4 -boot-info-table \ --grub2-boot-info \ -eltorito-alt-boot \ -e images/efiboot.img \ -no-emul-boot -isohybrid-gpt-basdat \ -R -J \ -- \ -padding included \ -compliance no_emul_toc \ -stdio_sync off
Inspect the resulting boot equipment and partition layout:
"$xorriso" -indev "$new_iso" -report_el_torito plain -report_system_area plain
This should report:
El Torito catalog : 71 1 El Torito images : N Pltf B Emul Ld_seg Hdpt Ldsiz LBA El Torito boot img : 1 BIOS y none 0x0000 0x00 4 72 El Torito boot img : 2 UEFI y none 0x0000 0x00 20288 328252 El Torito img path : 1 /images/eltorito.img El Torito img opts : 1 boot-info-table grub2-boot-info El Torito img path : 2 /images/efiboot.img System area options: 0x00004a00 System area summary: MBR grub2-mbr cyl-align-off GPT ISO image size/512 : 1333960 Partition offset : 0 MBR heads per cyl : 64 MBR secs per head : 32 MBR partition table: N Status Type Start Blocks MBR partition : 1 0x80 0x00 0 1333960 MBR partition : 2 0x00 0xef 1313008 20288 MBR partition path : 2 /images/efiboot.img GPT : N Info GPT disk GUID : 2040609c42db0e43ad879c51ee608215 GPT entry array : 2 248 separated GPT lba range : 64 1333896 1333959 GPT partition name : 1 490053004f004800790062007200690064003100 GPT partname local : 1 ISOHybrid1 GPT partition GUID : 1 2040609c42db0e43ad869c51ee608215 GPT type GUID : 1 a2a0d0ebe5b9334487c068b6b72699c7 GPT partition flags: 1 0x1000000000000001 GPT start and size : 1 1313008 20288 GPT partition path : 1 /images/efiboot.img
-------------------------------------------------------------------------
(For now i have to thank the starter of this thread for causing me to test above unusual setup and to run my big regression test for the first time since february 2021. Three bugs showed up and are now hopefully fixed.)
Have a nice day :)
Thomas
On Sat, Apr 23, 2022 at 10:43:39AM +0200, Thomas Schmitt wrote:
Hi,
i uploaded a new GNU xorriso development tarball for the next experiment with the Dell XPS 15 L502X of Dominik 'Rathann' Mierzejewski.
I followed your instructions and confirmed that this method still works with qemu bios/uefi/cdrom/hd as well has UEFI laptop, MacBook Air, and old Dell Inspiron 1525.
If this fixes the boot issues with the XPS 15 then it's probably worth using this instead of the 'clean' GPT method and then revisit later once BIOS support finally goes away.
Thanks,
Brian
Hi,
Brian C. Lane wrote:
If this fixes the boot issues with the XPS 15 then it's probably worth using this instead of the 'clean' GPT method and then revisit later once BIOS support finally goes away.
(Now i am not sure whether i shall hope for a significant test result.)
It should be noted that Ubuntu publishes 'nearly clean' GPT with boot flag MBR partition since a year. 22.04 is out and still bears this layout. So there are not many machines around which suffer from the yet unknown particular demand of the Dell XPS 15 L502X.
I am still pondering what to try next when the experiment results are in.
If test_oldlayout.iso boots, then the next step would possibly to deface the GPT of boot-grub2-f36.iso by changing the type of MBR partition 1 from 0xEE to 0x83 or 0x00. If that does not boot, then i would ask for setting the boot flag of partition 1. (These are not proposed solutions. The experiments shall just tell what exactly the BIOS expects and does not get from boot-grub2-f36.iso.)
If test_oldlayout.iso does not boot, then i am quite clueless. I doubt that the heavily illegal GPT partition for the overall ISO in the old Fedora ISOs does the trick. I also doubt that the presence of the macboot.img is significant for a legacyBIOS. It is mentioned with type 0x00 in the MBR partition table and as "Apple UFS" in GPT.
So only success with test_oldlayout.iso will bring more insight. But that might also perpetuate the really undesirable test_oldlayout.iso.
Have a nice day :)
Thomas
On Tue, Apr 26, 2022 at 5:17 PM Thomas Schmitt scdbackup@gmx.net wrote:
Hi,
Brian C. Lane wrote:
If this fixes the boot issues with the XPS 15 then it's probably worth using this instead of the 'clean' GPT method and then revisit later once BIOS support finally goes away.
(Now i am not sure whether i shall hope for a significant test result.)
It should be noted that Ubuntu publishes 'nearly clean' GPT with boot flag MBR partition since a year. 22.04 is out and still bears this layout. So there are not many machines around which suffer from the yet unknown particular demand of the Dell XPS 15 L502X.
I am still pondering what to try next when the experiment results are in.
If test_oldlayout.iso boots, then the next step would possibly to deface the GPT of boot-grub2-f36.iso by changing the type of MBR partition 1 from 0xEE to 0x83 or 0x00. If that does not boot, then i would ask for setting the boot flag of partition 1. (These are not proposed solutions. The experiments shall just tell what exactly the BIOS expects and does not get from boot-grub2-f36.iso.)
If test_oldlayout.iso does not boot, then i am quite clueless. I doubt that the heavily illegal GPT partition for the overall ISO in the old Fedora ISOs does the trick. I also doubt that the presence of the macboot.img is significant for a legacyBIOS. It is mentioned with type 0x00 in the MBR partition table and as "Apple UFS" in GPT.
So only success with test_oldlayout.iso will bring more insight. But that might also perpetuate the really undesirable test_oldlayout.iso.
I don't know if it's been mentioned, but until very recently, pretty much all TianoCore based UEFI implementations failed to boot protective MBR marked GPT partitions. I know this because I fixed that issue last year[1] when working on hybridizing Fedora Cloud images, and the issue has existed since at least 2007.
That doesn't mean that all proprietary UEFI implementations retain this bug from their forked TianoCore implementations that make up their UEFI codebases, but it would not surprise me if a lot of them do.
[1]: https://github.com/tianocore/edk2/commit/b3db0cb1f8d163f22b769c205c6347376a3...
Hi,
Neal Gompa wrote:
I don't know if it's been mentioned,
IIRC, Chris Murphy did.
but until very recently, pretty much all TianoCore based UEFI implementations failed to boot protective MBR marked GPT partitions. [...] https://github.com/tianocore/edk2/commit/b3db0cb1f8d163f22b769c205c6347376a3...
First of all: Thanks for fixing this.
But yes, the problem is known and a workaround is known in form of the little dummy MBR partition which takes the plight of carrying the boot/active flag. Old Tianocore will find the 0xEE partition first, see no boot flag there, and break the loop before it inspects the dummy partition.
Ubuntu's current ISOs have this dummy MBR partition of type 0x00 and size 1. They seem to boot with the EFI implementations which are around.
(Not a complaint towards Tianocore but just an observation:
That loop is still barely specs compliant. But now it is on the liberal side. UEFI 2.8 says "5.2.3 Protective MBR [...] One of the Partition Records shall be as defined in table 12, reserving the entire space on the disk after the Protective MBR itself for the GPT disk layout. [...] The remaining Partition Records shall each be set to zeros. " Well, i guess "table 12" was meant as "table 20" which describes the partition slot of type 0xEE. Whether "shall" should be implemented as "must" is a matter of hairsplitting. A picky loop would not break at the first success but rather check all four partition slots. On the other hand it would be nice if it would project the "may be ignored" of "5.2.1 Legacy Master Boot Record (MBR) [...] A Partition Record that contains an OSType value of zero or a SizeInLBA value of zero may be ignored. " onto the rules for a Protective MBR.
It would be unfortunate if this fragile compromise would be broken by increased pickiness of EFI implementations. To my understanding only the liberal interpretation of a "shall" and a slightly displaced "may" in the specs protects it. )
I am somewhat astonished that no BIOS machine was encountered yet in the tests of boot-grub2-f36.iso which would refuse to boot with its pure GPT-marking MBR table but would boot if the boot/active flag is set somwhere in the MBR partition table. Testers of grub-mkrescue ISOs and of Ubuntu ISOs encountered such machines.
Have a nice day :)
Thomas
Hello, Thomas. Thanks for your patience. I would've gotten the results sooner, but life got in the way.
On Tuesday, 26 April 2022 at 23:18, Thomas Schmitt wrote:
Hi,
If test_oldlayout.iso does not boot, then i am quite clueless.
I'm afraid that this is the case. I'll try to re-do the ISO mangling procedure again, but I don't think I made any mistakes in copy-pasting your command lines.
I doubt that the heavily illegal GPT partition for the overall ISO in the old Fedora ISOs does the trick. I also doubt that the presence of the macboot.img is significant for a legacyBIOS. It is mentioned with type 0x00 in the MBR partition table and as "Apple UFS" in GPT.
So only success with test_oldlayout.iso will bring more insight. But that might also perpetuate the really undesirable test_oldlayout.iso.
Sorry to be the bearer of bad news, then. :/
Regards, Dominik
Hi,
i wrote:
If test_oldlayout.iso does not boot, then i am quite clueless.
Dominik 'Rathann' Mierzejewski wrote:
I'm afraid that this is the case.
What a naughty machine. :))
I'll think about making a BIOS-only GRUB ISO, just for testing. (Am busy with finding out how to merge the two Debian DLBD ISOs to a single bootable all-in-one ISO with 78 GiB, 58,000 packages and official attire.)
------------------------------------------------------------------------ Some info about peculiar BIOS machines:
In the course of lengthy https://bugs.launchpad.net/ubuntu/+source/casper/+bug/1922342 we found machines which would probably boot fine with the current candidate for a new Fedora ISO but show a delay of ~ 6 minutes with the current Ubuntu ISOs. Obviously their firmwares don't play well with GRUB when the dummy MBR partition with the boot flag is present: - tablet computer "motion computing j3400". - mainboard Gigabyte H61M-D2H-USB3 - possibly mainboard Gigabyte GA-970A-DS3 (It is slow on booting, but we got no confirmation that removing the dummy MBR partiton would make it faster) All are about 10 years old.
On the other hand we re-confirmed that not only some laptops but also the following stationary machines do not boot at all from USB stick if no MBR boot flag is present: - HP Compaq dc7700 - HP Compaq dc7900 (again about 10 years old). So these machines will most probably not boot with the current candidate for a new Fedora ISO.
It might be helpful to publish a shell script as workaround for the HPs, which puts the dummy MBR partition entry into the new Fedora ISO. From Ubuntu's https://git.launchpad.net/ubuntu/+source/casper/tree/scripts/casper-helpers#... by Steve Langasek:
# detection of dash vs busybox/bash if [ "$(echo -e foo)" = "-e foo" ]; then escape_arg="" else escape_arg=-e fi echo $escape_arg -n '\0200\00\01\00\00\00\01\00\00\00\00\00\01\00\00\00' \ | dd of=$DEVICE bs=1 seek=462 conv=notrunc count=16
(This is done after fdisk added a GPT partition for persistent data and made the MBR partition table specs compliant, like in the new Fedora ISO. $DEVICE is the USB stick from which the ISO booted. I would propose the affected Fedora users to do this with the ISO image before putting it on the USB stick. Maybe it has to be repeated on the stick after the first boot if the software in the ISO creates a new GPT partition, too.)
Have a nice day :)
Thomas
Hi,
another experiment proposal for Dominik 'Rathann' Mierzejewski:
Zeroize the GPT of a copy of boot-grub2-f36.iso and adjust MBR partition 1. Do this in bash for the $'\x...' gestures:
# Create the playground cp boot-grub2-f36.iso test.iso
# Zeroize the main GPT dd if=/dev/zero bs=1 seek=512 count=32256 conv=notrunc of=test.iso
# Zeroize the header block of the backup GPT dd if=/dev/zero bs=512 seek=1334219 count=1 conv=notrunc of=test.iso
# Set the boot flag to MBR partition 1 echo $'\x80' | dd bs=1 seek=446 count=1 conv=notrunc of=test.iso
# Change the partition type to 0x17 (as with BIOS-only ISOLINUX isohybrid) echo $'\x17' | dd bs=1 seek=450 count=1 conv=notrunc of=test.iso
# Set the start of MBR partition 1 to LBA 64, so that it is mountable echo $'\x40' | dd bs=1 seek=454 count=1 conv=notrunc of=test.iso
# Append 63 blocks of 512 bytes to compensate for the new start LBA # with the old partition size. (Else partition editors will hate the # image because it is too small for the partition.) dd if=/dev/zero bs=512 count=63 >>test.iso
Check whether partition editors like the result and see the boot flag:
$ /sbin/fdisk -l test.iso ... Disklabel type: dos ... Device Boot Start End Sectors Size Id Type test.iso1 * 64 1334282 1334219 651.5M 17 Hidden HPFS/NTFS
Put test.iso onto the USB stick as usual.
The result is supposed to boot from USB stick only via legacy BIOS. EFI would still see the System Partition on a CD or DVD because it is announced by the El Torito catalog.
If it does not boot on the Dell XPS 15 L502X, try whether it boots on some other BIOS machine. I only roughly tested that qemu-system-x86_64 -enable-kvm -m 512 -hda test.iso boots to the GRUB menu.
Have a nice day :)
Thomas
Hi,
i wrote:
Does the following bash hack "repair" the USB stick so that it boots on the affected machine ? echo $'\x80' | dd of=/dev/sdX conv=notrunc bs=1 seek=446 count=1
Dominik 'Rathann' Mierzejewski wrote:
No, it doesn't. Still getting "Operation System not found".
A new firmware riddle.
Does it boot with ubuntu-21.10-desktop-amd64.iso ? (That's supposed to be pure GPT with the little extra MBR partition.)
FWIW, I can see the following messages in dmesg when inserting the stick into another machine: [18881.575576] GPT:Primary header thinks Alt. header is not at the end of the disk. [18881.575580] GPT:1334219 != 15974399 [18881.575582] GPT:Alternate GPT header not at the end of the disk. [18881.575583] GPT:1334219 != 15974399
That has to be expected because the ISO image is smaller than the storage device on which it was put. (That's a reason why i promoted MBR partition table for ISOs, until the first EFI impementations showed up which demand a smell of GPT.)
A GPT problem is unlikely to influence the first stage of BIOS booting.
[18881.575584] GPT: Use GNU Parted to correct GPT errors.
If you follow this advice then possibly the boot/active flag of the 0xEE MBR partition will be reset to 0. (I remember that Ubuntu had a sanitizing step after booting which made the USB stick specs compliant and thus prevented further booting of the stick on the affected BIOS machine until the dummy partition was recreated. https://bugs.launchpad.net/ubuntu-cdimage/+bug/1899308 especially https://bugs.launchpad.net/ubuntu-cdimage/+bug/1899308/comments/60 )
I'll try the grub2-mkrescue ISO next.
Once made with and once without extra option --mbr-force-bootable please. (We are now using a long stick to poke around in the mist.)
Have a nice day :)
Thomas
On Tuesday, 19 April 2022 at 09:20, Thomas Schmitt wrote:
Hi,
i wrote:
Does the following bash hack "repair" the USB stick so that it boots on the affected machine ? echo $'\x80' | dd of=/dev/sdX conv=notrunc bs=1 seek=446 count=1
Dominik 'Rathann' Mierzejewski wrote:
No, it doesn't. Still getting "Operation System not found".
A new firmware riddle.
Does it boot with ubuntu-21.10-desktop-amd64.iso ? (That's supposed to be pure GPT with the little extra MBR partition.)
It doesn't. Same error.
[...]
I'll try the grub2-mkrescue ISO next.
Once made with and once without extra option --mbr-force-bootable please. (We are now using a long stick to poke around in the mist.)
Image generated with:
$ grub2-mkrescue -o output-mbr.iso --mbr-force-bootable minimal
gives me the "Operation System not found" BIOS error again.
I'll try modifying the ISO from the beginning of this thread according to your instructions from another subthread.
Regards, Dominik
On Sun, Apr 17, 2022 at 03:12:59PM +0200, Dominik 'Rathann' Mierzejewski wrote:
I've tested your ISO with:
- Dell XPS 15 L502X 8GB RAM (BIOS only) CD-R physical media
Booted and started the installer reasonably fast
- same machine USB media (written to whole device using dd)
Boot fails with "Operation system not found" error. Same USB stick boots fine on another machine (UEFI+SecureBoot), so the medium is fine.
In my experience this mean the BIOS didn't find anything it wants to boot. Does a current F35 iso boot on this system when written to USB?
Thanks,
Brian
On Monday, 18 April 2022 at 19:00, Brian C. Lane wrote:
On Sun, Apr 17, 2022 at 03:12:59PM +0200, Dominik 'Rathann' Mierzejewski wrote:
I've tested your ISO with:
- Dell XPS 15 L502X 8GB RAM (BIOS only) CD-R physical media
Booted and started the installer reasonably fast
- same machine USB media (written to whole device using dd)
Boot fails with "Operation system not found" error. Same USB stick boots fine on another machine (UEFI+SecureBoot), so the medium is fine.
In my experience this mean the BIOS didn't find anything it wants to boot. Does a current F35 iso boot on this system when written to USB?
Fedora 35 MATE-Compiz Live ISO boots fine.
Regards, Dominik
Booted on the old Dell r710 via classic bios.
On 4/13/22 19:52, Brian C. Lane wrote:
A huge thanks to Thomas Schmitt for posting xorrisofs arguments :)
Here is a lorax PR switching to grub2 for BIOS and changing the layout of the iso as described in his post:
Hey Brian, thanks for sharing. Is this an effort to drop the isolinux/syslinux stack from the equation?
In Fedora CoreOS we're using isolinux (for BIOS) and GRUB2 for UEFI for the ISO image we deliver. We're not opposed to using GRUB2 for BIOS too (happy to in fact), but we want to make sure things keep working.
Currently we use isohybrid [1] (also from syslinux package) to modify the ISO to be able to be booted on a USB key. Would we be able to still get that functionality without isohybrid from the syslinux package?
[1] https://github.com/coreos/coreos-assembler/blob/b8f61207c7a4ce345a1f136823da...
Hi,
Dusty Mabe wrote:
Currently we use isohybrid [1] (also from syslinux package) to modify the ISO to be able to be booted on a USB key. Would we be able to still get that functionality without isohybrid from the syslinux package?
Yes. The GRUB2 MBR code will get the equivalent treatment already when the ISO image gets created. A postprocessing run will not be needed for getting the ability to boot from USB stick. The ESP will be marked by an entry in a GPT partition table.
But some BIOS implementations might not recognize the bootability of the stick when pure GPT is used. We have the example of Dell XPS 15 L502X which is not fully explored yet. GRUB2 development encountered other machines which did not boot if not some MBR partition beared the boot/active flag.
This is not particularly a GRUB problem. But the isohybrid program of ISOLINUX always creates a MBR partition with boot/active flag. The price is that isohybrid never produces a valid GPT.
https://github.com/coreos/coreos-assembler/blob/b8f61207c7a4ce345a1f136823da...
Insofar the comment in line 639 of src/cmd-buildextend-live is not fully correct: "Add MBR, and GPT with ESP, ..." The ESP is marked by isohybrid --uefi in a UEFI compliant way in the MBR partition table as partition of type 0xEF. isohybrid creates a GPT indeed. But is is not valid because not announced by a MBR partition of type 0xEE. Further the GPT partition entry which marks the ESP does not have the prescribed type GUID for EFI, but rather the type GUID for a Microsoft basic data partition.
The new partition layout in boot-grub2-f36.iso would make the comment fully correct: The GPT is valid and the ESP entry has the right type GUID. But then you would have to disable the run of /usr/bin/isohybrid and thus need no comment at all.
You would also have to switch from genisoimage to xorrisofs in the "x86_64" case. Several options would have to be changed.
Have a nice day :)
Thomas
-
Brandon Nielsen -
Brian C. Lane -
Chris Murphy -
Dominik 'Rathann' Mierzejewski -
Dusty Mabe -
Geraldo Simião Kutz -
Hans de Goede -
John Reiser -
Neal Gompa -
Philip Rhoades -
Thomas Schmitt -
Łukasz Posadowski