Hi,
I've been trying to get the Fedora 10 Snapshot ISO to work boot via a USB key on my MacBook Pro using the --mactel option, and while I think GRUB's EFI image has a bug in it since no kernels boot [1], it gave me another idea.
I thought it would be pretty cool if we could synchronize the GPT and MSDOS partition tables (complete with MBR), similar to how it's used in BootCamp/rEFIt so that any given live media could be used with EFI or with BIOS. I've packaged rEFIt [2] (which includes gptsync) but since there's only one partition on the Live USB media, gptsync immediately says "nothing to do" and exits. One way around this would be to use two-partitions on the live media, but that would troublesome since Windows will only recognize the first readable partition. The other way would be to patch gptsync and have it translate the EFI partition to "FAT32" for the MSDOS partition table.
I'd like to hear your comments/feedback on this - Do you think it's a feature worth spending time on? Any other ideas for workarounds?
Stewart
[1] https://bugzilla.redhat.com/show_bug.cgi?id=466954 [2] http://downloads.diffingo.com/refit-0.11-1.fc9.src.rpm
On Thu, 2008-10-16 at 22:44 -0400, Stewart Adam wrote:
I thought it would be pretty cool if we could synchronize the GPT and MSDOS partition tables (complete with MBR), similar to how it's used in BootCamp/rEFIt so that any given live media could be used with EFI or with BIOS. I've packaged rEFIt [2] (which includes gptsync) but since there's only one partition on the Live USB media, gptsync immediately says "nothing to do" and exits. One way around this would be to use two-partitions on the live media, but that would troublesome since Windows will only recognize the first readable partition. The other way would be to patch gptsync and have it translate the EFI partition to "FAT32" for the MSDOS partition table.
First for the obvious question -- did you change the partition table on the USB stick you're testing with to be a GPT partition table? And then had one partition inside that?
You can't change the partition type of the GPT partition to anything other than 'ee' in the MSDOS partition table if you want to keep EFI happy with it.
Multiple partitions might be a workaround, although, yeah, it would kind of suck. Probably worth trying I guess.
Jeremy
Jeremy Katz wrote:
First for the obvious question -- did you change the partition table on the USB stick you're testing with to be a GPT partition table? And then had one partition inside that?
You can't change the partition type of the GPT partition to anything other than 'ee' in the MSDOS partition table if you want to keep EFI happy with it.
Multiple partitions might be a workaround, although, yeah, it would kind of suck. Probably worth trying I guess.
Jeremy
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
For the tests I just created a new stick with --mactel (it told me it would erase all data and make a GPT table for me) and from there I just ran gptsync which said "nothing to do". So essentially, it's still an unmodified USB stick...
Stewart
On Fri, 2008-10-17 at 08:00 -0400, Stewart Adam wrote:
Jeremy Katz wrote:
First for the obvious question -- did you change the partition table on the USB stick you're testing with to be a GPT partition table? And then had one partition inside that?
You can't change the partition type of the GPT partition to anything other than 'ee' in the MSDOS partition table if you want to keep EFI happy with it.
Multiple partitions might be a workaround, although, yeah, it would kind of suck. Probably worth trying I guess.
For the tests I just created a new stick with --mactel (it told me it would erase all data and make a GPT table for me) and from there I just ran gptsync which said "nothing to do". So essentially, it's still an unmodified USB stick...
With a quick look at the gptsync code, this might actually be a bug in gptsync -- it looks like it only does the sync if the number of partitions doesn't match. So getting rid of that check and making it unconditionally sync might help
Jeremy
Jeremy Katz wrote:
On Fri, 2008-10-17 at 08:00 -0400, Stewart Adam wrote:
Jeremy Katz wrote:
First for the obvious question -- did you change the partition table on the USB stick you're testing with to be a GPT partition table? And then had one partition inside that?
You can't change the partition type of the GPT partition to anything other than 'ee' in the MSDOS partition table if you want to keep EFI happy with it.
Multiple partitions might be a workaround, although, yeah, it would kind of suck. Probably worth trying I guess.
For the tests I just created a new stick with --mactel (it told me it would erase all data and make a GPT table for me) and from there I just ran gptsync which said "nothing to do". So essentially, it's still an unmodified USB stick...
With a quick look at the gptsync code, this might actually be a bug in gptsync -- it looks like it only does the sync if the number of partitions doesn't match. So getting rid of that check and making it unconditionally sync might help
Jeremy
Hm, I tried again and I got my errors mixed up. It actually says that there's no data partition (only the EFI system one) so there's no need to sync as the EFI system partition is always marked EFI Protective in the MBR. If it isn't, then installing anything to the MBR risks erasing the entire GPT data structure...
So the after all the only option seems to be to use multiple partitions, which I don't think will work well. I'm still going to try a few things though... If GRUB can install to the MBR on the "EFI Protective" partition and it realizes that "EFI Protective" is just FAT32, things might work out on a single partition.
Stewart
Hi,
Here's the patch that I spoke about in my last email: http://downloads.diffingo.com/livecd-iso-to-disk.sh-efi.patch
What it changes: * Exit messages when syslinux's MBR files can't be found * Unmounts $USBDEV after creating GPT, as HAL attempts to automount * Allows for hybrid (GPT+MBR) USB keys * Adds --skipcopy to skip copying the live OS * Adds --format option to replace --mactel + --reset-mbr * --reset-mbr will now reset the MBR, GPT or not. * if --format is specified, a valid partition isn't required (only a valid device instead)
Notes worth mentioning: * --format + --mactel = new GPT table, resets PMBR (GPT's Protective MBR) * --format = new MSDOS table, resets MBR * syslinux support isn't complete let, hopefully coming soon * GPT hybrid tables requires /usr/{lib,share}/syslinux/gptmbr.bin which is only in the newest syslinux (not available in the Fedora repos yet).
Hope this is useful!
Stewart
On 10/17/08 10:52 AM, Stewart Adam wrote:
Hm, I tried again and I got my errors mixed up. It actually says that there's no data partition (only the EFI system one) so there's no need to sync as the EFI system partition is always marked EFI Protective in the MBR. If it isn't, then installing anything to the MBR risks erasing the entire GPT data structure...
So the after all the only option seems to be to use multiple partitions, which I don't think will work well. I'm still going to try a few things though... If GRUB can install to the MBR on the "EFI Protective" partition and it realizes that "EFI Protective" is just FAT32, things might work out on a single partition.
Stewart
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
On 10/29/08 6:18 PM, Stewart Adam wrote:
- Adds --skipcopy to skip copying the live OS
Oops, forgot to mention two things. I know this feature seems a little odd, so I wanted to explain a bit more... --skipcopy doesn't skip all copying, just the live OS which makes it useful for testing the boot configuration. It's also useful for USB media, where testing over and over would normally be wasting limited write cycles it takes a lot of waiting for the 700MB file copy finish. ALso, say the MBR gets lost or you've transferred the live OS files from a friend's key, you can just refresh the MBR and bootloader config instead without touching the live OS and persistant home.
- syslinux support isn't complete let, hopefully coming soon
For this I mean GPT+MBR syslinux support, that made it sound like the patch broke the syslinux boot ;)
Stewart
On Wed, 2008-10-29 at 18:18 -0400, Stewart Adam wrote:
Here's the patch that I spoke about in my last email: http://downloads.diffingo.com/livecd-iso-to-disk.sh-efi.patch
Cool. From a quick look, it looks okay. But given where we are in the Fedora 10 cycle, I'm going to hold off on applying it until after we get Fedora 10 out the door to avoid destabilizing things for people (especially as feature freeze was a month or so ago)
Jeremy
On 10/29/08 8:38 PM, Jeremy Katz wrote:
On Wed, 2008-10-29 at 18:18 -0400, Stewart Adam wrote:
Here's the patch that I spoke about in my last email: http://downloads.diffingo.com/livecd-iso-to-disk.sh-efi.patch
Cool. From a quick look, it looks okay. But given where we are in the Fedora 10 cycle, I'm going to hold off on applying it until after we get Fedora 10 out the door to avoid destabilizing things for people (especially as feature freeze was a month or so ago)
Jeremy
No problem, that gives us some more time to work out the GPT+syslinux problems.
Stewart
On Wednesday, October 29 2008, Stewart Adam said:
On 10/29/08 8:38 PM, Jeremy Katz wrote:
On Wed, 2008-10-29 at 18:18 -0400, Stewart Adam wrote:
Here's the patch that I spoke about in my last email: http://downloads.diffingo.com/livecd-iso-to-disk.sh-efi.patch
Cool. From a quick look, it looks okay. But given where we are in the Fedora 10 cycle, I'm going to hold off on applying it until after we get Fedora 10 out the door to avoid destabilizing things for people (especially as feature freeze was a month or so ago)
No problem, that gives us some more time to work out the GPT+syslinux problems.
Okay, as I come back to looking at this, the patch still looks okay and I've even given it a quick test now :-) Have you had any luck/progress on GPT+syslinux before I try to look at it some more?
On 1/16/09 4:09 PM, Jeremy Katz wrote:
On Wednesday, October 29 2008, Stewart Adam said:
On 10/29/08 8:38 PM, Jeremy Katz wrote:
On Wed, 2008-10-29 at 18:18 -0400, Stewart Adam wrote:
Here's the patch that I spoke about in my last email: http://downloads.diffingo.com/livecd-iso-to-disk.sh-efi.patch
Cool. From a quick look, it looks okay. But given where we are in the Fedora 10 cycle, I'm going to hold off on applying it until after we get Fedora 10 out the door to avoid destabilizing things for people (especially as feature freeze was a month or so ago)
No problem, that gives us some more time to work out the GPT+syslinux problems.
Okay, as I come back to looking at this, the patch still looks okay and I've even given it a quick test now :-) Have you had any luck/progress on GPT+syslinux before I try to look at it some more?
There's been some good progress - I sent a few messages to the syslinux mailing lists and hpa plans on fixing the GPT MBR in the near future, so we have that part of the problem covered. For now, I guess we can just comment out the "if" that chooses between mbr.bin or gptmbr.bin so that the USB key would be only EFI or only MBR compatible until this is resolved.
The other half of the problem is that recent kernels hang when booting from EFI (bug 466954). Peter Jones was able to find out which commit broke EFI booting, but we're not sure how to fix it yet.
Stewart
On Saturday, January 17 2009, Stewart Adam said:
On 1/16/09 4:09 PM, Jeremy Katz wrote:
On Wednesday, October 29 2008, Stewart Adam said:
On 10/29/08 8:38 PM, Jeremy Katz wrote:
On Wed, 2008-10-29 at 18:18 -0400, Stewart Adam wrote:
Here's the patch that I spoke about in my last email: http://downloads.diffingo.com/livecd-iso-to-disk.sh-efi.patch
Cool. From a quick look, it looks okay. But given where we are in the Fedora 10 cycle, I'm going to hold off on applying it until after we get Fedora 10 out the door to avoid destabilizing things for people (especially as feature freeze was a month or so ago)
No problem, that gives us some more time to work out the GPT+syslinux problems.
Okay, as I come back to looking at this, the patch still looks okay and I've even given it a quick test now :-) Have you had any luck/progress on GPT+syslinux before I try to look at it some more?
There's been some good progress - I sent a few messages to the syslinux mailing lists and hpa plans on fixing the GPT MBR in the near future, so we have that part of the problem covered. For now, I guess we can just comment out the "if" that chooses between mbr.bin or gptmbr.bin so that the USB key would be only EFI or only MBR compatible until this is resolved.
Given that we didn't previously copy over an MBR for the mactel case, leaving the 'if' doesn't really seem to hurt anything. I get to EFI grub with the gptmbr.bin present and then when it's fixed, maybe things will magically just start to work. So based on that, I think I'm going to push what you had to git and then we can have it more easily testable
The other half of the problem is that recent kernels hang when booting from EFI (bug 466954). Peter Jones was able to find out which commit broke EFI booting, but we're not sure how to fix it yet.
Yeah, I've been following Peter's sob story of debugging that. He's getting closer. I think.
Jeremy
On 1/19/09 10:55 AM, Jeremy Katz wrote:
On Saturday, January 17 2009, Stewart Adam said:
There's been some good progress - I sent a few messages to the syslinux mailing lists and hpa plans on fixing the GPT MBR in the near future, so we have that part of the problem covered. For now, I guess we can just comment out the "if" that chooses between mbr.bin or gptmbr.bin so that the USB key would be only EFI or only MBR compatible until this is resolved.
Given that we didn't previously copy over an MBR for the mactel case, leaving the 'if' doesn't really seem to hurt anything. I get to EFI grub with the gptmbr.bin present and then when it's fixed, maybe things will magically just start to work. So based on that, I think I'm going to push what you had to git and then we can have it more easily testable
Good news - syslinux's gptmbr.bin is working fine. I was using the partition type UUID, not its unique UUID. With this correction, things work fine when booting with BIOS or EFI!
I've attached a patch which re-adds the check for gptmbr.bin and if it's found, then the script will now automatically grab the partition's unique UUID and write a compatible MBR. It should work on all drives with an LBA size of 512 bytes.
The patch also includes a minor fix: If --format is specified and the user has accepted the first warning by hitting <enter>, then it will automatically unmount all partitions on the device so that the parted commands don't fail.
Stewart
On Monday, January 26 2009, Stewart Adam said:
I've attached a patch which re-adds the check for gptmbr.bin and if it's found, then the script will now automatically grab the partition's unique UUID and write a compatible MBR. It should work on all drives with an LBA size of 512 bytes.
The patch also includes a minor fix: If --format is specified and the user has accepted the first warning by hitting <enter>, then it will automatically unmount all partitions on the device so that the parted commands don't fail.
Applied, thanks!
Jeremy
On 1/26/09 9:52 PM, Jeremy Katz wrote:
Applied, thanks!
Jeremy
Thanks for commiting it so fast! I've been testing the two configurations all night, and funnily enough the gpt+mbr setup works better than the mbr setup alone on my home machine: If I don't specify --mactel, I can boot in QEMU fine but my machine (GA-965P-S3 mobo) refuses to boot it, saying that it can't find the kernel image for "linux".
There's also a small bug that I forgot to account for in the last patch - $BOOTCONFIG is overwritten if --mactel is set, so the syslinux.cfg file is left in an inconsistent state with root=CDLABEL=<...> and rootfstype=iso9660. The attached patch fixes this problem by using $BOOTCONFIG and $BOOTCONFIG_EFI.
Stewart
Hi,
I've been testing out creating using a hybrid GPT+MBR stick so that EFI and BIOS-based systems can boot off of the one stick and share a persistent home. The good news is that it works and the changes are fairly simple - essentially, I just removed the conditional statements that prevented the non-mactel stuff from being executed when --mactell was passed. I also had to rearrange a few blocks so that the GPT partitions happens first, since the MBR should only be installed after GPT has initialized (and therefore it will install to GPT's protective MBR).
The bad news is syslinux doesn't support GPT at the moment because we can't set a MBR partition as "bootable" in a GPT partition table. GRUB doesn't seem to care about this and will boot fine once installed to /dev/sdb, although the only bug I hit then was that booting from "USB-HDD" and "USB-ZIP" in the BIOS didn't work... I had to select "HDD" and then choose "USB-0". Is this why the scripts currently use syslinux instead of making the jump to GRUB?
Even if we only use GRUB as a last resort (when --mactel is passed), I still think this is a worthfile feature most recent motherboards support booting from USB as a HDD instead of USB-ZIP.
Stewart
On Mon, 2008-10-27 at 23:03 -0400, Stewart Adam wrote:
I've been testing out creating using a hybrid GPT+MBR stick so that EFI and BIOS-based systems can boot off of the one stick and share a persistent home. The good news is that it works and the changes are fairly simple - essentially, I just removed the conditional statements that prevented the non-mactel stuff from being executed when --mactell was passed. I also had to rearrange a few blocks so that the GPT partitions happens first, since the MBR should only be installed after GPT has initialized (and therefore it will install to GPT's protective MBR).
Cool!
The bad news is syslinux doesn't support GPT at the moment because we can't set a MBR partition as "bootable" in a GPT partition table. GRUB doesn't seem to care about this and will boot fine once installed to /dev/sdb, although the only bug I hit then was that booting from "USB-HDD" and "USB-ZIP" in the BIOS didn't work... I had to select "HDD" and then choose "USB-0". Is this why the scripts currently use syslinux instead of making the jump to GRUB?
The biggest reason is that we use isolinux for CDs (because grub's cd booting code is incompatible with a fair number of machines) and using the same bootloader for both live image over CD and USB is a good thing.
That said, syslinux definitely has better compatibility for booting off of various USB things since hpa has spent a lot of time dealing with BIOS bugs ;) It might be worth sending a mail to the syslinux list and see if hpa has any thoughts... he tends to be up for making syslinux as usable as possible for everywhere.
Even if we only use GRUB as a last resort (when --mactel is passed), I still think this is a worthfile feature most recent motherboards support booting from USB as a HDD instead of USB-ZIP.
Yeah, if we have to, I guess we could fall back to this. But probably at this point, it's too late for Fedora 10
Jeremy
livecd@lists.fedoraproject.org
-
Jeremy Katz -
Stewart Adam