Ross Smith wrote:
I don't think it will be an issue. I've already been bitten by this with a ~1300 efi booted desktops and would welcome the updated configuration.
Speaking of which ...
Q: Shouldn't we be using 260MiB FAT32 as a minimum EFI System Partition (ESP) size going forward, especially as 4KiB logical blocks become regular?
o Just some history (detail) ...
- For NT6.0 (Vista), Microsoft recommended (and its installers created) a 200MiB FAT32 ESP - This size has seemingly "stuck" with most distros, even though ... - For NT6.1+ (7+/2008+), Microsoft recommended (and its installers created) a 100MiB FAT32 ESP However ... - If the logical block size is 4KiB, the smallest FAT32 file system is 260MiB Although ... - The ESP can be FAT16 (just like good'ole ARC) instead of FAT32 - Every ESP I've used, including Windows 7 and 8 x64, seems to install fine on a FAT32 ESP
Hence why I'm bringing it up. Just curious if there has been much discussion on this. I did some google searches on the archives and couldn't find anything.
-- bjs
-- Bryan J Smith - http://www.linkedin.com/in/bjsmith E-mail: b.j.smith at ieee.org or me at bjsmith.me
Sorry, important typo there ...
Bryan Smith b.j.smith@ieee.org wrote:
... However ...
- If the logical block size is 4KiB, the smallest FAT32 file system is 260MiB
Although ...
- The ESP can be FAT16 (just like good'ole ARC) instead of FAT32
- Every ESP I've used, including Windows 7 and 8 x64, seems to install
fine on a FAT16 ESP
^^^^^^ Corrected (not FAT32)
-- bjs
On Fri, Apr 15, 2016 at 6:16 PM, Bryan Smith b.j.smith@ieee.org wrote:
Ross Smith wrote:
I don't think it will be an issue. I've already been bitten by this with a ~1300 efi booted desktops and would welcome the updated configuration.
Speaking of which ...
Q: Shouldn't we be using 260MiB FAT32 as a minimum EFI System Partition (ESP) size going forward, especially as 4KiB logical blocks become regular?
Well, mkdosfs defaults to FAT16 and 4K cluster size on a 200M partition, so that should continue to work, even though the UEFI spec says to use FAT32 on system partitions. So chances are doing nothing isn't going to break anything.
In practice, both Windows and OS X use FAT32, but they also use cluster sizes smaller than 4K. Presumably both of them will transition to 256+M EFI system partitions, rather than use FAT16.
o Just some history (detail) ...
- For NT6.0 (Vista), Microsoft recommended (and its installers
created) a 200MiB FAT32 ESP
- This size has seemingly "stuck" with most distros, even though ...
- For NT6.1+ (7+/2008+), Microsoft recommended (and its installers
created) a 100MiB FAT32 ESP However ...
- If the logical block size is 4KiB, the smallest FAT32 file system is 260MiB
Although ...
- The ESP can be FAT16 (just like good'ole ARC) instead of FAT32
- Every ESP I've used, including Windows 7 and 8 x64, seems to install
fine on a FAT32 ESP
Hence why I'm bringing it up. Just curious if there has been much discussion on this. I did some google searches on the archives and couldn't find anything.
https://bugzilla.redhat.com/show_bug.cgi?id=1046577 https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/ https://www.freedesktop.org/wiki/MatthewGarrett/BootLoaderSpec/ and maybe https://www.redhat.com/archives/anaconda-devel-list/2014-July/msg00002.html
Ideally there'd be an -E flag for mkdosfs to deal with this. The UEFI spec says firmware is expected to support FAT32, FAT16, and FAT12 so it probably doesn't matter that the spec also says the system partition (non-removable) should be FAT32, where only removables get FAT16 or FAT12. The other thing -E would make possible is ensuring the proper invariant "EFI file system" is what's actually being created, rather than pure FAT.
Chris Murphy wrote:
Well, mkdosfs defaults to FAT16 and 4K cluster size on a 200M partition, so that should continue to work, even though the UEFI spec says to use FAT32 on system partitions. So chances are doing nothing isn't going to break anything.
Understood. And I've even verified at least the Windows 7 installer seems to use a FAT16 just fine, while Windows 8 will still boot from it (didn't try installing to one with Windows 8).
I haven't had much time with Windows 10.
In practice, both Windows and OS X use FAT32, but they also use cluster sizes smaller than 4K. Presumably both of them will transition to 256+M EFI system partitions, rather than use FAT16.
My apologies if I wasn't clear (I should have used the term "sector") ...
A storage device, of 4KiB [logical] _sector_ (correcting my prior, inappropriate use of "block") _forces_ a 4KiB+ block/cluster size, so 260MiB now becomes the minimum FAT32. We're now seeing some systems where new Windows 10 systems use 4KiB logical sector size.
https://bugzilla.redhat.com/show_bug.cgi?id=1046577 https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/ https://www.freedesktop.org/wiki/MatthewGarrett/BootLoaderSpec/ and maybe https://www.redhat.com/archives/anaconda-devel-list/2014-July/msg00002.html
Okay, I see you there. Thanx for pointing this out! ;)
Ideally there'd be an -E flag for mkdosfs to deal with this.
Indeed, that would take it out of the hands of Anaconda et al., and put it in the tool. That way the tool can be changed, and everything downstream using it modified.
The UEFI spec says firmware is expected to support FAT32, FAT16, and FAT12 so it probably doesn't matter that the spec also says the system partition (non-removable) should be FAT32, where only removables get FAT16 or FAT12. The other thing -E would make possible is ensuring the proper invariant "EFI file system" is what's actually being created, rather than pure FAT.
I like Garrett's comments. ;)
If there is a spec that says one thing for one, another thing for another, we might run into a case of such implementation, and suddenly an uEFI implementation that doesn't like FAT16 for an ESP.
Murphy's Law.
How Anaconda deals with this possible issue, I don't know. But given today's disk sizes, and how 4KiB physical is common, while 512B logical only remains because BOOTMGR was ignorant originally (let alone NTLDR has always been), I'm not sure we're doing the right thing, long-term.
-- bjs
<off-topic> In any case, and I don't expect Anaconda to be the place where this "greater issue" ever gets addressed ...
But with Windows x64 version 7, 8 and 10 OEM guides recommending the #1 and #2 partitions to be the ESP + Reserved partitions (ignoring the separate "Tools" partition in 8 and 10), I've pretty much solved it by using the first 0.5GiB as follows ...
1 - 384MiB (383MiB), EF00h, EFI System Partition (esp) 384 - 512MiB (128MiB), 0C01h, Microsoft Reserved (msftres)
With regards to the ESP, using an 383MiB (starting at 1MiB, sector 2048 for 512B, 256 for 4KiB) solves the problem altogether. There shouldn't ever be any issue. I can also load up a shell, tools and other things, which would be cool for a distro to offer (assuming they can be distributed. I'm even partial to rEFInd, which I'd love to see a distro ship as an option, because it solves so many multi-boot issues, but that's another story.
With regards to the MS Reserved, if I do a Google research, everyone keeps assuming the 2nd is unused, yet I've done some hexdumps and found bytes _do_ go in there, especially if Windows is installed on the same system (still trying to find a case where just reading the disk in Windows does similarly). I'd also love to see the option to create this as the #2 partition in a new GPT disk label, in an installer, if the system is also going to be multibooting with Windows x64.
But again, I understand this is really outside the realm of Anaconda. </off-topic>
-- Bryan J Smith - http://www.linkedin.com/in/bjsmith E-mail: b.j.smith at ieee.org or me at bjsmith.me
On Fri, Apr 15, 2016 at 08:16:19PM -0400, Bryan Smith wrote:
Ross Smith wrote:
I don't think it will be an issue. I've already been bitten by this with a ~1300 efi booted desktops and would welcome the updated configuration.
Speaking of which ...
Q: Shouldn't we be using 260MiB FAT32 as a minimum EFI System Partition (ESP) size going forward, especially as 4KiB logical blocks become regular?
It's probably worth filing a bug about.
Brian C. Lane wrote:
It's probably worth filing a bug about.
Yeah, I might file a bug that is a bit "more broad," especially after Chris Murphy pointed out his [bz#1046577].
E.g., in addition to 1046577#c3 from Adam on the spec, even #c1 from Rod Smith (of rEFInd et al. fame) pointed out that some uEFI implementations take issue with various FAT32 ESP sizes that aren't at least 512MiB too, well beyond the 4KiB logical sector requirements of 260MiB.
[ SIDE NOTE: I think this was because of an early Vista requirement where Microsoft expected the recovery "Tools" to be self-contained in the FAT32 formatted ESP. As most PC OEMs use tools that don't boot/run well in a vFAT file system, they changed it to a separate NTFS file system with Windows 7. ]
So ... where do we "draw the line" on what situations Anaconda should deal with? And how much development should I expect to sack you guys with?
Now add in my totally non-Linux comment that in multi-boot configuration, where Windows is expected, Microsoft expects an 0C01h (msftres) partition, and ideally (per OEM guides for Windows x64 version 7, 8 and now 10), it should be after the ESP (#1 and #2, or #2 and #3 if there is a "Tools" in #1).
Yeah, this we'll get out-of-control if we try to accommodate every situation, every uEFI misdesign, the uEFI spec and even multi-boot with non-Linux.
So ... might I have a suggestion? And if you like it, I'll file the RFE myself, with the justification.
How about we solve this with a checkbox ... or two?
Suggested Checkbox One: [ ] Other OSes (including Windows x64) will also be installed Conditions ... (i.e., don't show checkbox unless met) - Native 64-bit uEFI Storage Services - Drive is being wiped w/EFI System Partition (ESP) created Results ... - Part 1: 1-384MiB EF00h (esp) FAT32 = 383MiB (>260MiB) - Part 2: 384-512MiB 0C01h (msftres) = 128MiB
Suggested Checkbox Two: [ ] Create a large EFI System Partition (ESP) Conditions ... (i.e., don't show checkbox unless met) - Native 64-bit uEFI Storage Services - Drive is being wiped w/EFI System Partition (ESP) created Results ... - Part 1: 1-640MiB EF00h (esp) FAT32 = 639MiB (>512MiB) And if first checkbox selected, also ... - Part 2: 640-768MiB 0C01h (msftres) = 128MiB
The "Checkbox Two" might be going overboard. I can leave that out of the RFE.
But "Checkbox One" would be very much appreciated for those of us who regularly install Windows x64 version 7, 8 and 10 in Native 64-bit uEFI Storage Services mode.
-- bjs
[bz#1046577] https://bugzilla.redhat.com/show_bug.cgi?id=1046577
-- Bryan J Smith - http://www.linkedin.com/in/bjsmith E-mail: b.j.smith at ieee.org or me at bjsmith.me
On Mon, Apr 18, 2016 at 12:28 PM, Bryan Smith b.j.smith@ieee.org wrote:
Brian C. Lane wrote:
It's probably worth filing a bug about.
Yeah, I might file a bug that is a bit "more broad," especially after Chris Murphy pointed out his [bz#1046577].
E.g., in addition to 1046577#c3 from Adam on the spec, even #c1 from Rod Smith (of rEFInd et al. fame) pointed out that some uEFI implementations take issue with various FAT32 ESP sizes that aren't at least 512MiB too, well beyond the 4KiB logical sector requirements of 260MiB.
I'd find the missing "EFI file system" document, or confirm it doesn't exist, before planning work. The UEFI spec uses imprecise language, it doesn't say "FAT32 must be used on internal drives" it says it should be used on system partitions. Well all of these things are system partitions. Later it says the firmware should support all three FAT variants, so does it really matter if it's all mixed and matched?
So ... where do we "draw the line" on what situations Anaconda should deal with? And how much development should I expect to sack you guys with?
Maybe mkdosfs -E just makes the proper "EFI file system" instead of pure FAT, whatever that difference is. And still takes options from something else. That something else could be storaged? I can predict people will break these ESPs beyond repair, they're mounted persistently all the time on Linux and if there's a crash they're really not crash tolerant for some time (minute? minutes?) after the last modification.
Now add in my totally non-Linux comment that in multi-boot configuration, where Windows is expected, Microsoft expects an 0C01h (msftres) partition, and ideally (per OEM guides for Windows x64 version 7, 8 and now 10), it should be after the ESP (#1 and #2, or #2 and #3 if there is a "Tools" in #1).
Yeah, this we'll get out-of-control if we try to accommodate every situation, every uEFI misdesign, the uEFI spec and even multi-boot with non-Linux.
So ... might I have a suggestion? And if you like it, I'll file the RFE myself, with the justification.
How about we solve this with a checkbox ... or two?
Suggested Checkbox One: [ ] Other OSes (including Windows x64) will also be installed Conditions ... (i.e., don't show checkbox unless met)
- Native 64-bit uEFI Storage Services
- Drive is being wiped w/EFI System Partition (ESP) created
Results ...
- Part 1: 1-384MiB EF00h (esp) FAT32 = 383MiB (>260MiB)
- Part 2: 384-512MiB 0C01h (msftres) = 128MiB
Suggested Checkbox Two: [ ] Create a large EFI System Partition (ESP) Conditions ... (i.e., don't show checkbox unless met)
- Native 64-bit uEFI Storage Services
- Drive is being wiped w/EFI System Partition (ESP) created
Results ...
- Part 1: 1-640MiB EF00h (esp) FAT32 = 639MiB (>512MiB)
And if first checkbox selected, also ...
- Part 2: 640-768MiB 0C01h (msftres) = 128MiB
You want to support installing Windows after Fedora? Why? I don't know what "Native 64-bit uEFI Storage Services" means or how that affects the use case.
Chris Murphy wrote:
You want to support installing Windows after Fedora? Why?
No, I merely suggested we consider adding the _option_, with a checkbox, to Anaconda so it creates a GPT disk label that is compatible with Windows x64. In addition to specifying FAT32, along with the aforementioned 4KiB logical sector reality requiring 260MiB, Microsoft also considers the 0C01h non-optional on GPT. So I thought it was a good way to offer something with a checkbox, that's all.
I don't know what "Native 64-bit uEFI Storage Services" means or how that affects the use case.
I just mean native uEFI boot, not uEFI w/CSM which provides Legacy 16-bit BIOS Int13h Disk Services.
-- bjs
On Fri, Apr 22, 2016 at 2:28 AM, Bryan Smith b.j.smith@ieee.org wrote:
Chris Murphy wrote:
You want to support installing Windows after Fedora? Why?
No, I merely suggested we consider adding the _option_, with a checkbox, to Anaconda so it creates a GPT disk label that is compatible with Windows x64.
You don't want it supported, but you want it as an option? I don't know what that means either. If it's there, it needs to be run by UI/Ux folks, needs plain language clarity, needs to be translated, and needs a significantly larger matrix of tests by QA/QE than exists now for Windows installed before Fedora. I don't see on what basis this option goes in if it's not assured of being understandable and working. If it's not proven to work, why have it in there? And proving that it works will take a ton of effort compared to the current way of doing it which expects Windows first.
The other factor is that the vast majority of OEM Windows installers are not Microsoft installers. Microsoft's will install to free space. The OEM installers typically don't. They obliterate the target drive in its entirety in favor of a fixed layout that can't be modified by the user at all.
In addition to specifying FAT32, along with the aforementioned 4KiB logical sector reality requiring 260MiB, Microsoft also considers the 0C01h non-optional on GPT. So I thought it was a good way to offer something with a checkbox, that's all.
It's convincing that the ESP needs to use 4KiB clusters to support 4kn drives. But I'm unconvinced any UEFI spec mandates that the ESP needs to be FAT32. It's suggested that this ought to be true but the language I've seen so far doesn't say it must be true. So either mkdosfs needs a special flag to do the logic ensuring on 4kn drives it uses 4/8KiB clusters, or some component within the installer needs to pass mkdosfs that option, otherwise obviously the system will not boot.
Chris Murphy wrote:
You don't want it supported, but you want it as an option? I don't know what that means either. If it's there, it needs to be run by UI/Ux folks, needs plain language clarity, needs to be translated,
Yes, I understand that.
Again, understand the context ... 1) _Only_ when creating a _new_ GPT disk label (wiping the disk) 2) Come up with as generic of a text as possible, e.g.,
"Format this new GPT disk to maximize compatibility with several non-Linux OSes"
and needs a significantly larger matrix of tests by QA/QE than exists now for Windows installed before Fedora.
Absolutely _not_. Period. The _opposite_. There is *0* guarantees because that OS installs _after_ will work.
All it does is ...
A) Reserve the 128MiB type 0C01h partition (msftres) as partition #2 after the ... B) >=260MiB FAT32 EF00h EFI System Partition (esp) partition #1
That's it! *0* guarantees, just "maximum compatibility."
Again, I recommended 1-384MiB (383MiB) and 384-512MiB (1288MiB), respectively, using the first 0.5GiB of the disk.
I don't see on what basis this option goes in if it's not assured of being understandable and working.
Just because most IT people are ignorant GPT doesn't mean it's not ideal.
I.e., a lot of people think the type 0C01h partition is "optional" and "not used" for GPT when running Windows x64. That's hardly the case. It's also the order Microsoft has, and continues, to recommend.
If it's not proven to work, why have it in there?
What are you talking about?!
This has been Microsoft _own_ recommendations since original NT6.0 Vista x64 release, and continues through today for _all_ x64 releases. The 0C01h (msftres) partition _removes_ the need for hidden sectors and other crap. It's there for a reason.
At most they've changed the "minimum" FAT32 ESP recommendation, and now they are starting to recommend larger because 4KiB sectors requires at least 260MiB -- something that has nothing to do with the OS, but the FAT32 file system itself. We've only avoided it to date because most OSes implement "logical" sector size separate from "physical."
So ... I was recommending we honor
And proving that it works will take a ton of effort compared to the current way of doing it which expects Windows first. The other factor is that the vast majority of OEM Windows installers are not Microsoft installers. Microsoft's will install to free space. The OEM installers typically don't. They obliterate the target drive in its entirety in favor of a fixed layout that can't be modified by the user at all.
Obviously you haven't heard a word I said.
Of course OEMs pre-install Windows, and then people install Linux after. That's _not_ who I'm attempting to address.
I'm talking about professionals, corporations and others who install their own OSes, the ones with Microsoft and Red Hat Enterprise agreements, as well as enthusiasts at home or in development enviroments.
Last time I checked, that's a lot of workstations, as many do _not_ come pre-installed. We're talking a lot of the higher-end systems here. I'm not talking Jane or Joe user who does it the way you said.
I already and purposely pre-create GPT disk labels for this reason, and I'm _not_ the only one. We create a disk label with maximum compatibility, ensuring we avoid issues.
It's convincing that the ESP needs to use 4KiB clusters to support 4kn drives. But I'm unconvinced any UEFI spec mandates that the ESP needs to be FAT32.
So ... why not give the user the option of FAT32 without confusing them?
"Format this new GPT disk to maximize compatibility with several non-Linux OSes"
We don't have to go into all the asterisks and nooks'n cranies. One checkbox makes ... A) FAT32 ESP of at least 260MiB B) Reserves the 128MiB type 0C01h partition
Done. One option, so many potential issues avoided for a *new* ... again ... *new* GPT disk label.
It's suggested that this ought to be true but the language I've seen so far doesn't say it must be true.
And I'm trying to get you to understand that this is a very, very valid option for people who _wipe_ their disks and create a _new_ GPT disk label. Something we can do in Anaconda.
So either mkdosfs needs a special flag to do the logic ensuring on 4kn drives it uses 4/8KiB clusters, or some component within the installer needs to pass mkdosfs that option, otherwise obviously the system will not boot.
Again, I thought the "way forward" was to look at this was to address several "realities" of modern uEFI (w/o CSM) multi-booting. That includes looking at the fact that it goes beyond just the ESP too.
One checkbox, so many issues avoided ... But _only_ when the installer is creating a _new_ GPT disk label.
-- bjs
P.S. Please, please understand that I'm just trying to avoid a lot of issues that many people run into, with a simple checkbox when someone installs Linux on a new or wiped disk (not an existing one).
-- Bryan J Smith - http://www.linkedin.com/in/bjsmith E-mail: b.j.smith at ieee.org or me at bjsmith.me
anaconda-devel@lists.fedoraproject.org
-
Brian C. Lane -
Bryan Smith -
Chris Murphy