Status of armhfp grubs2-efi on top of Uboot

[Mark Verlinde]

Hi,

A while ago (around the freeze of FC29) grub2-efi on top of Uboot was proposed as the
(potentially) default setup for armhfp too. Unfortunately (IMHO) it did not materialize
and I'm searching for the game-breaker.

On buggzilla found a few hints,  " we have at least a grub2 <-> kernel
bug" (1) among them, they did not make the holdup clear to me.

My investigation so-far:
* It works pretty good with a close to upstream build grub, not with the default gub2
packages. Which makes me believe loading an arm32 kernel as an Portable Executable instead
of grub's LoadImage() and StartImage() services may be a game-stopper. NOTE : I
understand this is necessary for chain loading efi stubs for secure-boot. IMO this is not
the pursued  goal here.

* Since kernel patch "efi: libstub/arm: account for firmware reserved memory at the
base of RAM" (2) it works on RPI's too (given a "custom" build grub2)

All feedback is appreciated,
grtz Mark  

1) https://bugzilla.redhat.com/show_bug.cgi?id=1602948
2) https://patchwork.kernel.org/patch/11189097/

[Gerd Hoffmann]

On Thu, Jun 25, 2020 at 04:11:05AM -0000, Mark Verlinde wrote:

>  A while ago (around the freeze of FC29) grub2-efi on top of Uboot
> was
>  proposed as the (potentially) default setup for armhfp too.
>  Unfortunately (IMHO) it did not materialize and I'm searching for the
>  game-breaker. 
Well.  It's a mess with the huge stack of non-upstream grub2 patches
Fedora has.

>  My investigation so-far: * It works pretty good with a close to
>  upstream build grub, not with the default gub2 packages. Which makes
>  me believe loading an arm32 kernel as an Portable Executable instead
>  of grub's LoadImage() and StartImage() services may be a game-stopper.
>  NOTE : I understand this is necessary for chain loading efi stubs for
>  secure-boot. IMO this is not the pursued  goal here. 
Looked at this a while ago, for booting armhfp guests in kvm.  IIRC the
state of affairs is:  grub2 can boot armhfp kernels like aarch64
kernels, using the efi stub.  upstream grub2 does this and it works
(2.03+ I think).  Upstream kernel had bugs in efi stub boot path the
past but they are long fixed.

Dunno what Fedora grub2 is doing here.  It loads the kernel successfully
but then the kernel hangs.  It claims to be 2.04 so it should be new
enough for this to work.  Just going compile vanilla upstream grub and
use that unfortunately doesn't work that easily any more due to
non-upstream BLS support fedora grub2 has.  Also such a setup tends to
break on updates, when a new grub2.rpm lands.

I don't feel like wading through the fedora grub2 patches, trying to
find the broken one.  systemd-boot isn't available for armhfp either.
So I gave up building on EFI (i.e. edk2 + grub2) for my VMs.

I'm using a patched[1] qemu uboot image as firmware now & attach the
fedora server image as sata disk.  Done.  Probably not the answer you
where looking for ...

take care,
  Gerd

[1] 3-liner to turn off flash.  qemu flash emulation seems to work only
    with tcg but not with kvm ...

[Mark Verlinde]

Thank you (@Gerd)

>  Just going compile vanilla upstream grub and
>  use that unfortunately doesn't work that easily any more due to
>  non-upstream BLS support fedora grub2 has.   
There is still the grubby-deprecated package to fallback too..

>  Also such a setup tends to break on updates, when a new grub2.rpm
> lands. 
Yea, tent to epoch my "custom" builds, ATM it is epoch 2 for a custom grub2

>  I don't feel like wading through the fedora grub2 patches, trying
> to
>  find the broken one.  systemd-boot isn't available for armhfp either.
>  So I gave up building on EFI (i.e. edk2 + grub2) for my VMs. 
Still doing this, also for el7 where grub2 is totally absent for armhfp.

>  I'm using a patched[1] qemu uboot image as firmware now &
> attach the
>  fedora server image as sata disk.  Done.  Probably not the answer you
>  where looking for ... 
 
Well this actually is one the use-cases my looking for, get rid of those
"external" kernels in armhfp VM's

Grtz Mark

[Peter Robinson]

Hey,

Sorry about the late reply on this.

>  A while ago (around the freeze of FC29) grub2-efi on top of Uboot was
> proposed as the (potentially) default setup for armhfp too. Unfortunately (IMHO) it did
> not materialize and I'm searching for the game-breaker.
>
>  On buggzilla found a few hints,  " we have at least a grub2 <-> kernel
> bug" (1) among them, they did not make the holdup clear to me.
>
>  My investigation so-far:
>  * It works pretty good with a close to upstream build grub, not with the default gub2
> packages. Which makes me believe loading an arm32 kernel as an Portable Executable instead
> of grub's LoadImage() and StartImage() services may be a game-stopper. NOTE : I
> understand this is necessary for chain loading efi stubs for secure-boot. IMO this is not
> the pursued  goal here.
>
>  * Since kernel patch "efi: libstub/arm: account for firmware reserved memory at the
> base of RAM" (2) it works on RPI's too (given a "custom" build grub2)
So I finally got around to spending some time on this last week and
closed out the last of the issues I had so it's now working, albeit
with very minimal testing. We currently have images building for
Minimal/Server/Workstation and I've booted Minimal on a couple of
devices using U-Boot/UEFI/grub2/kernel without issues.

There's nightly builds of F33 and rawhide:
https://kojipkgs.fedoraproject.org/compose/branched/latest-Fedora-33/comp...
https://kojipkgs.fedoraproject.org/compose/branched/latest-Fedora-33/comp...
https://kojipkgs.fedoraproject.org/compose/branched/latest-Fedora-33/comp...

The UEFI enabled images are the ones with the arch at the end rather
than in the middle, eg: Fedora-Minimal-33-20201010.n.0.armhfp.raw.xz

[Gerd Hoffmann]

  Hi,

>  So I finally got around to spending some time on this last week and
>  closed out the last of the issues I had so it's now working, albeit
>  with very minimal testing. We currently have images building for
>  Minimal/Server/Workstation and I've booted Minimal on a couple of
>  devices using U-Boot/UEFI/grub2/kernel without issues. 
Tried that now.  F33 finally works flawlessly with uefi in armv7 virtual
machines.  Great.

take care,
  Gerd