livecd-iso-to-disk & editliveos: Expand partitioning and filesystem support

[Frederick Grose]

See https://github.com/livecd-tools/livecd-tools/pull/176.

A cumulative update, best reviewed by testing.

Enable formatting of a separate EFI System Partition and Apple HFS+ boot
  partition based on the files efiboot.img and macboot.img in the /images
  directory of the source.  This means EFI booting is possible with overlay
  sizes as large as the free space on the primary partition.
Support F2FS type formatting for the device root, home, or overlay
  filesystems.
Provide boot support for F2FS-formatted discs with an EFI Boot Stub loader
  from dracut, as GRUB support for F2FS is faulty at this time.
Allow conversion of the root filesystem type between ext[234], btrfs, xfs,
  & F2FS and their available block sizes with editliveos.
Allow a Device-mapper target to be the target device for installation.
  This would enable concurrent multi device installation through a
  preconfigured Device-mapper mirror target.

Also,
  - introduce the --noesp option to --format to avoid the extra partitions;
  - on UEFI booted hosts, offer to create a UEFI boot entry for F2FS-based
    installations;
  - do not require SYSLINUX-EXTLINUX on the installation host;
  - take greater advantage of BASH case statements for code structure;
  - use fallocate on filesystem creation to reserve storage space for
    embedded filesystems;
  - add a trap ERR line for error reporting in livecd-iso-to-disk;
  - provide better support for isoscan/filename boots from .iso files;
  - delete the uncompressed image file just after compression to make
    more space available, if preservation of temporary files is not selected;
  - use bind mounts for /run, /sys, & /dev in liveimage-mount;
  - update documentation.

ref: ... as GRUB support for F2FS is faulty at this time.

I found that the following command run against a livecd-iso-to-disk
installed LiveUSB,

sudo grub2-install -v --boot-directory=/run/media/
\liveuser/ESP/EFI/BOOT --compress=xz --modules="part_gpt part_msdos
f2fs" --efi-directory=/run/media/ \liveuser/ESP/EFI --no-bootsector
--removable -s --target=x86_64-efi --recheck /run/media/liveuser/ESP/

produces a BOOTX64.EFI that recognizes the F2FS filesystem on listing
(hd0,gpt1) but scrolls blank lines continuously when listing (hd0,gpt1)/
I suspect that f2fs.mod is not fully compatible with f2fs ver 1.14.

Confirmation and followup on this would be appreciated.

I haven't been able to locate the source code that specifies the modules
included in the EFI boot loaders that are provided with the Fedora Live
.isos. A pointer to that would be appreciated too.

[Frederick Grose]

On Mon, Jan 25, 2021 at 2:04 AM Frederick Grose <fgrose(a)gmail.com&gt; wrote:

>  See https://github.com/livecd-tools/livecd-tools/pull/176.
>
>  A cumulative update, best reviewed by testing.
>
>  Enable formatting of a separate EFI System Partition and Apple HFS+ boot
>    partition based on the files efiboot.img and macboot.img in the /images
>    directory of the source.  This means EFI booting is possible with overlay
>    sizes as large as the free space on the primary partition.
>  Support F2FS type formatting for the device root, home, or overlay
>    filesystems.
>  Provide boot support for F2FS-formatted discs with an EFI Boot Stub loader
>    from dracut, as GRUB support for F2FS is faulty at this time.
>  Allow conversion of the root filesystem type between ext[234], btrfs, xfs,
>    & F2FS and their available block sizes with editliveos.
>  Allow a Device-mapper target to be the target device for installation.
>    This would enable concurrent multi device installation through a
>    preconfigured Device-mapper mirror target.
>
>  Also,
>    - introduce the --noesp option to --format to avoid the extra partitions;
>    - on UEFI booted hosts, offer to create a UEFI boot entry for F2FS-based
>      installations;
>    - do not require SYSLINUX-EXTLINUX on the installation host;
>    - take greater advantage of BASH case statements for code structure;
>    - use fallocate on filesystem creation to reserve storage space for
>      embedded filesystems;
>    - add a trap ERR line for error reporting in livecd-iso-to-disk;
>    - provide better support for isoscan/filename boots from .iso files;
>    - delete the uncompressed image file just after compression to make
>      more space available, if preservation of temporary files is not selected;
>    - use bind mounts for /run, /sys, & /dev in liveimage-mount;
>    - update documentation.
>
>  ref: ... as GRUB support for F2FS is faulty at this time.
>
>  I found that the following command run against a livecd-iso-to-disk installed LiveUSB,
>
>  sudo grub2-install -v --boot-directory=/run/media/ \liveuser/ESP/EFI/BOOT --compress=xz
> --modules="part_gpt part_msdos f2fs" --efi-directory=/run/media/
> \liveuser/ESP/EFI --no-bootsector --removable -s --target=x86_64-efi --recheck
> /run/media/liveuser/ESP/
>
>  produces a BOOTX64.EFI that recognizes the F2FS filesystem on listing
>  (hd0,gpt1) but scrolls blank lines continuously when listing (hd0,gpt1)/
>  I suspect that f2fs.mod is not fully compatible with f2fs ver 1.14.
>
>  Confirmation and followup on this would be appreciated.
>
>  I haven't been able to locate the source code that specifies the modules
>  included in the EFI boot loaders that are provided with the Fedora Live
>  .isos. A pointer to that would be appreciated too.
>
>  
An incompatibility has been confirmed for F2FS extra_attr formatted
filesystems with GRUB f2fs.mod. See

https://lists.gnu.org/archive/html/help-grub/2021-01/msg00009.html
and
https://savannah.gnu.org/bugs/?59976
The new force-pushed commit now says, (
https://github.com/livecd-tools/livecd-tools/commit/7c6c53144608eb4c60f0d...
)

Provide a GRUB EFI boot binary with basic F2FS filesystem support for
      x86_64 architecture systems.
Provide boot support for F2FS extra_attr formatted discs with an EFI Boot
      Stub loader from dracut, as GRUB support for this format is faulty at
      this time.

When run on a UEFI booted host, livecd-iso-to-disk will offer to write a
UEFI boot entry into the host computer UEFI Boot Manager. It also now
reports on the situation, if relevant, when the script ends.

I'm still searching for where in the Fedora tool chain the GRUB EFI boot
binaries' module lists are specified and composed. Adding f2fs.mod to the
default filesystem list would be the goal.

[Neal Gompa]

On Mon, Feb 8, 2021 at 7:12 PM Frederick Grose <fgrose(a)gmail.com&gt; wrote:
>  On Mon, Jan 25, 2021 at 2:04 AM Frederick Grose <fgrose(a)gmail.com&gt; wrote:
> >
> > See https://github.com/livecd-tools/livecd-tools/pull/176.
> >
> > A cumulative update, best reviewed by testing.
> >
> > Enable formatting of a separate EFI System Partition and Apple HFS+ boot
> >   partition based on the files efiboot.img and macboot.img in the /images
> >   directory of the source.  This means EFI booting is possible with overlay
> >   sizes as large as the free space on the primary partition.
> > Support F2FS type formatting for the device root, home, or overlay
> >   filesystems.
> > Provide boot support for F2FS-formatted discs with an EFI Boot Stub loader
> >   from dracut, as GRUB support for F2FS is faulty at this time.
> > Allow conversion of the root filesystem type between ext[234], btrfs, xfs,
> >   & F2FS and their available block sizes with editliveos.
> > Allow a Device-mapper target to be the target device for installation.
> >   This would enable concurrent multi device installation through a
> >   preconfigured Device-mapper mirror target.
> >
> > Also,
> >   - introduce the --noesp option to --format to avoid the extra partitions;
> >   - on UEFI booted hosts, offer to create a UEFI boot entry for F2FS-based
> >     installations;
> >   - do not require SYSLINUX-EXTLINUX on the installation host;
> >   - take greater advantage of BASH case statements for code structure;
> >   - use fallocate on filesystem creation to reserve storage space for
> >     embedded filesystems;
> >   - add a trap ERR line for error reporting in livecd-iso-to-disk;
> >   - provide better support for isoscan/filename boots from .iso files;
> >   - delete the uncompressed image file just after compression to make
> >     more space available, if preservation of temporary files is not selected;
> >   - use bind mounts for /run, /sys, & /dev in liveimage-mount;
> >   - update documentation.
> >
> > ref: ... as GRUB support for F2FS is faulty at this time.
> >
> > I found that the following command run against a livecd-iso-to-disk installed
> LiveUSB,
> >
> > sudo grub2-install -v --boot-directory=/run/media/ \liveuser/ESP/EFI/BOOT
> --compress=xz --modules="part_gpt part_msdos f2fs" --efi-directory=/run/media/
> \liveuser/ESP/EFI --no-bootsector --removable -s --target=x86_64-efi --recheck
> /run/media/liveuser/ESP/
> >
> > produces a BOOTX64.EFI that recognizes the F2FS filesystem on listing (hd0,gpt1) but
> scrolls blank lines continuously when listing (hd0,gpt1)/
> > I suspect that f2fs.mod is not fully compatible with f2fs ver 1.14.
> >
> > Confirmation and followup on this would be appreciated.
> >
> > I haven't been able to locate the source code that specifies the modules included
> in the EFI boot loaders that are provided with the Fedora Live .isos. A pointer to that
> would be appreciated too.
>
>
>
>  An incompatibility has been confirmed for F2FS extra_attr formatted filesystems with GRUB
> f2fs.mod. See
>
>  https://lists.gnu.org/archive/html/help-grub/2021-01/msg00009.html
>  and
>  https://savannah.gnu.org/bugs/?59976
>  The new force-pushed commit now says,
> (https://github.com/livecd-tools/livecd-tools/commit/7c6c53144608eb4c60f0d...)
>
>  Provide a GRUB EFI boot binary with basic F2FS filesystem support for
>        x86_64 architecture systems.
>  Provide boot support for F2FS extra_attr formatted discs with an EFI Boot
>        Stub loader from dracut, as GRUB support for this format is faulty at
>        this time.
>
>  When run on a UEFI booted host, livecd-iso-to-disk will offer to write a UEFI boot entry
> into the host computer UEFI Boot Manager. It also now reports on the situation, if
> relevant, when the script ends.
>
>  I'm still searching for where in the Fedora tool chain the GRUB EFI boot
> binaries' module lists are specified and composed. Adding f2fs.mod to the default
> filesystem list would be the goal. 
Javier, do you know anything about this?



-- 
真実はいつも一つ！/ Always, there's only one truth!