Fedora 35 Cloud image && virt-sysprep

[Pavel Raiskup]

Hello,

anyone had a successful experience with Fedora 35 Cloud images, and
guestfish/virt-sysprep?

Seems like we switched from ext4 or xfs to 'btrfs', and guestfish
doesn't work with the images, am I right?  At least I had problems
on EL8 hypervisors so far.

Pavel

[Neal Gompa]

On Mon, Nov 29, 2021 at 9:24 AM Pavel Raiskup <praiskup(a)redhat.com&gt; wrote:
>  Hello,
>
>  anyone had a successful experience with Fedora 35 Cloud images, and
>  guestfish/virt-sysprep?
>
>  Seems like we switched from ext4 or xfs to 'btrfs', and guestfish
>  doesn't work with the images, am I right?  At least I had problems
>  on EL8 hypervisors so far.
>  
There's a libguestfs container you can use on EL8 systems for this
purpose: https://src.fedoraproject.org/container/libguestfs



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Daniel P. Berrangé]

On Mon, Nov 29, 2021 at 03:24:12PM +0100, Pavel Raiskup wrote:
>  Hello,
>  
>  anyone had a successful experience with Fedora 35 Cloud images, and
>  guestfish/virt-sysprep?
>  
>  Seems like we switched from ext4 or xfs to 'btrfs', and guestfish
>  doesn't work with the images, am I right?  At least I had problems
>  on EL8 hypervisors so far. 
As a general rule, libguestfs is only guaranteed to work if the
host OS is the same or newer than the guest OS. It might (and often
does) work if the host OS is older than the guest, but that's pot
luck.

The libguestfs appliance use the host kernel to access the filesystems,
so if the guest OS kernel was newer, its filesystem may have used features
that the host OS kernel doesn't understand.

IOW, my assumption is that Fedora 35 xfs is using fetures that RHEL8
xfs doesn't support.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Pavel Raiskup]

On Monday, November 29, 2021 4:02:23 PM CET Daniel P. Berrangé wrote:
>  On Mon, Nov 29, 2021 at 03:24:12PM +0100, Pavel Raiskup wrote:
>  > Hello,
>  > 
>  > anyone had a successful experience with Fedora 35 Cloud images, and
>  > guestfish/virt-sysprep?
>  > 
>  > Seems like we switched from ext4 or xfs to 'btrfs', and guestfish
>  > doesn't work with the images, am I right?  At least I had problems
>  > on EL8 hypervisors so far.  > 
>  As a general rule, libguestfs is only guaranteed to work if the
>  host OS is the same or newer than the guest OS. It might (and often
>  does) work if the host OS is older than the guest, but that's pot
>  luck.
>  
>  The libguestfs appliance use the host kernel to access the filesystems,
>  so if the guest OS kernel was newer, its filesystem may have used features
>  that the host OS kernel doesn't understand.
>  
>  IOW, my assumption is that Fedora 35 xfs is using fetures that RHEL8
>  xfs doesn't support. 
Ah, not xfs, but you helped me.  Now I understand what the message means :-)

$ virt-sysprep -a ...
[   0.0] Examining the guest ...
virt-sysprep: error: libguestfs error: inspect_os: mount exited with status 
32: mount: /tmp/btrfse5L6v2: unknown filesystem type 'btrfs'.

This is the absence of 'btrfs' on EL8.  So there's no way out from this
on any currently supported RHEL, I bet.

Pavel

>  Regards,
>  Daniel
>  -- 
>  |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
>  |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
>  |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>

[Daniel P. Berrangé]

On Mon, Nov 29, 2021 at 04:38:26PM +0100, Pavel Raiskup wrote:
>  On Monday, November 29, 2021 4:02:23 PM CET Daniel P. Berrangé
> wrote:
>  > On Mon, Nov 29, 2021 at 03:24:12PM +0100, Pavel Raiskup wrote:
>  > > Hello,
>  > > 
>  > > anyone had a successful experience with Fedora 35 Cloud images, and
>  > > guestfish/virt-sysprep?
>  > > 
>  > > Seems like we switched from ext4 or xfs to 'btrfs', and guestfish
>  > > doesn't work with the images, am I right?  At least I had problems
>  > > on EL8 hypervisors so far.  > 
>  > As a general rule, libguestfs is only guaranteed to work if the
>  > host OS is the same or newer than the guest OS. It might (and often
>  > does) work if the host OS is older than the guest, but that's pot
>  > luck.
>  > 
>  > The libguestfs appliance use the host kernel to access the filesystems,
>  > so if the guest OS kernel was newer, its filesystem may have used features
>  > that the host OS kernel doesn't understand.
>  > 
>  > IOW, my assumption is that Fedora 35 xfs is using fetures that RHEL8
>  > xfs doesn't support.
>  
>  Ah, not xfs, but you helped me.  Now I understand what the message means :-)
>  
>  $ virt-sysprep -a ...
>  [   0.0] Examining the guest ...
>  virt-sysprep: error: libguestfs error: inspect_os: mount exited with status 
>  32: mount: /tmp/btrfse5L6v2: unknown filesystem type 'btrfs'.
>  
>  This is the absence of 'btrfs' on EL8.  So there's no way out from this
>  on any currently supported RHEL, I bet. 
I've not tried, but the container Neal points to in the other reply
may still work, if it includes a Fedora kernel vmlinux that libguestfs
can leverage. The host OS kernel is merely libguestfs' default choice,
you can point it to other kernels if you have some available

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Richard W.M. Jones]

This seems like it was fairly comprehensively answered already:
RHEL 8 doesn't support btrfs.

However about XFS, we are in fact going to backport the two missing
features (bigtime and inobtcnt) to RHEL 8.  The bugs are unfortunately
private but for reference they are:

https://bugzilla.redhat.com/show_bug.cgi?id=2022903
https://bugzilla.redhat.com/show_bug.cgi?id=2024201

Note that this will allow you to open RHEL 9 / Fedora 35 XFS
filesystems in RHEL 8, but maybe not create them from scratch using
RHEL 8 tools like mkfs.xfs.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

[Pavel Raiskup]

On Monday, November 29, 2021 4:46:42 PM CET Daniel P. Berrangé wrote:
>  On Mon, Nov 29, 2021 at 04:38:26PM +0100, Pavel Raiskup wrote:
>  > On Monday, November 29, 2021 4:02:23 PM CET Daniel P. Berrangé wrote:
>  > > On Mon, Nov 29, 2021 at 03:24:12PM +0100, Pavel Raiskup wrote:
>  > > > Hello,
>  > > > 
>  > > > anyone had a successful experience with Fedora 35 Cloud images, and
>  > > > guestfish/virt-sysprep?
>  > > > 
>  > > > Seems like we switched from ext4 or xfs to 'btrfs', and guestfish
>  > > > doesn't work with the images, am I right?  At least I had problems
>  > > > on EL8 hypervisors so far.  > 
>  > > As a general rule, libguestfs is only guaranteed to work if the
>  > > host OS is the same or newer than the guest OS. It might (and often
>  > > does) work if the host OS is older than the guest, but that's pot
>  > > luck.
>  > > 
>  > > The libguestfs appliance use the host kernel to access the filesystems,
>  > > so if the guest OS kernel was newer, its filesystem may have used features
>  > > that the host OS kernel doesn't understand.
>  > > 
>  > > IOW, my assumption is that Fedora 35 xfs is using fetures that RHEL8
>  > > xfs doesn't support.
>  > 
>  > Ah, not xfs, but you helped me.  Now I understand what the message means :-)
>  > 
>  > $ virt-sysprep -a ...
>  > [   0.0] Examining the guest ...
>  > virt-sysprep: error: libguestfs error: inspect_os: mount exited with status 
>  > 32: mount: /tmp/btrfse5L6v2: unknown filesystem type 'btrfs'.
>  > 
>  > This is the absence of 'btrfs' on EL8.  So there's no way out from this
>  > on any currently supported RHEL, I bet.
>  
>  I've not tried, but the container Neal points to in the other reply
>  may still work, if it includes a Fedora kernel vmlinux that libguestfs
>  can leverage. The host OS kernel is merely libguestfs' default choice,
>  you can point it to other kernels if you have some available 
I can confirm it works, libguestfs-appliance brings kernel-core into the
image.

Though, can I somehow manually point the host's libguestfs to the
alternative kernel (e.g. from extracted kernel-core.rpm from F35)?  I
mean.., it is really painful to fix our scripting to use the containerized
libguestfs.  If so, any command touching host filesystem needs to be
changed (e.g. copy-in/copy-out).  Also command-line arguments (-a) need to
be changed.

But it is indeed good enough for manual, one-time image tweaks (there are
some minor issues, like absence of Fedora 35 Dockerfile, the image isn't
in registries, etc.).

Pavel

[Daniel P. Berrangé]

On Wed, Dec 01, 2021 at 11:20:05PM +0100, Pavel Raiskup wrote:
>  On Monday, November 29, 2021 4:46:42 PM CET Daniel P. Berrangé
> wrote:
>  > On Mon, Nov 29, 2021 at 04:38:26PM +0100, Pavel Raiskup wrote:
>  > > On Monday, November 29, 2021 4:02:23 PM CET Daniel P. Berrangé wrote:
>  > > > On Mon, Nov 29, 2021 at 03:24:12PM +0100, Pavel Raiskup wrote:
>  > > > > Hello,
>  > > > > 
>  > > > > anyone had a successful experience with Fedora 35 Cloud images, and
>  > > > > guestfish/virt-sysprep?
>  > > > > 
>  > > > > Seems like we switched from ext4 or xfs to 'btrfs', and
> guestfish
>  > > > > doesn't work with the images, am I right?  At least I had
> problems
>  > > > > on EL8 hypervisors so far.  > 
>  > > > As a general rule, libguestfs is only guaranteed to work if the
>  > > > host OS is the same or newer than the guest OS. It might (and often
>  > > > does) work if the host OS is older than the guest, but that's pot
>  > > > luck.
>  > > > 
>  > > > The libguestfs appliance use the host kernel to access the filesystems,
>  > > > so if the guest OS kernel was newer, its filesystem may have used
> features
>  > > > that the host OS kernel doesn't understand.
>  > > > 
>  > > > IOW, my assumption is that Fedora 35 xfs is using fetures that RHEL8
>  > > > xfs doesn't support.
>  > > 
>  > > Ah, not xfs, but you helped me.  Now I understand what the message means :-)
>  > > 
>  > > $ virt-sysprep -a ...
>  > > [   0.0] Examining the guest ...
>  > > virt-sysprep: error: libguestfs error: inspect_os: mount exited with status 
>  > > 32: mount: /tmp/btrfse5L6v2: unknown filesystem type 'btrfs'.
>  > > 
>  > > This is the absence of 'btrfs' on EL8.  So there's no way out from
> this
>  > > on any currently supported RHEL, I bet.
>  > 
>  > I've not tried, but the container Neal points to in the other reply
>  > may still work, if it includes a Fedora kernel vmlinux that libguestfs
>  > can leverage. The host OS kernel is merely libguestfs' default choice,
>  > you can point it to other kernels if you have some available
>  
>  I can confirm it works, libguestfs-appliance brings kernel-core into the
>  image.
>  
>  Though, can I somehow manually point the host's libguestfs to the
>  alternative kernel (e.g. from extracted kernel-core.rpm from F35)?  I
>  mean.., it is really painful to fix our scripting to use the containerized
>  libguestfs.  If so, any command touching host filesystem needs to be
>  changed (e.g. copy-in/copy-out).  Also command-line arguments (-a) need to
>  be changed. 
You can set env vars

       SUPERMIN_KERNEL
       SUPERMIN_KERNEL_VERSION
       SUPERMIN_MODULES

see man guestfish(1) or supermin(1) 

Depending on what you are doing, this may or may not be sufficient. It
depends whether you trigger anything that requires the userspace tools
to know about the filesystem or not.

If not you'll also need to get the full appliance image. See more info
at

  https://libguestfs.org/libguestfs-make-fixed-appliance.1.html

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|