On Wed, 2024-04-10 at 11:22 -0400, Neal Gompa wrote:
On Wed, Apr 10, 2024 at 10:54 AM Tadej Janež <tadej.j(a)nez.si>
wrote:
>
> On Tue, 2024-04-09 at 10:00 -0400, Neal Gompa wrote:
> > On Tue, Apr 9, 2024 at 9:53 AM Tadej Janež <tadej.j(a)nez.si>
> > wrote:
> > >
> > > Hey Fedora Cloud WG!
> > >
> > > Firstly, thanks for providing Fedora for the popular cloud
> > > providers!
> > >
> > > Since Fedora 35, the Fedora Cloud images use btrfs by default
> > > [1].
> > > For my deployments, I would like to use ext4 or xfs, so my
> > > question
> > > are:
> > > 1) Is it possible to change the root file system at deploy
> > > time?
> > > 2) If not, how could one change the cloud images to use a
> > > different
> > > root file system?
> > >
> >
> > It is not possible to change at deployment time, you would need
> > to
> > build your own custom images.
> >
>
> Ok, thanks for clarifying!
>
> > Our images are now defined here:
> >
https://pagure.io/fedora-kiwi-descriptions/blob/rawhide/f/teams/cloud/clo...
> >
> > If you want to use something else, you'd want to have your own
> > version
> > of the definitions and modify that file to use the filesystem of
> > your
> > choice.
>
> I see, one would need to modify the "filesystem" key of the
> selected
> Cloud-Base-<provider> image.
>
> Are there instructions on how to build a custom cloud image?
>
The repository README includes a quickstart section for building your
own images.
> However, even if I can easily build a custom cloud image, the
> overhead
> of maintaining custom cloud images is very high.
>
> For every Fedora release and cloud provider, I would need to build
> the
> image and then with IaC (e.g. Terraform), handle uploading the
> custom
> image and using it...
>
Yeah, it is more overhead, for sure.
Thinking out loud... the modification in my case would not be a custom
tailored image for my use case but more similar to how Fedora project
builds different spins if people want to use a non-GNOME desktop
environment or ALT images for people using alternative architectures.
Is there a possibility to reuse the existing image building
infrastructure and add tasks for building images with e.g. ext4 as the
rootfs?
> >
> > Is there a particular reason you want to use ext4 or xfs for your
> > rootfs? Typically the pattern we see is that people attach a
> > secondary
> > volume or use S3 and put their data on that instead of the
> > rootfs.
> >
>
> Yes, I use that pattern as well. Usually, /srv or /var would be on
> a
> separate block storage and formatted with the file system of
> choice.
>
> The reasons why I would want to use, e.g. ext4 for the rootfs,
> would
> be:
>
> 1. Familiarity. I've mainly been using ext4 or LVM+ext4 (with LUKS
> underneath) for the last 2 decades. I know the tools and I know
> what
> "care" such filesystems need.
>
> 2. Maturity. Ext2/3/4 have been round for quite longer than btrfs
> and
> there are very little "unknowns" or "surprises" with it.
>
> 3. Simplicity. When provisioning machines with a cloud provider, I
> actually don't need the LVM+ext4 combination because the cloud
> provider
> would typically handle the things LVM would handle for a non-cloud
> machine, e.g. increasing the block storage size, snapshotting, ...
>
> Please, don't read this as a critique against btrfs, just me trying
> to
> explain why I would find it nicer to just use ext4 for the rootfs
> as
> well.
>
That's totally fair. I will point out that one of the big reasons we
use Btrfs is actually to enable space efficiency (which can lead to
cost savings across the board). The used storage is roughly 40% less
than on ext4 due to leveraging both transparent compression and
reflinks.
The other reason is to make it easy to do replication for backups and
other purposes.
You may also find it to be beneficial for easily avoiding certain
kinds of failure cases that can happen on ext4 (such as inode
exhaustion with containers), since btrfs dynamically allocates inodes
per subvolume.
You can also grow (and shrink!) the volume live while the system is
mounted and online safely.
Thanks for enumerating these benefits. And I agree that these benefits
alone justify the reason for switching to btrfs by default.
I might be tempted to check out the btrfs status page again and start
playing with it.