Fix fallocate issue on cloud-init 19.4 for Fedora 33 cloud images

[Federico Ressi]

Hello all,

I am writing to this list because I found out F33 cloud image cloud init
support for creating swap files looks to be broken probably because of a
known Linux 5.7+ kernel issue [1].

The problem is cloud-init is trying to create a new swap file by using
fallocate command that is not working well (kernel is complaining the file
has holes when executing swapon command just later). The easy workaround
for this issue is to use dd command instead of fallocate command in
cloud-init.

Because I don't know the whole procedure required for submitting a patch I
am writing to you in the hope you can help me in having the F33 cloud image
fixed.
Please find attached to this e-mail the simple fix for cloud init 19.4 I
just wrote. The patch should apply the procedure documented in Fedora docs
[2]

Thank you in advance.
Federico

[1] https://marcelhauri.ch/blog/swapon-swapfile-has-holes/
[2]
https://docs.fedoraproject.org/en-US/Fedora/14/html/Storage_Administratio...

[Dusty Mabe]

On 4/26/21 10:17 AM, Federico Ressi wrote:
>  Hello all, 
Hey Frederico,


>  
>  I am writing to this list because I found out F33 cloud image cloud init support for
> creating swap files looks to be broken probably because of a known Linux 5.7+ kernel issue
> [1].
>  
>  The problem is cloud-init is trying to create a new swap file by using fallocate command
> that is not working well (kernel is complaining the file has holes when executing swapon
> command just later). The easy workaround for this issue is to use dd command instead of
> fallocate command in cloud-init.
>  
>  Because I don't know the whole procedure required for submitting a patch I am writing
> to you in the hope you can help me in having the F33 cloud image fixed.
>  Please find attached to this e-mail the simple fix for cloud init 19.4 I just wrote. The
> patch should apply the procedure documented in Fedora docs [2] 
Do you know if this is fixed in upstream cloud-init already? Fedora 34, which is about to
be released tomorrow has cloud-init-20.4-1.fc34.
> 
> Thank you in advance.
> Federico
> 
> [1] https://marcelhauri.ch/blog/swapon-swapfile-has-holes/
<https://marcelhauri.ch/blog/swapon-swapfile-has-holes/> 
>
[2] https://docs.fedoraproject.org/en-US/Fedora/14/html/Storage_Administra...
<https://docs.fedoraproject.org/en-US/Fedora/14/html/Storage_Administratio...

[Chris Murphy]

On Mon, Apr 26, 2021 at 8:18 AM Federico Ressi <fressi(a)redhat.com&gt; wrote:
>  Hello all,
>
>  I am writing to this list because I found out F33 cloud image cloud init support for
> creating swap files looks to be broken probably because of a known Linux 5.7+ kernel issue
> [1].
>
>  The problem is cloud-init is trying to create a new swap file by using fallocate command
> that is not working well (kernel is complaining the file has holes when executing swapon
> command just later). The easy workaround for this issue is to use dd command instead of
> fallocate command in cloud-init. 
fallocate default mode is zero, and doesn't create holes. If there are
holes, it's a kernel bug, and it needs to be fixed and kernel updated.
It's also worth making sure cloud-init is using fallocate's default
mode of zero.

The simplest work around is to just create a swap partition instead of
a swapfile, when using cloud images that have the buggy kernel. Or
alternatively don't create either one, and instead write a config to
/etc/systemd/zram-generator.conf so the installation uses swap on a
compressed zram device.

>  Because I don't know the whole procedure required for submitting
> a patch I am writing to you in the hope you can help me in having the F33 cloud image
> fixed. 
I don't think cloud images get reissued after release. But you can
reproduce that process and make your own cloud image that's identical
to the Fedora one, except for the kernel. That way you can use a newer
kernel that doesn't have the bug.

Or like Dusty suggests, give the Fedora 34 Cloud images a whirl. Out tomorrow!

-- 
Chris Murphy

[Eduardo Otubo]

>  On 4/26/21 10:17 AM, Federico Ressi wrote:
>  
>  Hey Frederico,
>  
>  
>  
>  Do you know if this is fixed in upstream cloud-init already? Fedora 34, which is about
> to
>  be released tomorrow has cloud-init-20.4-1.fc34. 
This is fixed by the upstream commit:

commit 6603706eec1c39d9d591c8ffa0ef7171b74d84d6
Author: Eduardo Otubo <otubo(a)redhat.com&gt;
Date:   Thu Jan 23 17:41:48 2020 +0100

    Do not use fallocate in swap file creation on xfs. (#70)

    When creating a swap file on an xfs filesystem, fallocate cannot be used.
    Doing so results in failure of swapon and a message like:
     swapon: swapfile has holes

    The solution here is to maintain a list (currently containing only XFS)
    of filesystems where fallocate cannot be used. The, on those fileystems
    use the slower but functional 'dd' method.

    Signed-off-by: Eduardo Otubo <otubo(a)redhat.com&gt;
    Co-authored-by: Adam Dobrawy <naczelnik(a)jawnosc.tk&gt;
    Co-authored-by:  Scott Moser <smoser(a)brickies.net&gt;
    Co-authored-by: Daniel Watkins <daniel(a)daniel-watkins.co.uk&gt;

    LP: #1781781

And is available on cloud-init >= 20.1, which is the case for Fedora 34.

In case this is a critical bug for you, please file a BZ and assign it to me. I
can check the possibility for a backport.

Thanks!

[Chris Murphy]

tl;dr Creating a swapfile with fallocate should work on XFS now.

Found this thread from about 3 years ago
https://www.spinics.net/lists/linux-mm/msg147100.html

There were a few things that needed work to fix it, and I'm not sure
which one finally did it, but this 2019 patch was part of that series:
https://lore.kernel.org/linux-xfs/[e-mail]/

That is in 5.7, which is what Fedora 33 shipped with.
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/fs/...

Anyway...even though I can't nail down exactly when it got fixed,
today I asked the XFS maintainer about all of this and he said
fallocate'd swapfiles should work. And I also tested it with kernel
5.11.16 and 5.12 and it does work.

--
Chris Murphy