[releng] Issue #7400: Rethink how we handle networking config with createImage (oz / ImageFactory)

[Adam Williamson]

adamwill reported a new issue against the project: `releng` that you are following:
``
I spent a bit of time digging into the issues around
https://pagure.io/fedora-kickstarts/pull-request/366 this morning, and I figure we can
improve this area more generally, but I don't think the optimal path is *completely*
obvious, so I thought I'd file an issue for discussion before filing any PRs.

As things stand, I believe, for all `createImage` tasks, we wind up with oz running the
installer in a VM such that a single network interface is brought up, using a udev
'persistent' name. anaconda then writes out an ifcfg file for that
'persistent' name - e.g. `ifcfg-ens3` or `ifcfg-ens0p3` or something - to the
installed system.

If you poke about in fedora-kickstarts a bit, you'll find that five kickstarts then do
stuff to try and 'clean this up'. We can ignore `fedora-cloud-bigdata` and
`fedora-cloud-experimental`, I think, so we're left with `fedora-atomic`, which does
this:

    bootloader --timeout=1 --append="no_timer_check console=tty1
console=ttyS0,115200n8 console=ttyAMA0 console=hvc0 net.ifnames=0"
    network --bootproto=dhcp --device=link --activate --onboot=on

    ....

    # Remove any persistent NIC rules generated by udev
    rm -vf /etc/udev/rules.d/*persistent-net*.rules
    # And ensure that we will do DHCP on eth0 on startup
    cat > /etc/sysconfig/network-scripts/ifcfg-eth0 << EOF
    DEVICE="eth0"
    BOOTPROTO="dhcp"
    ONBOOT="yes"
    TYPE="Ethernet"
    PERSISTENT_DHCLIENT="yes"
    EOF

    ....

    # For trac ticket https://pagure.io/atomic-wg/issue/128
    rm -f /etc/sysconfig/network-scripts/ifcfg-ens3

`fedora-cloud-base`, which does this:

    bootloader --timeout=1 --append="no_timer_check net.ifnames=0 console=tty1
console=ttyS0,115200n8"
    network --bootproto=dhcp --device=link --activate --onboot=on

    ....

    # When we build the image with oz, dracut is used 
    # and sets up a ifcfg-en<whatever> for the device. We don't 
    # want to use this, we use eth0 so it is always the same. 
    # So we remove all these ifcfg-en<whatever> devices so 
    # The 'network' service can come up cleanly.
    rm -f /etc/sysconfig/network-scripts/ifcfg-en*

And `fedora-disk-base`, which does this:

    network --bootproto=dhcp --device=link --activate
    bootloader --timeout=1

    # The enp1s0 interface is a left over from the imagefactory install, clean this up
    rm -f /etc/sysconfig/network-scripts/ifcfg-enp1s0

Note they all do something different for trying to remove the 'predictably'-named
interface configuration produced by the installer. We touched the `cloud-base` version
most recently, and it's most probably the best approach.

`disk-base` doesn't try to disable 'persistent' naming for the installed
system, which I believe is intentional and correct, as that kickstart is for the ARM disk
images which will be booted on real hardware with potentially varying network adapters and
should probably respect the distro default to use 'persistent' naming on real
hardware. I don't know how this results in the network coming up on first boot of
these images, if it even does, but hey, that seems a bit out of scope.

`atomic` and `cloud-base` both try to disable 'persistent' naming for the
installed system. `cloud-base` used to try and do it just by removing udev rules, which
was dumb, so @nirik recently changed that to add `net.ifnames=0` to the kickstart
`bootloader` line, which basically disables udev 'predictable' naming (the udev
rules will see the param and not rename the device). `atomic` seems to do a
belt-and-braces approach ATM: it *both* sets `net.ifnames=0` in the `bootloader` line
*and* tries to remove udev rules, badly (they don't live in `/etc/udev/rules.d` any
more).

One obvious cleanup would be to make all three use the `rm -f
/etc/sysconfig/network-scripts/ifcfg-en*` strategy for removing the
'predictably'-named interface config file. Another obvious cleanup would be to
take the udev rule deletion bits out of `fedora-atomic`, since they're not doing
anyone any good there.

There is another thing we can do, though. oz actually permits (since
https://github.com/clalancette/oz/commit/14ad1922aa8c0922aaa2a3f9e52daa26... , I
think that's version 0.13.0) the passing of arbitrary extra kernel parameters to the
installer, at least for the RHEL/Fedora 'url' install type, which is what
`createImage` uses. You just have to add a `kernelparam` entry to the template.

We could send a PR for Koji to use this to disable 'predictable' interface naming
during the image creation; either to just *always* include a `kernelparam` value of
`biosdevname=0 net.ifnames=0` for `createImage` tasks, or to make this configurable via
the command line somehow and then have pungi / pungi-fedora pass it in for the images we
want to use it for (handwave handwave).

This should mean anaconda would write an `ifcfg-eth0` into the installed system, and for
Atomic and Cloud images, in theory we wouldn't have to mung *anything* in the
kickstart. @kevin points out that the file might in fact specify a MAC address, which we
would have to filter out, but we might be able to tweak the kickstart to avoid that,
I'll look into it.

For ARM images, we'd just want to tweak the kickstart to remove that `ifcfg-eth0` file
instead of the name it currently tries to remove - but this is slightly *better* than the
current situation, where we've had at least one case where the 'predictable'
interface name suddenly became unpredictable. At least in this particular workflow, the
name `eth0` for the sole interface used during the image build should be *truly*
predictable.

Anyway, that's where I'm up to. Thoughts / comments / ideas on this?

@kevin @mohanboddu @dustymabe @walters @mikem @lsedlar 
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/7400