[releng] Issue #7403: Consider no longer producing the Workstation repo tree
in composes
by Neal Gompa
ngompa reported a new issue against the project: `releng` that you are following:
``
Recently, we stopped producing [the Cloud repo tree](https://pagure.io/pungi-fedora/pull-request/577) as part of composes to speed things up and reduce the number of useless deliverables.
I suggest we also stop producing the Workstation repo tree for composes, as we don't use it or need it. Today, for the Workstation Edition, we produce two main artifacts: the live ISO and the branded netinstall ISO.
As far as I'm aware, the branded net install ISO differs from the main Fedora netinstall ISO only in branding and defaults through the anaconda productimg embedded within. It uses the `Everything` repo tree like the regular netinstall ISO, and is functionally similar to the regular netinstall ISO.
Since the Workstation netinstall doesn't need it, and we don't produce a Workstation install DVD ISO, I do not see a reason to keep producing the Workstation repo tree. Not producing the tree would help reduce the compose time significantly, too.
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/7403
4 years, 7 months
[releng] Issue #7445: [RFE][PATCH] make $releasever return "rawhide" on
Rawhide
by Kamil Páral
kparal reported a new issue against the project: `releng` that you are following:
``
**Background**
If we want to have a more reliable and stable Rawhide, we need to make it easier to test and automate. That means eliminating the differences between Rawhide and stable releases and reducing the necessary manual maintenance steps as much as possible. You can read more about related issues in https://pagure.io/releng/issue/7398 and https://pagure.io/copr/copr/issue/267.
**Problem**
Currently dnf variable `$releasever` returns a number (29) on Rawhide, but all the repos are stored in `rawhide/` directory, not `29/` (as with stable releases). There are good reasons for this, but it has consequences. It forces the official fedora repos to be split between `fedora-repos` and `fedora-repos-rawhide` (because you can't rely on a variable and have to hardcode "rawhide" in the repo path) and breaks copr and any other third-party repos. Basically for all repos, you need to have two separate versions - rawhide and non-rawhide - and always correctly detect and install the right one. I'd like to propose improvements in this area and discuss it with you in this ticket.
**Proposed solution 1**
Here's a trivial patch for `fedora-release`:
```
diff --git a/fedora-release.spec b/fedora-release.spec
index ecca47f..b4b66f2 100644
--- a/fedora-release.spec
+++ b/fedora-release.spec
@@ -1,6 +1,7 @@
%define release_name Rawhide
%define dist_version 29
%define bug_version rawhide
+%define releasever rawhide
# All changes need to be submitted as pull requests in pagure
# The package can only be built by a very small number of people
@@ -19,6 +20,7 @@ Obsoletes: redhat-release
Provides: redhat-release
Provides: system-release
Provides: system-release(%{version})
+Provides: system-release(releasever) = %{releasever}
# Kill off the fedora-release-nonproduct package
Provides: fedora-release-nonproduct = %{version}
```
This adds provision `system-release(releasever) = rawhide` to the `master` branch of `fedora-release`. Therefore, this provision will only be present for Rawhide version of that package. It uses DNF's [detect_releasever()](https://github.com/rpm-software-management/dnf/blob/... logic to populate `$releasever` with `rawhide` string (the new provides) instead of `29` (the version of the package). (Note: This is currently broken in DNF due to a [bug](https://bugzilla.redhat.com/show_bug.cgi?id=1568366), but it will be fixed in the next DNF release).
The outcome is that all repos can now use `$releasever` in URLs, because it will get replaced by `rawhide` and therefore reach the correct destination. That means you can use the same repo file as in a stable release for COPR or other third-party repo and it will work fine.
If the user wants to switch to Branched after branching has happened, they'd run e.g.:
```
sudo dnf distrosync fedora-release\* --releasever=28
```
**Proposed solution 2**
This is a similar approach to the first solution, but creates a `fedora-release-rawhide` subpackage:
```
diff --git a/fedora-release.spec b/fedora-release.spec
index ecca47f..74637f1 100644
--- a/fedora-release.spec
+++ b/fedora-release.spec
@@ -1,6 +1,7 @@
%define release_name Rawhide
%define dist_version 29
%define bug_version rawhide
+%define releasever rawhide
# All changes need to be submitted as pull requests in pagure
# The package can only be built by a very small number of people
@@ -33,6 +34,15 @@ BuildArch: noarch
%description
Fedora release files such as various /etc/ files that define the release.
+%package rawhide
+Summary: Fedora release files for Rawhide
+Provides: system-release(releasever) = %{releasever}
+Requires: fedora-release = %{version}-%{release}
+
+%description rawhide
+This identifies the system as Rawhide for the package manager, causing Rawhide
+repositories to be used.
+
%package atomichost
Summary: Base package for Fedora Atomic-specific default configurations
Provides: system-release-atomichost
@@ -315,6 +325,9 @@ glib-compile-schemas %{_datadir}/glib-2.0/schemas &> /dev/null || :
%{_prefix}/lib/systemd/system-preset/99-default-disable.preset
+%files rawhide
+
+
%files atomichost
%{!?_licensedir:%global license %%doc}
%license LICENSE
```
The difference here is that you can install/uninstall `fedora-release-rawhide` any time at will, which marks/unmarks your system to be following the Rawhide stream. The benefit is that you can switch your system from Rawhide to Branched before the branching actually happens, and your system automatically picks up the right repos after branching (which is awesome, especially for our automation needs). The downside is that both `rawhide/` and `29/` repo URLs/paths need to be present and working during the whole life cycle of Rawhide, so that you can switch any time. And this doesn't apply just to official Fedora repos, but ideally also to COPR and other third-party repos. COPR devs [wanted to avoid](https://pagure.io/copr/copr/issue/267) duplicated content or maintaining symlinks, but I guess they could be convinced. But other third-party repos might not follow this approach and the whole concept might be confusing for them (however, a good question is how many of those repos actually work on Rawhide already).
So to summarize, this is how you'd switch your system to Rawhide:
```
# use dnf system-upgrade to upgrade to Rawhide
sudo dnf install fedora-release-rawhide
```
And switching from Rawhide to Branched:
```
sudo dnf remove fedora-release-rawhide
sudo dnf distrosync # if branching already happened
```
Fresh Rawhide installation would receive `fedora-release-rawhide` by default, of course.
Overall, this adds more user control at the expense of more infra work. Not sure if this is worth it or not.
**Proposed solution 3**
For the sake of completeness, I'll mention another approach how to achieve the goal without using new RPM provides. `$releasever` value can be overridden by a file
like this:
```
$ cat /etc/dnf/vars/releasever
rawhide
```
If this file was owned by `fedora-release-rawhide`, it would be very similar to solution 2 - you can switch the Branched/Rawhide stream any time. The implementation detail here is whether to mark this file as a config file or not, so that it doesn't e.g. stay around even after you remove `fedora-release-rawhide`, or that it doesn't e.g. conflict with an already existing file at that location (if the user wanted to override `$releasever` already for any reason).
Solution 2 seems a bit cleaner here because you don't need to bother with corner cases involving config file management.
**Possible future steps for Fedora Releng**
Once `$releasever` returns `rawhide` on Rawhide, you can (if you wish) drop `fedora-repos-rawhide` and use the same `fedora-repos` package everywhere (ideally also create empty `updates/rawhide` and `updates/testing/rawhide` repos as requested in https://pagure.io/releng/issue/7398). Some repo properties will still probably have different values (like `metadata_expire`), but that can be easily adjusted in the spec file and you can have the same source tarball for all releases, if you wish. This would make the environment even more consistent for users (the repo names would be named the same in all releases).
**Known pitfalls**
There's one known problem with any of the approaches suggested above, called PackageKit. PackageKit doesn't use DNF to figure out `$releasever`, nor it uses the same logic. Instead, it parses `VERSION_ID` from `/etc/os-release` ([source1](https://github.com/hughsie/PackageKit/blob/2f1c4b820b056efc989be..., [source2](https://github.com/hughsie/PackageKit/blob/1e7858b1b67120b377adc...). So any changes described here will not apply to PackageKit and it will still return a number (e.g. 29) as `$releasever`. That is something that of course needs to get resolved as well, but before investing time into fixing it, I first wanted to know whether this whole idea gets approved or not.
There are several approaches how to fix this in PackageKit, either retrieving `$releasever` from libdnf (when on Fedora), or implementing the same detection logic as in DNF, or perhaps adding `VERSION_CODENAME=Rawhide` to `/etc/os-release` and then special-casing this in PackageKit (however, this would break if solution 2 or 3 is used and the user can switch between streams arbitrarily). However, I'd like to first talk about the concept itself, and only after that start hammering out the implementation details with PackageKit developers.
**Discussion**
Please tell me what do you think about the proposed changes. Does it make sense? Have I overlooked something important? Are there better ways to solve the aforementioned issues?
Thank you.
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/7445
4 years, 7 months
[releng] Issue #7400: Rethink how we handle networking config with
createImage (oz / ImageFactory)
by Adam Williamson
adamwill reported a new issue against the project: `releng` that you are following:
``
I spent a bit of time digging into the issues around https://pagure.io/fedora-kickstarts/pull-request/366 this morning, and I figure we can improve this area more generally, but I don't think the optimal path is *completely* obvious, so I thought I'd file an issue for discussion before filing any PRs.
As things stand, I believe, for all `createImage` tasks, we wind up with oz running the installer in a VM such that a single network interface is brought up, using a udev 'persistent' name. anaconda then writes out an ifcfg file for that 'persistent' name - e.g. `ifcfg-ens3` or `ifcfg-ens0p3` or something - to the installed system.
If you poke about in fedora-kickstarts a bit, you'll find that five kickstarts then do stuff to try and 'clean this up'. We can ignore `fedora-cloud-bigdata` and `fedora-cloud-experimental`, I think, so we're left with `fedora-atomic`, which does this:
bootloader --timeout=1 --append="no_timer_check console=tty1 console=ttyS0,115200n8 console=ttyAMA0 console=hvc0 net.ifnames=0"
network --bootproto=dhcp --device=link --activate --onboot=on
....
# Remove any persistent NIC rules generated by udev
rm -vf /etc/udev/rules.d/*persistent-net*.rules
# And ensure that we will do DHCP on eth0 on startup
cat > /etc/sysconfig/network-scripts/ifcfg-eth0 << EOF
DEVICE="eth0"
BOOTPROTO="dhcp"
ONBOOT="yes"
TYPE="Ethernet"
PERSISTENT_DHCLIENT="yes"
EOF
....
# For trac ticket https://pagure.io/atomic-wg/issue/128
rm -f /etc/sysconfig/network-scripts/ifcfg-ens3
`fedora-cloud-base`, which does this:
bootloader --timeout=1 --append="no_timer_check net.ifnames=0 console=tty1 console=ttyS0,115200n8"
network --bootproto=dhcp --device=link --activate --onboot=on
....
# When we build the image with oz, dracut is used
# and sets up a ifcfg-en<whatever> for the device. We don't
# want to use this, we use eth0 so it is always the same.
# So we remove all these ifcfg-en<whatever> devices so
# The 'network' service can come up cleanly.
rm -f /etc/sysconfig/network-scripts/ifcfg-en*
And `fedora-disk-base`, which does this:
network --bootproto=dhcp --device=link --activate
bootloader --timeout=1
# The enp1s0 interface is a left over from the imagefactory install, clean this up
rm -f /etc/sysconfig/network-scripts/ifcfg-enp1s0
Note they all do something different for trying to remove the 'predictably'-named interface configuration produced by the installer. We touched the `cloud-base` version most recently, and it's most probably the best approach.
`disk-base` doesn't try to disable 'persistent' naming for the installed system, which I believe is intentional and correct, as that kickstart is for the ARM disk images which will be booted on real hardware with potentially varying network adapters and should probably respect the distro default to use 'persistent' naming on real hardware. I don't know how this results in the network coming up on first boot of these images, if it even does, but hey, that seems a bit out of scope.
`atomic` and `cloud-base` both try to disable 'persistent' naming for the installed system. `cloud-base` used to try and do it just by removing udev rules, which was dumb, so @nirik recently changed that to add `net.ifnames=0` to the kickstart `bootloader` line, which basically disables udev 'predictable' naming (the udev rules will see the param and not rename the device). `atomic` seems to do a belt-and-braces approach ATM: it *both* sets `net.ifnames=0` in the `bootloader` line *and* tries to remove udev rules, badly (they don't live in `/etc/udev/rules.d` any more).
One obvious cleanup would be to make all three use the `rm -f /etc/sysconfig/network-scripts/ifcfg-en*` strategy for removing the 'predictably'-named interface config file. Another obvious cleanup would be to take the udev rule deletion bits out of `fedora-atomic`, since they're not doing anyone any good there.
There is another thing we can do, though. oz actually permits (since https://github.com/clalancette/oz/commit/14ad1922aa8c0922aaa2a3f9e52daa26... , I think that's version 0.13.0) the passing of arbitrary extra kernel parameters to the installer, at least for the RHEL/Fedora 'url' install type, which is what `createImage` uses. You just have to add a `kernelparam` entry to the template.
We could send a PR for Koji to use this to disable 'predictable' interface naming during the image creation; either to just *always* include a `kernelparam` value of `biosdevname=0 net.ifnames=0` for `createImage` tasks, or to make this configurable via the command line somehow and then have pungi / pungi-fedora pass it in for the images we want to use it for (handwave handwave).
This should mean anaconda would write an `ifcfg-eth0` into the installed system, and for Atomic and Cloud images, in theory we wouldn't have to mung *anything* in the kickstart. @kevin points out that the file might in fact specify a MAC address, which we would have to filter out, but we might be able to tweak the kickstart to avoid that, I'll look into it.
For ARM images, we'd just want to tweak the kickstart to remove that `ifcfg-eth0` file instead of the name it currently tries to remove - but this is slightly *better* than the current situation, where we've had at least one case where the 'predictable' interface name suddenly became unpredictable. At least in this particular workflow, the name `eth0` for the sole interface used during the image build should be *truly* predictable.
Anyway, that's where I'm up to. Thoughts / comments / ideas on this?
@kevin @mohanboddu @dustymabe @walters @mikem @lsedlar
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/7400
4 years, 9 months