Hi folks,
it's this time of the year, we should do some major filesystem surgery!
The preparations for https://fedoraproject.org/wiki/Changes/Unify_bin_and_sbin have been put in place and I want to do the rebuild of filesystem.rpm rpm.rpm, and other packages that will effectuate the merge.
What has happened so far:
1. The selinux policy has been updated to treat make /usr/sbin equivalent to /usr/bin. This is necessary to avoid selinux becoming very unhappy when paths are changed [0,1].
2. I prepped a pull request for filesystem.rpm to do the merge [2]. This is described in detail below. The patches have been tentatively acked, so I want to merge & build them, after another round of testing. Doing this will be the real beginning of the merge.
3. [3] is the counterpart for rpm; it makes rpm report that %_sbindir==%_bindir.
4. Over the last two months I submitted a bunch of dist-git pull requests with titles like "Fix build when %_bindir==%_sbindir" [e.g. 4] and "Add compat sbin Provides" [e.g. 5]. Both types are written in a way that it's safe to merge them at any time. The maintainers of those packages got notifications and a bunch got merged.
5. As a remnant of the usr-merge transition, we had a packaging rule that packages MUST use pre-usr-merge paths. It turns out that almost no packages got this right. The rule was removed in [6].
6. The packaging guidelines have been updated for the sbin-merge [7].
Big thanks to everyone who responded to the patches, made reviews and merges!
What will happen next week:
0. I'll create a side tag for the builds described below.
1. When filesystem.rpm with the patch [2] is built, new installations with that package will have /usr/sbin as a symlink. This includes buildroots. The patch for rpm [3] will we merged and built right afterwards, so that the declaration by rpm macros matches reality on disk.
2. Some packages (of those that have files in /usr/sbin) will start to FTBFS in a buildroot with merged-sbin. For example, when they cannot move %_sbindir/foo to %_bindir/foo or the other way. Patches like "Fix build when %_bindir==%_sbindir" [4] fix those cases. It's not necessary to merge them immediately, but only if the package actually needs to be rebuilt.
3. Some packages will start to FTI in an merged-sbin installation. The error looks like this: Transaction failed: Rpm transaction failed. - file /usr/sbin/sestatus conflicts between attempted installs of policycoreutils-3.6-5.fc41.x86_64 and policycoreutils-3.6-5.fc41.x86_64 - file /usr/sbin/named-checkzone conflicts between attempted installs of bind-utils-32:9.18.26-1.fc41.x86_64 and bind-utils-32:9.18.26-1.fc41.x86_64 (I guess rpm/dnf could do a slightly better error message here…)
Those packages need to rebuilt ASAP. I'll rebuild them in the side tag too, with a commit title "Rebuilt for the bin-sbin merge". In many cases, the patches to fix FTBFS will need to be applied first. In those cases, I'll merge the "Fix build" pull requests.
4. We have packages which use filepath Requires on paths in %_sbindir. Such packages will FTI when the _providing_ package is rebuilt with the new value of %_sbindir. To keep those packages working, I made a list of all such filepath dependencies in Fedora and prepared patches for the provider packages to add a virtual Provides for the old name, e.g. [5]. This means that the provider package has Provides:/usr/sbin/foo before the merge and Provides:/usr/bin/foo,/usr/sbin/foo when rebuilt after the merge.
I'll rebuild all packages that need to add the virtual Provides in the side tag too.
(In case anyone is wondering, why like this, the answer is that there are few different ways the transition could be handled, but I think this one the least painful. Dependent packages will remain installable in pre-merge and post-merge systems. This means we don't have a flag day when everything needs to be rebuilt. Also, there are many more "consumer" than provider packages so the total number of required changes is relatively small. Also, if we learn about third-party packages or other uses on the filepath provides, we can add more Provides to make the transition easier.)
Those Provides can be removed after the transition is fully complete. I think we shouldn't hurry with that, they can stay around for a few years. We still have virtual Provides for the usr-merge in many packages after all.
5. After the side tag is merged, the merge is on for users. As mentioned above, any new installations will have /bin, /sbin, and /usr/sbin symlinks to /usr/sbin. This means that paths /bin/foo, /sbin/foo, /usr/sbin/foo, and /bin/foo all point to the same file. This is the target state.
On existing installations, we'll have a bunch of packages with files and symlinks in directory /usr/sbin. filesystem.rpm has scriptlets to create symlinks from /usr/sbin/foo to /usr/bin/foo for any packaged files that were in /usr/sbin. (There is a hardcoded list.) This means that on unmerged systems, the old path continues to work, and that if a virtual Provides for the old path is provided, the old path actually continues to work.
filesystem.rpm has a virtual Provides:filesystem(unmerged-sbin-symlinks) to declare that it implements this functionality. Packages which depend on this functionality have Requires:filesystem(unmerged-sbin-symlinks).
On existing installations, as long as /usr/sbin contains real files or symlinks that don't point to /usr/bin, the system remains unmerged. As packages are rebuilt and upgraded, the number of such cases will go down. filesystem.rpm has a %posttrans scriptlet to check, and once it detects that /usr/sbin only contains symlinks to /usr/bin, it'll replace the directory itself by a symlink. At that point, we're at the target state too.
6. systemd has been patched to skip /usr/sbin and /usr/local/sbin in $PATH if those are symlinks. (Strictly speaking, this item belongs in the previous section, but its effect is not visible until after the merge is done.)
Items 1,3,4 specify packages to be rebuilt. I'll prepare an update list and send it out with a CC to maintainers sometime this week.
[0] https://github.com/fedora-selinux/selinux-policy/pull/2077 [1] https://src.fedoraproject.org/rpms/selinux-policy/c/df27da1e6e8a0247924eb968... [2] https://src.fedoraproject.org/rpms/filesystem/pull-request/11 [3] https://src.fedoraproject.org/rpms/rpm/pull-request/56 [4] https://src.fedoraproject.org/rpms/sanlock/pull-request/13 [5] https://src.fedoraproject.org/rpms/e2fsprogs/pull-request/7 [6] https://pagure.io/packaging-committee/pull-request/1355 [7] https://pagure.io/packaging-committee/pull-request/1361 [8] https://github.com/systemd/systemd/pull/32389/commits/0f36a4c897ff53eb0be3bd...
Zbyszek