2:17 p.m.
= Upgrade systems to createrepo_c 1.0 and change repositories metadata
settings <!-- The name of your change proposal --> =
{{Change_Proposal_Banner}}
Wiki -> https://fedoraproject.org/wiki/Changes/ChangeComposeSettings
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
This is a proposal for upgrading systems which produce composes to
createrepo_c > 1.0 and changing some options used to create Fedora
repositories metadata. Note that some of these changes are inevitable
due to createrepo_c >= 1.0 behavioral change. We aim to change both
Rawhide/F41, then move all following releases to the new settings,
while preserving most of the current settings for releases <= 40.
== Owner ==
* Name: [[User:mattia| Mattia Verga]], [[User:kevin| Kevin Fenzi]]
* Email: mattia(a)fedorapeople.org, kevin(a)fedorapeople.org
== Detailed Description ==
With createrepo_c < 1.0 we're currently using different settings for
Rawhide repository metadata and stable releases repository metadata.
* Rawhide
** gzip compression for all metadata
** no updateinfo.xml file is generated (no updates repository exists)
** sqlite database of repodata is generated and compressed with gzip as well
** comps repodata is made available both as uncompressed xml and gzipped
** zchunk is active
** DRPMs disabled
* F40
** gzip compression for primary metadata
** updateinfo.xml (for updates repository) is compressed with XZ
** sqlite database of repodata is generated and compressed with BZ2
** comps repodata is made available both as uncompressed xml and gzipped
** zchunk is active
** DRPMs disabled (approved change for F40)
* F<40
** gzip compression for primary metadata
** updateinfo.xml (for updates repository) is compressed with XZ
** sqlite database of repodata is generated and compressed with BZ2
** comps repodata is made available both as uncompressed xml and gzipped
** zchunk is active
** DRPMs enabled
With createrepo_c > 1.0 moving to zstd as the default compression
type, we want to have consistent settings for all new releases and to
specify those settings manually, so that possible future changes of
defaults don't cause unexpected breakages. So we propose the following
settings:
* Rawhide/F>=41
** use `--general-compress-type` set to zstd to compress all metadata to zstd
** updateinfo.xml (for updates repository) compressed with zstd as well
** disable generating the additional sqlite database, as it was only
useful for yum
** comps repodata will be available only zstd compressed
** zchunk is active
** DRPMs disabled
* F<=40
** nothing should change by using the `--compatibility` flag of
createrepo_c >= 1.0
Note that updating bodhi-composer to use createrepo_c >= 1.0 will also
introduce an unavoidable change into EPEL8 updates repositories: comps
repodata will be made available only in compressed format (which is
set to XZ in EPEL8). Other EPEL releases (EPEL7 and EPEL9) can use the
`--compatibility` flag to maintain actual settings.
== Feedback ==
The zstd compression type was chosen to match createrepo_c settings.
As an alternative, we might want to choose xz, especially after
zlib-ng has been made the default in Fedora and brought performance
improvements.
The sqlite database distributed alongside the repodata is not useful
for dnf, but it might be used by some external consumer we're not
aware. Please do let us know.
== Benefit to Fedora ==
We're aiming at having consistent defaults for Rawhide and stable
releases and avoid future changes to createrepo_c defaults to cause
unexpected changes to repodata. Also, by using better compression
methods and avoid generating the sqlite database we're reducing
repodata disk usage.
== Scope ==
* Proposal owners:
** change createrepo_c settings for Rawhide in compose-rawhide01
(which is already upgraded to f39)
** upgrade bodhi-composer to f39 and have it using createrepo_c >= 1.0.0
** upgrade (if a new release is released in time) or backport patch
into bodhi to support all createrepo_c settings
** change bodhi's `createrepo_c.ini` in ansible to use the new
settings for F>=41
* Other developers:
* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Community Initiatives:
== Upgrade/compatibility impact ==
No change should be noticed while upgrading from previous releases.
== How To Test ==
DNF normal day usage should not be affected: upgrading/installing
packages should work as before, maybe a little faster in downloading
repodata.
== User Experience ==
User experience should not be affected.
Possible external custom repodata consumers might stop working and
will need to adjust to the new compression method.
== Dependencies ==
== Contingency Plan ==
* Contingency mechanism: revert back to old createrepo_c < 1.0 and
previous settings and wait for the next day compose.
* Contingency deadline: F41-beta freeze
* Blocks release? Yes
== Documentation ==
https://docs.pagure.org/pungi/configuration.html#createrepo-settings
== Release Notes ==
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney