March 2018 - rel-eng - Fedora Mailing-Lists

[releng] Issue #7398: Make Rawhide repository handling more suitable for automated processing

by Kamil Páral

kparal reported a new issue against the project: `releng` that you are following: `` **Background** I've been working on Fedora QA automation for many years. All these years, I've had to fight with the way how Fedora structures its repositories especially regarding Rawhide. I can't even count in how many tools we needed to define the current Rawhide <-> `$releasever` mapping and maintain that mapping. That includes special paths for Rawhide (or Branched) in terms of repository location. Today I finally forced myself to file this ticket and ask Releng for improving this from their side (while maintaining full compatibility with existing approaches). **Problem** Currently, `$releasever` and repository handling uses a completely different approach for stable releases, Branched and Rawhide. Stable releases use `$releasever` NN number when accessing repos, either over mirrormanager: https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$... or directly using a repo path: http://dl.fedoraproject.org/pub/fedora/linux/releases/$releasever/ They have `fedora`, `updates` and `updates-testing` system repos defined. Branched release uses `$releasever` for mirrormanager and for repo path, but it's placed in `development/` directory rather then `releases/`: http://dl.fedoraproject.org/pub/fedora/linux/development/$releasever/ It has `fedora`, `updates` (empty) and `updates-testing` (empty or used, depending on release schedule) system repos defined. Rawhide *doesn't* use `$releasever` neither for mirrormanager nor for repo path, and it's again placed in `development/`: http://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/ It has `rawhide` system repo defined (no `fedora`, `updates` or `updates-testing`). Due to Rawhide not using `$releasever`, it needs to have its custom set of repository files (and a differently named base repo), which point to the "right" mirrormanager and baseurl URLs. So in conclusion, this is the list of things that don't work: * Accessing a particular release repo path without the knowledge of its current status (some are in `development/`, some are in `releases/`). * Accessing Rawhide repo using `$releasever` (which **is** provided by the system, just doesn't work), in particular: https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$... http://dl.fedoraproject.org/pub/fedora/linux/development/$releasever/ All these are not accessible. * Using the same repository config file in any Fedora release, making use of `$releasever` in URLs (breaks on Rawhide). This doesn't involve only Fedora repo files, but also any third-party repo files. * Querying other releases' repodata from Rawhide, by using `dnf --releasever=` (because repo URLs are hardcoded to Rawhide). * Using commands like `dnf --enablerepo=updates-testing` from Rawhide. The consequence of all of this is that our tools are littered with if clauses, doing special things for Rawhide (and sometimes Branched). We need to have separate configs for different releases, often [quite complicated](https://pagure.io/taskotron/libtaskotron/blob/8a7f76c5a336d0b8e5ec99d7996b3baf39c4ddb0/f/conf/yumrepoinfo.conf.example). We can't use the same additional repositories in all releases. We need to do hacks to figure out if a provided disk image is Rawhide or not, or deciding if we're running on Rawhide release during runtime. And the worst of all, we need to maintain all of this and reconfigure it several times per year (branching, going stable). This is error-prone, tedious and causes a delay during which things crash or provide incorrect results (e.g. between the branching happens and the configs are updated and deployed). I'd like all of these issue to go away, and be maintenance free. **Proposed solution** 1) Teach mirrormanager to recognize both `rawhide` and its matching `$releasever` at the same time. When Rawhide is 29, these two URLs should return the same content: https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-rawhide&arch=x86_64 https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-29&arch=x86_64 2) Create empty `updates` and `updates-testing` repositories for rawhide. These repos will be forever empty. Make them exposed in mirrormanager again under both names (29 and rawhide). This will allow to use the same DNF commands in stable releases and in Rawhide without side-effects. 3) Create a directory in repo layout that contains symlinks to all releases regardless of their release status. For example let's create http://dl.fedoraproject.org/pub/fedora/linux/releases/all showing this content: ``` linux/ ├── development │ ├── 28 │ ├── 29 -> rawhide │ └── rawhide └── releases ├── 26 ├── 27 └── all ├── 26 -> ../26 ├── 27 -> ../27 ├── 28 -> ../../development/28 ├── 29 -> rawhide └── rawhide -> ../../development/rawhide ``` As you can see, all releases are addressable from there. Not only that, but when Rawhide is 29, both these names are included and translate to the same target. This is important, because it allows people to use `$releasever` everywhere. 4) Have a `$releasever` number symlink also for `updates` and `updates/testing` paths: ``` linux/ └── updates ├── 26 ├── 27 ├── 28 ├── 29 -> rawhide ├── rawhide └── testing ├── 26 ├── 27 ├── 28 ├── 29 -> rawhide └── rawhide ``` **Future improvements** The changes proposed above would improve the situation immensely for me, and for other developers/tool authors interacting with releases in a similar fashion. But when this is in place, other, releng-related improvements could be achieved. For example, `fedora-repos-rawhide` could be got rid of. Currently you need to maintain two different set of repos, one for rawhide and one for others. With the above changes, you could use the same repos for all releases, i.e. `fedora`, `updates` and `updates-testing`. The last two would be empty for rawhide. The base repo would automatically work using `$releasever`. However, this way the release would follow the `$releasever` even after branching happens, so people running Rawhide would suddenly run Branched. That might not be desired, and could be solved by `fedora-repos-rawhide` containing just a single file - a config drop-in for DNF, that would override `$releasever` to `rawhide` (this part would need to be implemented, because it seems it's not possible today). With this approach, just by installing `fedora-repos-rawhide`, the users would make sure they always stay on Rawhide. The package contents would never need to be updated, same as `fedora-repos` wouldn't. Everything would work without regular maintenance. CC @ausil @kevin @puiterwijk @dmach @jskladan @tflink @adamwill `` To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7398

6 years, 1 month

9
23
0 / 0

[releng] Issue #7360: Bugzilla Assignee Overrides: text unclear

by Nikos Mavrogiannopoulos

nmav reported a new issue against the project: `releng` that you are following: `` I'm reading the following text [from the fedora-scm-requests project](https://pagure.io/releng/fedora-scm-requests). ``` To override the default assignee in Bugzilla, find/or create the YAML file that configures the repository you want to set (e.g. rpms/python-requests). The bugzilla_contact key in the YAML file is used to override this with a dictionary, where each key represents a product (e.g. "Fedora EPEL" or "Fedora"). Each value should specify the FAS username or FAS group (groups are signified by starting with a "@") of the default Bugzilla ticket assignee for the component in that product. ``` It is unclear to me what to do to override the bugzilla assignee. Questions I have: * How to find or create the yaml file? * Where does it have to be (in the package repo), subdir? * Does it have to have a specific name or any yaml file would do? * Could you give an example of such a file? `` To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7360

6 years, 1 month

3
5
0 / 0

[releng] Issue #7418: F28 Beta: some 32 bit images are missing

by Robert Mayr

robyduck reported a new issue against the project: `releng` that you are following: `` Hi releng, I was updating the websites and saw we are missing several 32 bit images. For example I can see only Security and Python-Classroom 32 bit images for Labs, while for Spins we don't have the KDE 32 bit image. I know these are not blocking and I'll drop them, but it would be good to know if this is a compose issue (it seems not related to the issue we had for F27 Beta though) or if there has just been a copy/sync error, as koji gives me the status as completed: https://kojipkgs.fedoraproject.org/compose/28/Fedora-28-20180328.1/STATUS Thanks. `` To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7418

6 years, 1 month

2
2
0 / 0

[releng] Issue #7298: Unretire compton - Compositor for X11

by Abhiram Kuchibhotla

axeld reported a new issue against the project: `releng` that you are following: `` compton - Compositor for X11 Package Review: https://bugzilla.redhat.com/show_bug.cgi?id=1532042 Branches: master f27 `` To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7298

6 years, 1 month

3
4
0 / 0

[pungi-fedora] PR #592: Sync Server to /pub/fedora-secondary/releases/development/{release}

by Mohan Boddu

mohanboddu opened a new pull-request against the project: `pungi-fedora` that you are following: `` Sync Server to /pub/fedora-secondary/releases/development/{release} `` To reply, visit the link below or just reply to this email https://pagure.io/pungi-fedora/pull-request/592

6 years, 1 month

2
1
0 / 0

[pungi-fedora] PR #591: Sync Server to /pub/fedora-secondary/releases/development/{release}

by Mohan Boddu

mohanboddu opened a new pull-request against the project: `pungi-fedora` that you are following: `` Sync Server to /pub/fedora-secondary/releases/development/{release} `` To reply, visit the link below or just reply to this email https://pagure.io/pungi-fedora/pull-request/591

6 years, 1 month

2
1
0 / 0

[releng] Issue #7359: implantisomd5 sometimes fails during i686 live image composes

by Adam Williamson

adamwill reported a new issue against the project: `releng` that you are following: `` A few i686 live images seem to fail because implantisomd5 fails. Examples: [Design suite](https://koji.fedoraproject.org/koji/taskinfo?taskID=25368692), [Games](https://koji.fedoraproject.org/koji/taskinfo?taskID=25368729). The `program.log` is the most useful in both cases, it logs the failure: Running... implantisomd5 /var/tmp/lmc-work-z7_burpl/images/boot.iso ERROR: Error - Unable to open file /var/tmp/lmc-work-z7_burpl/images/boot.iso Return code: 1 Both images are fairly large - you can see the size a little earlier in the log, 2151 MB in one case, 4459 MB in the other. `` To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7359

6 years, 1 month

2
2
0 / 0

[releng] Issue #7412: Review change request for IDM products

by Mark Reynolds

mreynolds reported a new issue against the project: `releng` that you are following: `` Please review Fedora Change Request... https://fedoraproject.org/wiki/Changes/389-ds-base-remove-686 The core of the issue is with the 389-ds-base package, but this change also impacts FreeIPA and slapi-nis `` To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7412

6 years, 1 month

3
4
0 / 0

[releng] PR #7417: Removing src syncing for aarch64

by Mohan Boddu

mohanboddu opened a new pull-request against the project: `releng` that you are following: `` Removing src syncing for aarch64 `` To reply, visit the link below or just reply to this email https://pagure.io/releng/pull-request/7417

6 years, 1 month

2
1
0 / 0

[releng] Issue #7416: 20180329: Fedora Container Release

by Clement Verna

cverna reported a new issue against the project: `releng` that you are following: `` Can we release the following : - fedora container base image for f27 - fedora container layered image for f27 Many thanks `` To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7416

6 years, 1 month

2
7
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

rel-eng March 2018