Re: MBI (Playground 2.0)
On Thu, Jan 31, 2019 at 7:28 AM Igor Gnatenko
<ignatenkobrain(a)fedoraproject.org> wrote:
MayBe I …(can do something useful)?
Hello,
We've been discussing some (hopefully) nice idea with Mikolaj, Neal and Jakub how we
could improve packager (and user) experience and we have some proposal which will be
described below.
We would like to ask you to read it, understand it and ask us any questions you have.
From the Fedora Infrastructure we would like to ask for some machines to implement this
idea (you can find some more information in "Implementation details" part).
I would like to apologize for HTML email, but it is much easier to have it that way to
have better visibility and reading easiness.
Feel free to reply to this email or comment in google doc (there is a link on the
bottom).
Proposal Owners
Mikolaj Izdebski (mizdebsk) - Java SIG, Fedora infrastructure
Igor Gnatenko (ignatenkobrain) - Rust SIG, Golang SIG, Neuro SIG, RPM and libsolv
contributor
Neal Gompa (ngompa) - Rust SIG, Golang SIG, RPM contributor
Jakub Čajka (jcajka) - Golang SIG, Container SIG
This proposal aligns with the objective of improving the packager experience by
developing a platform whereby the proposal owners and others can experiment with
improvements that can be funneled back into the official production infrastructure.
Problems
Problem №1: Build-only packages
Some ecosystems have many build-only packages (packages which are used to build other
packages, without having them installed on end-user systems). Those ecosystems include
Java, Rust and Go.
It is extremely hard to keep up maintaining them due to lack of time/people. Upstreams
are usually changing quickly (sometimes when you update just one such package, you’d need
to update tens of the dependencies). Few facts about such packages:
They are almost always outdated in released versions of distribution;
They are often FTBFS (e.g. because there was compiler update but not package update,
broken deps, broken APIs due to deps rebases, …).
In Rust ecosystem, we abuse Rawhide to build and store such packages there and then
generate end-user application (e.g. ripgrep) which uses some of those packages and
produces binary for all supported releases. Those applications have high quality and are
supported.
Rawhide gating makes this much more complicated because builds appear in buildroot
slower, updating group of packages would need side tags and it’s just painful to work
with.
https://pagure.io/fesco/issue/2068
And, after all, those packages shouldn’t be shipped to users.
Problem №2: Testing of new rpm/koji/mock features/configuration
When developing new features in RPM (e.g. rich dependencies) or testing different Koji
configuration (e.g. removing gcc/gcc-c++ from the buildroot), it is required to make
custom configuration and try building things.
Problem №3: Developing modules
Modules are built in MBS as a single unit. It is hard to develop big modules by
progressively improving individual packages or small package groups. Scratch builds for
modules are not implemented. Local builds work differently from official module builds,
they don’t scale and don’t allow groups of people to work collaboratively. All these
problems can be solved by first developing a flat repository of packages in a shared
environment and then generating modulemd from such package set.
Problem №4: Testing things before push
Primary Fedora Koji and dist-git are not suited for packaging experimentation. Packagers
are expected to experiment on their own systems and push changes of successful experiments
only. But this approach doesn’t scale with number of maintained packages. Often you can
find commits like “d’oh, forgot to upload sources” or “let’s try with this settings”.
People need to build things somewhere quickly, do development and testing. And only after
that, they should push commit(s) to Fedora.
Solution
Separate Koji + Koschei deployed in Fedora infrastructure cloud;
Why does this need to be deployed in the fedora infrastructure cloud?
Seems like you could stand it up in AWS or somewhere else.
josh