On Thu, 25 Jun 2020 at 05:29, Dan Čermák <dan.cermak(a)cgc-instruments.com> wrote:
this will probably get buried given the immense amount of replies in
this thread already, but nevertheless, here's my 2cts.
Stephen John Smoogen <smooge(a)gmail.com> writes:
> On Sat, 20 Jun 2020 at 17:42, Neal Gompa <ngompa13(a)gmail.com> wrote:
>> On Sat, Jun 20, 2020 at 5:25 PM John M. Harris Jr <johnmh(a)splentity.com>
>> > On Saturday, June 20, 2020 4:42:17 AM MST Neal Gompa wrote:
>> > > TL;DR benefits of modularity for Fedora:
>> > >
>> > > * Automating build chains for producing artifacts
>> > > * Straightforward mechanism of producing non-rpm artifacts using our
>> > > existing tooling (modules -> flatpaks/containers/etc.)
>> > Both of these have nothing to do with Modularity, and can be done with
>> > existing RPMs.
>> They have everything to do with Modularity, because that layer is
>> where that stuff was implemented. Modularity was the result of the
>> efforts involved with Factory 2.0, which gave us a lot of improvements
>> in our build infrastructure tooling for the first time since 2007.
>> Most of that rolled out in 2017, a full ten years after the last
>> revamp of our infrastructure.
> I think that John and others aren't aware of how a module is built
> enough to understand what you meant by
> * Automating build chains for producing artifacts
> compared to how it is done normally.
> In normal times, a packager goes through a list of packages, updates
> spec files with new tags and rebuilds them one by one as needed..
> sometimes multiple times because of bootstrapping or finding out that
> the order you tried was wrong. A made up example from my days of doing
> this for a different place (this isn't what is needed anymore but long
> ago I had something similar):
> gcc [options 1]
> gcc [options 2]
> do them in one order and the apps came out working... do them in the
> wrong order and it might not. Rust, Java and other language stacks
> have similar loops. A packager may have to coordinate with multiple
> people to do this several times.
> In a module, you write that all down in the manifest with the order
> that you want packages built in and if you need to loop through them
> with different options. Then MBS does the builds in an automated
> fashion and it is repeatable. To me this is the biggest win here as
> for various groups of mass-rebuilds it cuts down errors when order
> matters and you have to do multiple ones to get from package set A to
> package set A+1.
This is actually freaking awesome! The one thing where our build system
is really laking is that we have to manually figure out the build order
and rebuild stuff manually. Other systems like OBS handle this
completely automatically, albeit bootstrapping on OBS has its issues as
Anyway, is there any chance that we can get this functionality split out
of MBS and add it to koji? If we then convince koschei to kick of real
builds in Rawhide, then we'd be in a much better shape build system wise
Not without packagers en-mass accepting changes to how 'THEIR'
packages are built. That middle gcc needs to have a different RPM
release than the one before it and the one after it. That means that
in our current work flow you could start working on a package and
between your starting on updating a .spec file and committing and
trying to push it.. it automatically got bumped several above because
someone else was doing these builds to get 'their' package built
correctly. This has been the gordian knot of our package
ownership/maintenance.. with various packagers reverting and undoing
things in the middle of someone else because THEIR package got updated
in a way they didn't want. This has been fought over since the first
days of the Fedora.us project and honestly never seems to go away.
The political solution was to assume that in a module you take
responsibility for all the packages in that module even if other
people 'own' their 'non-modular' versions. You can then do what is
needed to the packages and then not publish those special middle
versions of gcc which you needed but would have broken normal gcc for
everyone else.. or some other odd thing. [This again is all made up
examples.. it could be a language which seems to need you to build
each version only from a specific previous version and so you need to
chain an 'infinite' list to bootstrap to a newer version.]
I honestly do not know enough about OpenSuSE Build System to know how
they deal with this automatically. I had assumed it wasn't too
automatic but comes from how detailed Suse spec files are so that you
don't have a package solver 'guess' what was needed. That simplifies
the dependency graph a lot. Also that packagers do not 'own' things
in the same way they have in Fedora land so the politics is skipped.
Stephen J Smoogen.