On Tue, 2018-11-27 at 19:30 +0000, Peter Robinson wrote:
On Tue, Nov 27, 2018 at 6:26 PM Paul Frields
> On Tue, Nov 27, 2018 at 9:50 AM Dennis Gilmore <dennis(a)ausil.us> wrote:
> > > The customers RH serves have specific expectations, and in part that
> > > dictates how delivery tooling is done. Binding the community to that
> > > may be counterproductive. This is especially true now that RHEL 8 Beta
> > > is out -- it's the perfect time for us to be rethinking at that
> > Compose tooling is largely shared, delivery tooling is not, though
> > there is room for improvement and convergence on both sides. We should
> > also consider CentOS and how we can share tooling, process and
> > resources, they still use something else entirely. Everything used in
> > Fedora is developed in the open and can be contributed to by anyone.
> > We have been talking about making the whole process flexible and
> > dynamic for years. I know internally they would like to have many of
> > the same things Fedora wants.
> Yeah, it sounds like we're aiming in the same direction here. This is
> part of what I mean by "decomposing the compose." We have one rather
> monolithic process right now, all or nothing. And it does so much that
> it's impossible to get any fast results from it. That doesn't mean it
> doesn't work well for what it does. But it's not very flexible, as you
> pointed out.
Actually amusingly if you look back through the history post F-21 and
the massive delays that happened due to various non monolithic
processes one of the outcomes of the "get a release done with
Fedora.next" was to be able to do full composes every day because
prior to that we only did newRepo, a network install and live/cloud
images and often we'd get to Alpha TC1 right after branching to
discover a whole bunch of stuff was completely broken. We finally got
this with the move to "pungi 4" in Fedora 24, and in the intervening
releases we had huge amounts of complaints because rel-eng couldn't
provide all release artifacts to be pushed through CI/CD so
regressions could be fixed instantaneously. Yes, I'm aware things
change, and I often push them, but it's worth knowing the history how
we got to where we are!
Sure, I know about this (I did quite a bit of work around it). But I
don't see any kind of inconsistency or anything here. Yes, we needed to
ability to compose the whole distro in a saner and more organized way,
which is what Pungi 4 did, which was a significant improvement. But it
would *also* be extremely useful to be able to do smaller, faster
subsets of the compose.
It's not like that's what we were doing before, after all. As you say,
we still only did it *nightly*. And since it still involved a newRepo,
it still took forever.
> With a more flexible compose we could choose to do smaller tasks
> well as capitalize on other time savings) for something like a
> speculative testing build. That kind of build could be used to perform
> integration testing as well as other tasks (automated, not manual!).
Yes, and in some regards there's already parts of the project doing
this, the TWA release runs like this, as does the IoT release and I
mentioned above ways to break some of the bits up more.
I know about this too, and mentioned it in my initial mail in this
subthread. But it's all been done in a manual, ad-hoc way: we just
write entirely new Pungi config files for each new compose profile we
come up with. There's no notion of modularity, of being able to
actually define the individual elements and easily combine and
recombine them. There's no notion of inputs mapped to outputs, either,
really. I don't think this approach really scales much further than
we're currently doing it; I don't think it'd be practical to (for
instance) manually split the big 'Fedora' compose into 'Fedora-Live',
'Fedora-ISO', 'Fedora-ARM-Disk', etc. etc. etc. Trying to do that *just
along the same lines as we have created the existing 'new' composes*
would probably give us an unmaintainable mess.
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net