[fedora-arm] Fedora 15 ARM hardfp Virtual FAD (Fedora Activity Day) - FRIDAY

[Jon Masters]

On Sun, 2011-07-10 at 16:23 -0500, Matt Domsch wrote:
> On Sun, Jul 10, 2011 at 04:43:30PM +0100, Matthew Garrett wrote:
> > On Sat, Jul 09, 2011 at 11:45:33PM -0400, Jon Masters wrote:
> > 
> > > 	* Fedora should (IMO) institute mandatory mass rebuilds. Either every
> > > cycle, or every other cycle. I've briefly discussed with Dennis.
> > > Bootstrapping (and similar activities) are far easier with a clean set
> > > of deps, which is the case for F15. It should always be the case that we
> > > know everything builds and self-hosts through a mass rebuild per cycle.
> > 
> > This has been raised with FESCO in the past, and I don't think there's 
> > any fudnamental disagreement on it. But scheduling one mass rebuild per 
> > cycle doesn't prevent us ending up in a broken state unless we do it 
> > right at the end of the cycle, and right now that's problematic in terms 
> > of release process - rebuilding everything we've just QAed is an 
> > excellent way to introduce subtle breakage. So it really needs to be an 
> > out-of-archive verification rather than one that's targetted at the 
> > release, and we need the resources and manpower to handle it.
> 
> Alternately, we could take a lesson from our compatriots at openSUSE.
> Their openSUSE Build Service throws a combination of automated
> intelligence and hardware at the problem.  Given the package
> dependency tree, if package B BuildRequires package A, then every time
> A gets rebuilt, B is also bumped and rebuilt.

This is, in my opinion, the correct way to solve the problem. You can't
really know if something FTBFS unless you do this kind of thing. I'd go
further and advocate for sufficient hardware to continually rebuild
everything daily and automate bootstrap like Debian are looking into,
but that's all stuff for the future.

> This causes build
> breakage to get caught fairly early in the process (rather than via an
> asynchronous out-of-tree process), and the resulting packages are available in
> their equivalent of the rawhide tree for test and use.

It doesn't just benefit bootstrap either. Take (random example) the
recent CFLAGS change in redhat-rpm-config. What should happen at that
point is that every package is automatically rebuilt. Should it cause a
problem? No. But having packages randomly fail to build later because of
some change made months or even years earlier is something to fix.

I know I'm preaching to the choir here. There are many reasonable
reasons why some of these problems exist, and it's no failure on the
part of rel-eng folks, who do their best.

Jon.