distro release and reproduceability

[Matt Domsch]

On Wed, Apr 29, 2009 at 12:35:35AM +0200, Farkas Levente wrote:
> hi,
> i started a thread on centos-devel:
> http://lists.centos.org/pipermail/centos-devel/2009-April/004398.html
> but actually it's nothing to do with centos rather rhel and fedora.
> the question comes when i try to rebuild centos on itself. it turns out
> not possible what's more it's almost impossible to rebuild (or takes
> much more longer then the build time).
> the problem is the process how each packages and updates added to the
> distro ie. just the new packages rebuild and doesn't care whether the
> old packages still build. and easy example the latest mass rebuild.
> there were some problems with since new gcc can't compile old 'buggy'
> code, but there are many problems which has nothing to do with gcc, rpm
> or i586.
> the first question:
> - is it a goal for fedora to be able to rebuild itself?
> i hope the answer is yes:-)

yes.

> - how we can achieve this goal?
> imho the current policy and process is not good enough.

See http://fedoraproject.org/wiki/FTBFS

for one example of what is currently being done process-wise.

> 
> first of all it'd be useful to somehow automatically check all spec file
> during builds (eg. don't put check into %build, don't put build into
> %install or %prep etc).
> 
> the most important when a new or updated package build and added, then
> - at leat rebuild every package which depend on or requires the new
> package if all builds succeed add the new packages otherwise don't allow
> until other compilation fails.
> - but would be better same as 1. but also include all new rebuild in the
> updates which are different from it's last build (here some smart diff
> would be required eg. binary must be the same (what if newer gcc
> generate different/better code?) some other files can differ eg.
> generated docs can have some 'small' differences. this second step is a
> more complicated and requires a few more tools but would be better.
> 
> of course all of his requires a bit more build farm resource but produce
> a much better and stable distro.

Some projects such as openSUSE do throw a lot of compute resources at
the problem, rebuilding all dependent packages in the chain when a
lower-level package is rebuilt.  Fedora hasn't felt the need to go
quite this far.

Between the FTBFS build runs that I do, and the mass rebuilds that the
Release Engineering team did for Fedora 11, and those that Jacob has
done when major gcc changes are about ready, nearly all (but not
exactly all) packages can be rebuilt as expected.  The few (and it was
< 5% last time I looked, maybe significantly fewer even than that
after the F11 rebuilds) are quite problematic and mostly bit-rotten
code that people still have need for but upstream is stagnent.

Thanks,
Matt

-- 
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux