Phase out 32bit ppc due to bugfix and maintenance burden

[Phil Knirsch]

Hi everyone.

Over the past year the team working on the Power architecture for Fedora 
has been struggeling more and more with keeping 32bit alive and happy. 
This is simply due to the fact that we neither have the manpower to fix 
all the issues coming up over and over again for 32bit nor do we have 
the legacy hardware anymore for testing things (though the later is less 
of a problem).

A few weeks ago the team then sat together and talked about this for a 
while. At the end we bascially were left with 4 possible scenarios:

1) Manually modify all packages that continually fail on 32bit ppc and 
ExcludeArch them, together with the whole dep chain if necessary
2) Split off 32bit as a completely separate arch. That would require 
changes in yum and rpm as well as changes to koji and our infrastructure.
3) Someone with proven packager status from the community steps up and 
commits to do the work on fixing the 32bit ppc failures when they occur
4) Phase out and retire 32bit ppc over the course of the next Fedora release

Keeping the status quo wasn't an option to begin with, as we just can't 
continue with the current rising issues on 32bit ppc anymore.

So for 1) this would effectively mean we'd have to look at every failed 
build for 32bit ppc and put ExcludeArch: ppc in it. On top of that we'd 
then need to additionally look for all components that require or 
buildrequire recursively these packages and do the same for all those 
packages, too. Just for the java stack thats several hundreds of 
packages alone. Overall thats an enormous amount of effort and would 
have to be constantly done for any new 32bit ppc package failing, so we 
dropped this solution.

For 2) we discussed how that would look like and to what it would lead: 
If we'd split 32bit ppc and 64bit ppc in koji, we'd have to then treat 
them really as separate and distinct archs as we would never be able to 
guarantee that the same versions of all packages would be available for 
both 32bit ppc and 64bit ppc anymore (as thats kind of the point in the 
separation as well). In turn that means we'd then have to have a full 
2nd infrastructure set up for 32bit ppc, complete with hub and builders, 
nearly doubling our infrastructure. We'd also need changes in rpm, yum 
and all other package related tooling to not treat ppc and ppc64 as 
multilib anymore, as they wouldn't and couldn't be anymore (see point 
before with versions). So again, due to the massive impact of this 
separation we decided that that wouldn't be a workable solution either.

That leaves us with only option 3) and 4). For 3) we'd need someone 
really dedicated to actually fix the build issues on 32bit ppc, so 
proven packager is basically a necessity there. And that person would 
have to really commit to it. Being away for a month or 2 would block 
64bit builds for that time then as well, and thats what's really been 
hurting us more and more over the past year and what we want to get away 
from.

So unless 3) happens over the next few weeks, the only option at this 
point for us is to say goodbye to 32bit ppc for Fedora. The maintenance 
burden has grown just too big for the team to handle it and the quality 
of the 64bit ppc port suffers more and more because of it. We'll of 
course still keep the old 32bit trees around for anyone to use, enjoy or 
play around with, or even pick them up themselves and do their own thing 
with it. Or alternatively anyone can set up their on koji and 32bit ppc 
build infrastructure for building things[1]

The plan for now is to do this prior to the branching for Fedora 21 in 
Rawhide only. We will of course be continuing to do 32bit update builds 
for Fedora 20 and earlier until they go EOL.

According to the current schedule[2] and the planed mass rebuild for 
Fedora 21[3] that means we have to do this before the beginning of June. 
Therefore we're currently aiming at Friday Mai 16 to make this change.

We just wanted to communicate this early so not to surprise anyone and 
give ample time for everyone to look for alternatives.

Yours,
    The Fedora Secondary Arch Team for Power

[1] http://fedoraproject.org/wiki/Koji/ServerHowTo
[2] https://fedoraproject.org/wiki/Releases/21/Schedule
[3] https://fedorahosted.org/rel-eng/ticket/5877

-- 
Philipp Knirsch              | Tel.:  +49-711-96437-470
Manager Core Services        | Fax.:  +49-711-96437-111
Red Hat GmbH                 | Email: Phil Knirsch <pknirsch at redhat.com>
Wankelstrasse 5              | Web:   http://www.redhat.com/
D-70563 Stuttgart, Germany