ARM as a primary architecture

[Peter Robinson]

On Tue, Mar 20, 2012 at 6:10 PM, Jesse Keating <jkeating at redhat.com> wrote:
> On 3/20/12 9:30 AM, Jon Masters wrote:
>>
>> Hi again,
>>
>> I want to thank you, and everyone else in FESCo for talking with us
>> yesterday, and for looking over the proposal. Bear in mind, it's a work
>> in progress. We intend to have broader conversations over the coming
>> months and F18 is an optimistic goal. Nonetheless, I feel it is
>> achievable (we've done many more disruptive things!) but we have also
>> many points along the way at which we can back out, and remain SA.
>>
>> To address a few of these points...at least, I'll try :) First, just to
>> repeat, we took the proposal to FESCo yesterday in the spirit of "early
>> and often", not in the spirit of "final deliverable". Therefore, while
>> it is true we've not yet reached out to everyone for input, that is
>> because it is still a work in progress for us. We'll iterate based on
>> feedback, and based upon yesterday and reaching out to other groups.
>>
>> On 03/20/2012 11:52 AM, Peter Jones wrote:
>>
>>> 1) mechanisms need to be in place to get package maintainers access to
>>> fix
>>>     arm-specific bugs in their packages
>>
>>
>> So we have a tracker bug at the moment. Is that sufficient? If so, we
>> obviously should make sure that it is included in the proposal docs that
>> we have this in place and are using it.
>
>
> A tracker bug is certainly not sufficient.  It would be for a SA, but not
> PA.  Typically our users have a PA at their disposal, or failing that have
> ready access to a shared PA provided by the Fedora Project that they can log
> into and do their testing.

How was this handled in the case of PPC? My understanding is that due
to legal reasons the Fedora Project never officially provided access
to PPC machines. There were a number of machines that users could get
access to that were provided by individuals but these were never
officially provided by the Fedora project.

> Without ARM systems available for all the releases our maintainers have to
> support this is a non-starter.  I will also note that having to resort to
> using a remote system because the arch isn't generally locally at a
> maintainer's disposal /is/ going to introduce a delay in getting bugs
> resolved and builds out the door.  If the arch was ubiquitous in a way that
> lent itself to easy debugging and use that'd be a different matter, but I
> just don't see it as being there right now.

There's a number of cheap hardware becoming available such as the
Raspberry Pi as well as development boards that are available for
either purchase or people can apply to be part of a developer program
to get either discount or free hardware. How was this supported with
PPC? The PPC hardware was a lot more expensive (either Apple devices
or IBM) than the readily available ARM devices. We can also put a
means escalation in place too for those that don't want to purchase or
can't get ready access to HW for testing.

>>> 2) excludearch is not an option.  This is fundamentally contrary to being
>>>     a primary arch. In fact this is one of the defining characteristics
>>> of
>>>     a secondary arch.
>>
>>
>> Let's think about this some. ARM (32-bit) doesn't do Intel-style
>> microcode, MCE, or many other things that x86 does. I don't think that
>> means we should build packages that are x86-specific for ARM systems. We
>> generally believe that we're starting from a point of good momentum, but
>> there are some packages that won't ever be appropriate for ARM, and
>> there are others where the FTBFS has been longstanding, or there are
>> other (IMO valid) reasons why it might initially be Exclude. That
>> doesn't mean that there would be many such cases. Nonetheless, I think
>> it would be unreasonable to set the entry bar so high that there can be
>> no things left to fix. That basically retains the x86 monopoly.
>
>
> Perhaps Peter can clarify or soften this requirement a bit.  EXCLUDEARCH as
> a default action when a build fails on ARM is certainly not an option.  What
> would help your situation is generating a few lists of packages.  One list
> would be packages that you feel just don't make sense on ARM.  Another list
> would be the FTBFS you mention.  These lists can be debated and decided upon
> /before/ the migration to PA and the ExcludeArches can be in place before
> the switch is pulled.

There's a couple of different categories here.

1) x86 only hardware. There's things like dmidecode, cpuid, various
ACPI, numa, EFI and other HW specific things like intel GPU drivers
where it just doesn't make sense to build on ARM, just like it didn't
make sense to build the various PPC utils etc on x86. In some cases it
might be a short term exclusion as it's expected that the support will
come to ARM, EFI is the classic case here. Others like intel GPUs
never will.

2) packages that have x86 dependent code. spice comes into this
category and I've discovered a few others. This would need work from
someone, either the Fedora maintainer or upstream.

3) Packages that aren't supported with ARM upstream. In most cases
they're fairly old releases. Most of the ones in this category had
their last releases back in the early 2000s. There's no reason they
can't be built but they would need some attention, I don't see an
issue here as in most of these cases Fedora is already carrying a
series of patches for things like compiling on gcc 4.x anyway, it's
just a matter of time and most of these we're fixing as we go.

Ultimately as the person that has done 98% of the builds and lead the
building of rawhide the vast majority of the packages where we've
added ExcludeArch are where they are x86 or PPC only for a reason, in
fact in a lot of cases I've removed excludes (icedtea-web and a number
of other packages) to ensure we run on ARM. Where it's FTBFS on ARM
there's been bug reports and dialog with maintainers to get the issues
fixed. Most maintainers have actively assisted with this, some have
stated they don't care, in those cases we generally go and fix the
issues ourselves. In quite a few of the cases with build issues it's
actually people doing weird shit in spec files that cause the problem.

> Once the switch is pulled, that's when excludearch is unacceptable without
> FESCo or such involvement, just like it is with the other primary arches.
>  That's really a price of entry.

The ARM SIG has never seen excludearch as a fix to a non HW issue so
we have no problems with that. It some what amuses me actually that
there seems to be an expectation that we've been "fixing" things by
using excludearch. This is not, and never has been, the case. This is
a price we've been well and truly aware of from the start and it's
something that I as the lead pushing the builds have never seen
excludearch as a solution.

>>> 3) arm must be integrated to the formal release criteria
>>
>>
>> Agreed. We'll need to figure that out.
>>
>>> 4) when milestones occur, arm needs to be just as testible as other
>>>     primary architectures
>>
>>
>> So we have a new hire (hi Paul) who is looking at autoqa, and we're
>> going to pull together as much as we can here. It would help me to know
>> (and we're reaching out to QE separately - per my other mail) what you
>> would consider "testable" to mean, in terms of what you'd want to see.
>
>
> I think what is meant here is that we have to be able to get the arm version
> of the rpms installed onto an appropriate host and be able to easily run the
> programs to ensure that things are working as expected, more than just "It
> built, SHIP IT".

Absolutely! QA have already started to put together a proposal, in
fact around FUDCon NA timeframe and long before this proposal for
adding it as a primary arch. It's my opinion that building the
packages is the small and easy part of the process.

https://fedorahosted.org/fedora-qa/ticket/277

> Take a look at the release criteria we have currently have in place for
> alpha/beta/final.  ARM will need to follow it or have something extremely
> similar.  Granted the release criteria mostly involves the installation
> process, but it does include installs from live media. Installs /to/ a SD
> device and then booting said install to validate it could fit in there.

Exactly and it's what we wish to do. Ultimately there is a  lot of
work to be done on installers and install process. We were planning on
breaking this down into essentially 3 phases:
1) livecd-creator style command line tool to create a SD card or
similar from a livecd style image, setup the bootlaoder so you can
plug a card into a device and have it boot to the desired interface.
2) liveusb-creator style GUI to make the above much easier
3) Full anaconda support, we started talking with the anaconda team
about this as far back as FUDCon Milan. They have a lot of awesome
ideas surrounding the means of supporting installing these devices but
ultimately like everything else it needs to get done. Hence the
reasons we were looking at doing it in a phased approach.

Peter