kmod arch, ExlusiveArch, and buildsys?

[Warren Togami]

Hi Dan,

I am supposed to ask you about your opinion about how to ask buildsys to 
build for specific archs for a kernel module.  I believe that we should 
use ExclusiveArch with explicit listed archs, and buildsys should loop 
through each listed arch and build each.  This would also work for 
multi-arch performance intensive userspace packages similar to what we 
do in glibc or openssl in Core. [1]  Contributors may also list any 
other arch like s390x or alpha in ExclusiveArch, and buildsys simply 
ignores it if it isn't supported in Extras.

I believe this is the best approach to take for Extras, because Core has 
been similar to this for many years.  Do you have any opinion on this, 
or know of a better way to handle this?

Here is some related discussion:
<scop> the problem with exclusivearch is that let's say you list i586 
i686 x86_64 ppc and someone wants to rebuild it for alpha -> won't 
build, even if the package/software would work
<warren> scop, you can include other archs in the list, buildsys can 
simply ignore Extras non-supported archs.
<scop> warren, it's abuse of the tag nevertheless
<warren> scop, I'd personally disagree, but either way this is very 
simple and it already behaves similar to this in Core literally forever.
<warren> more importantly the simplicity of this allows us to move 
forward sooner
<warren> (assuming dcbw doesn't have a better idea)
<scop> that I agree with

Warren Togami
wtogami(a)redhat.com

[1]
However we should disallow this, unless someone proves through 
benchmarks that there is a huge benefit of publishing both i386 and i686 
packages and our tools support selection of the right arch directly. 
Currently I believe it doesn't.

[Mike A. Harris]

Warren Togami wrote:
>  Hi Dan,
>  
>  I am supposed to ask you about your opinion about how to ask buildsys to 
>  build for specific archs for a kernel module.  I believe that we should 
>  use ExclusiveArch with explicit listed archs, and buildsys should loop 
>  through each listed arch and build each.  This would also work for 
>  multi-arch performance intensive userspace packages similar to what we 
>  do in glibc or openssl in Core. [1]  Contributors may also list any 
>  other arch like s390x or alpha in ExclusiveArch, and buildsys simply 
>  ignores it if it isn't supported in Extras.
>  
>  I believe this is the best approach to take for Extras, because Core has 
>  been similar to this for many years.  Do you have any opinion on this, 
>  or know of a better way to handle this?
>  
>  Here is some related discussion:
>  <scop> the problem with exclusivearch is that let's say you list i586 
>  i686 x86_64 ppc and someone wants to rebuild it for alpha -> won't 
>  build, even if the package/software would work
>  <warren> scop, you can include other archs in the list, buildsys can 
>  simply ignore Extras non-supported archs.
>  <scop> warren, it's abuse of the tag nevertheless
>  <warren> scop, I'd personally disagree, but either way this is very 
>  simple and it already behaves similar to this in Core literally forever.
>  <warren> more importantly the simplicity of this allows us to move 
>  forward sooner
>  <warren> (assuming dcbw doesn't have a better idea)
>  <scop> that I agree with 
The alternative tag is ExcludeArch.  It's sometimes better to use
that, and simply exclude the arches we do _not_ want to build for,
but which there are machines that would otherwise go ahead and build
the packages.

The approach I've taken for the Xorg driver rpms is to use the
ExclusiveArch tag, and pre-fill them with the upstream default
drivers that got built on each arch previously, modulo any
unintentional ommisions/etc.  I covered sparc/sparc64/alpha as
well.

Using this mechanism to solve this type of problem automatically
has a hard coded assumption that you want to either include all
by default and then blacklist individual ones, or you want to
blacklist all by default and then include individual ones.

On a side note, except under very specific circumstances, do not
use "i386" or "i686" as an architecture to list in ExclusiveArch
or Excludearch.  Use the rpm macro "%{ix86}" instead, as that
expands to all existing architecture aliases past, present and
future.  Only use i386 or i686 et al. if you really do want to
intentionally restrict the package from being built on other x86
compatible architectures, such as i486 for example.

Keeping this data _in_ the rpms means that the package itself is
in sync with what arches it should be building on.  No external
synchronization is necessary.  Trying to store such information
somewhere else, and let the layer above take care of it (the
buildsystem, or the tree maker software, leaves things open to
be desynchronized, etc.  It also leaves more room for errors to
happen, such as a database being changed accidentally or restored
from backup and losing changes.


-- 
Mike A. Harris  *  Open Source Advocate  *  http://mharris.ca
                       Proud Canadian.

[Dan Williams]

On Fri, 2006-02-17 at 15:41 -0500, Mike A. Harris wrote:
>  Warren Togami wrote:
>  > Hi Dan,
>  > 
>  > I am supposed to ask you about your opinion about how to ask buildsys to 
>  > build for specific archs for a kernel module.  I believe that we should 
>  > use ExclusiveArch with explicit listed archs, and buildsys should loop 
>  > through each listed arch and build each.  This would also work for 
>  > multi-arch performance intensive userspace packages similar to what we 
>  > do in glibc or openssl in Core. [1]  Contributors may also list any 
>  > other arch like s390x or alpha in ExclusiveArch, and buildsys simply 
>  > ignores it if it isn't supported in Extras.
>  > 
>  > I believe this is the best approach to take for Extras, because Core has 
>  > been similar to this for many years.  Do you have any opinion on this, 
>  > or know of a better way to handle this?
>  > 
>  > Here is some related discussion:
>  > <scop> the problem with exclusivearch is that let's say you list i586 
>  > i686 x86_64 ppc and someone wants to rebuild it for alpha -> won't 
>  > build, even if the package/software would work
>  > <warren> scop, you can include other archs in the list, buildsys can 
>  > simply ignore Extras non-supported archs.
>  > <scop> warren, it's abuse of the tag nevertheless
>  > <warren> scop, I'd personally disagree, but either way this is very 
>  > simple and it already behaves similar to this in Core literally forever.
>  > <warren> more importantly the simplicity of this allows us to move 
>  > forward sooner
>  > <warren> (assuming dcbw doesn't have a better idea)
>  > <scop> that I agree with 
>  On a side note, except under very specific circumstances, do not
>  use "i386" or "i686" as an architecture to list in ExclusiveArch
>  or Excludearch.  Use the rpm macro "%{ix86}" instead, as that
>  expands to all existing architecture aliases past, present and
>  future.  Only use i386 or i686 et al. if you really do want to
>  intentionally restrict the package from being built on other x86
>  compatible architectures, such as i486 for example. 
Right, here's my take.

In the end, it might just be best to leave this up to the buildsystem,
since there's a "canonical" list of arches that a particular Distro
maintains.  There's two factors here:

1) The list of arches a _distro_ supports
2) The list of arches a particular package supports

The two are not identical.  And even though, for example, all of FC3 was
compiled for ppc internally, the original FC3 was not released for ppc
at all, and so therefore kernel modules should be released for FC3/ppc
either.  Control over the package set that the distro supports is up to
the distro maintainers and the build system.

So what I think we should end up doing here is to:

a) Let packages do whatever the hell they want with their Exclusive,
Exclude, BuildArch tags, including using %{ix86} as Mike suggests
b) Have the buildsystem recognize kmod packages somehow (which we have
to do anyway), then filter kmod packages through a "supported" list of
sub-arches, including i586, i686, x86_64, ppc, athlon.  There's some
support for this already in the buildsystem.

This puts a minimum burden on maintainers since lots of this arch stuff
is black magic anyway, and should probably be kept in the same place.
The nobody gets confused about arches, and there doesn't need to be a
50ft long wiki page about it.

Of course, all this is completely separate from the whole
xen/hugemem/smp/up fiasco, and much simpler IMHO than those.

Let me know what you think.

Dan