Primary Architectures: Another Proposal (RFC)

[Kevin Kofler]

Hi,

let me suggest a new proposal for the primary architecture controversy: 
Let's have exactly one architecture handled as primary in Koji, picked with 
the ONLY aim of making builds as fast as possible. Please note that while 
this may well end up to be x86_64 at least right now, this in principle need 
not be a commonly used architecture, or even a real architecture at all, 
even something like LLVM bytecode might qualify if the gain in build time 
from avoiding the final compilation step outweights the performance loss 
from having to JIT. It might also be, say, x86_64 with some additional 
instruction set extensions (SSE3 etc.), with the actual shipped baseline 
x86_64 packages built as secondary. Anything that speeds up builds is fair 
game. Then build ALL architectures as secondary, no matter how important 
they are (e.g. at least one of the x86 architectures would be treated as 
secondary at build system level, probably both (for consistency, and to 
allow for build speed optimizations different from the packages we want to 
ship to our users)).

This proposal would only affect the build system. All the other 
differentiations between primary and secondary architectures are outside of 
its scope.

Advantages:
* faster builds (no more waiting for all architectures to complete, only the 
fastest need complete)
* fair treatment of all architectures (ARM will be happy because they won't 
be a second-class citizen anymore)

Drawbacks:
?

        Kevin Kofler

[Miloslav Trmač]

On Sun, Apr 8, 2012 at 2:57 AM, Kevin Kofler <kevin.kofler(a)chello.at&gt; wrote:
>  let me suggest a new proposal for the primary architecture
> controversy:
>  Let's have exactly one architecture handled as primary in Koji, picked with
>  the ONLY aim of making builds as fast as possible. 
Interesting idea, having Koji wait only until one of the architectures
were built might be useful in any case.

<snip>
>  This proposal would only affect the build system. All the other
>  differentiations between primary and secondary architectures are outside of
>  its scope. 
How is this in practice different from
a) the package maintainer looking at the build logs of the
fastest-building architecture as soon as that architecture builds, and
ignoring the rest?  (The maintainer can do this right now.)
b) the existing primary architecture definition, where updates,
releases and release criteria wait for all primary architectures?


Specifically, I'm unsure about the interaction of this proposal with
bodhi (can an update be pushed to stable even if only the
fastest-building architecture was built and got karma?), and the
meaning of release deadlines (does the "beta change deadline" count
against the fastest-building architecture or all of them?) - it seems
to me that the natural answer to these questions is, in both cases,
"all architectures must be tested and usable for the update/release to
be published", and I can't see the difference in that case.  Perhaps
you have a different idea of how bodhi and the release dealines would
be treated?
    Mirek

[Kevin Kofler]

Miloslav Trmač wrote:
>  How is this in practice different from
>  a) the package maintainer looking at the build logs of the
>  fastest-building architecture as soon as that architecture builds, and
>  ignoring the rest?  (The maintainer can do this right now.) 
* The build might be faster if we find a way to build packages faster than 
the fastest shipped architecture. (As a simple example, we might try 
building packages with -O1, and stuff often used in builds (toolchain etc.) 
with -O3, whereas the shipped packages would still use -O2 as always.)
* Chain builds would be able to proceed immediately (on the primary 
architecture).
* Updates would be able to be filed as soon as the primary build completes 
(see below).

>  b) the existing primary architecture definition, where updates,
>  releases and release criteria wait for all primary architectures?
>  
>  Specifically, I'm unsure about the interaction of this proposal with
>  bodhi (can an update be pushed to stable even if only the
>  fastest-building architecture was built and got karma?), and the
>  meaning of release deadlines (does the "beta change deadline" count
>  against the fastest-building architecture or all of them?) - it seems
>  to me that the natural answer to these questions is, in both cases,
>  "all architectures must be tested and usable for the update/release to
>  be published", and I can't see the difference in that case.  Perhaps
>  you have a different idea of how bodhi and the release dealines would
>  be treated? 
The idea would be that you'd be able to file and queue the update in Bodhi 
as soon as the primary build completes, just as now, except that "primary" 
would now be the one, possibly fake, architecture. The updates could even 
get pushed right there, for the internal primary architecture only. They 
would then be synced to the actual architectures, just like secondary 
architectures are synced now. It'd be Koji's and/or Bodhi's job to 
automatically push the matching updates for the real architectures to the 
matching repositories as soon as the builds complete. The infrastructure 
needed for that is needed to properly support secondary architectures 
anyway!

        Kevin Kofler

[Jochen Schmitt]

On Sun, Apr 08, 2012 at 02:34:25PM +0200, Kevin Kofler wrote:
>  * Updates would be able to be filed as soon as the primary build
> completes 
>  (see below). 
Question: What should be happen, if the build fails on a architecture
which was not built as fast as the festest build.

I have in mind that the ARM architecture may become a primary architecture and
I can imaging that a package may build on x86_64 find, but fails on
arm. What should we do in this case?

Best Regards:

Jochen Schmitt

[Björn Persson]

Kevin Kofler wrote:
>  The idea would be that you'd be able to file and queue the update
> in Bodhi 
>  as soon as the primary build completes, just as now, except that "primary" 
>  would now be the one, possibly fake, architecture. The updates could even 
>  get pushed right there, for the internal primary architecture only. 
So what it boils down to is that you wish you didn't have to wait for the 
build to complete in Koji before you submit the update in Bodhi, and you're 
proposing this big waste of build server resources as a way of reducing the 
wait a bit. Did you consider that the wait may actually become longer as the 
build servers will be much busier when they have to build both the real 
packages and the build-speed-optimized throwaway packages?

If the Fedora community agrees that it would be good to be able to submit an 
update before the build has completed, then the right solution is to change 
the user interface of Bodhi to allow that. An even better solution might be a 
combined interface where you could start a set of builds and submit the update 
information as a single operation.

Björn Persson

[Kevin Kofler]

Björn Persson wrote:
>  So what it boils down to is that you wish you didn't have to wait
> for the
>  build to complete in Koji before you submit the update in Bodhi 
Not really, because I have no idea whether it will build in the first place 
before the build is complete (on any architecture)!

        Kevin Kofler

[Josh Boyer]

On Sun, Apr 8, 2012 at 6:25 PM, Kevin Kofler <kevin.kofler(a)chello.at&gt; wrote:
>  Björn Persson wrote:
> > So what it boils down to is that you wish you didn't have to wait for the
> > build to complete in Koji before you submit the update in Bodhi
>
>  Not really, because I have no idea whether it will build in the first place
>  before the build is complete (on any architecture)! 
Because you, as the maintainer of the package in question, aren't
building and testing the package locally before you submit the official
build?  Or you aren't leveraging scratch builds to do the same if your
local hardware is prohibitively slow?

I don't think we need to encourage more "throw it at the wall and see
if it sticks" maintainership.

josh

[Kevin Kofler]

Josh Boyer wrote:
>  Because you, as the maintainer of the package in question,
> aren't
>  building and testing the package locally before you submit the official
>  build?  Or you aren't leveraging scratch builds to do the same if your
>  local hardware is prohibitively slow? 
The local or scratch build is just a wasted build, then I have to redo the 
build as an official build, so I have to do an extra build for no reason. If 
I do all the builds as official and stick with the one that builds, that 
saves the final extra build.

>  I don't think we need to encourage more "throw it at the
> wall and see
>  if it sticks" maintainership. 
Why? It saves a build that'd otherwise be wasted, saving my time. It also 
doesn't preclude downloading and testing the output once it builds.

        Kevin Kofler

[Josh Boyer]

On Mon, Apr 9, 2012 at 9:55 PM, Kevin Kofler <kevin.kofler(a)chello.at&gt; wrote:
>  Josh Boyer wrote:
> > Because you, as the maintainer of the package in question, aren't
> > building and testing the package locally before you submit the official
> > build?  Or you aren't leveraging scratch builds to do the same if your
> > local hardware is prohibitively slow?
>
>  The local or scratch build is just a wasted build, then I have to redo the
>  build as an official build, so I have to do an extra build for no reason. If
>  I do all the builds as official and stick with the one that builds, that
>  saves the final extra build. 
"It builds" doesn't necessarily mean it works.  In the rawhide case,
you might very well submit something that builds but is totally
broken.  By the time you download the official build and test it
locally, it can already be in the day's compose and now you've just
inflicted a broken build on all the users.  And no, "rawhide eats
babies" is not a valid excuse IMHO.

In a stable or branched release, your method would work as the way
we use koji tags with -candidate prevents it from going directly to
users in a compose.

> > I don't think we need to encourage more "throw it at the
> wall and see
> > if it sticks" maintainership.
>
>  Why? It saves a build that'd otherwise be wasted, saving my time. It also
>  doesn't preclude downloading and testing the output once it builds. 
An ounce of prevention.  Personally, I'd rather "waste" a small bit of
my time to save inflicting broken stuff on users.  I can always work
on something else while that particular build is running.

josh

[Kevin Kofler]

Josh Boyer wrote:
>  "It builds" doesn't necessarily mean it works.  In the
> rawhide case,
>  you might very well submit something that builds but is totally
>  broken.  By the time you download the official build and test it
>  locally, it can already be in the day's compose and now you've just
>  inflicted a broken build on all the users.  And no, "rawhide eats
>  babies" is not a valid excuse IMHO. 
As far as Rawhide goes, IMHO, if it builds, it's good enough for Rawhide.

"If it compiles, ship it!" – Linus Torvalds

        Kevin Kofler

[Horst H. von Brand]

Kevin Kofler <kevin.kofler(a)chello.at&gt; wrote:
>  Josh Boyer wrote:
>  > "It builds" doesn't necessarily mean it works.  In the rawhide case,
>  > you might very well submit something that builds but is totally
>  > broken.  By the time you download the official build and test it
>  > locally, it can already be in the day's compose and now you've just
>  > inflicted a broken build on all the users.  And no, "rawhide eats
>  > babies" is not a valid excuse IMHO.
>  
>  As far as Rawhide goes, IMHO, if it builds, it's good enough for Rawhide. 
That is just irresponsible. BTW, there are too few rawhide consumers as
things stand; this would make rawhide be russian roulette, but with 5
bullets instead of 1.

>  "If it compiles, ship it!" – Linus Torvalds 
Riiight... one tongue-in-cheek comment by the well-known punster Linus
leads to distribution wide policy.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                    Fono: [phone]
Universidad Tecnica Federico Santa Maria             [phone]
Casilla 110-V, Valparaiso, Chile 2340000       Fax:  [phone]

[Kevin Kofler]

Horst H. von Brand wrote:
>  That is just irresponsible. BTW, there are too few rawhide consumers
> as
>  things stand; this would make rawhide be russian roulette, but with 5
>  bullets instead of 1. 
Rawhide IS already Russian roulette. ;-)

If you use it, you get to keep the pieces.

        Kevin Kofler

[Al Dunsmuir]

On Tuesday, April 10, 2012, 11:27:59 AM, Kevin Kofler wrote:
>  Horst H. von Brand wrote:
> > That is just irresponsible. BTW, there are too few rawhide consumers as
> > things stand; this would make rawhide be russian roulette, but with 5
> > bullets instead of 1. 
>  Rawhide IS already Russian roulette. ;-) 
Perhaps  the complaint is that you may be loading all cylinders in the
gun with live rounds?

[Horst H. von Brand]

Kevin Kofler <kevin.kofler(a)chello.at&gt; wrote:

>  let me suggest a new proposal for the primary architecture
> controversy: 
>  Let's have exactly one architecture handled as primary in Koji, picked with 
>  the ONLY aim of making builds as fast as possible. 
[...]

>  Advantages:
>  * faster builds (no more waiting for all architectures to complete, only the 
>  fastest need complete)
>  * fair treatment of all architectures (ARM will be happy because they won't 
>  be a second-class citizen anymore)
>  
>  Drawbacks: 
Nobody will _run_ that, so no testing other that "it builds".
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                    Fono: [phone]
Universidad Tecnica Federico Santa Maria             [phone]
Casilla 110-V, Valparaiso, Chile 2340000       Fax:  [phone]

[Ralf Ertzinger]

Hi.

On Sat, 07 Apr 2012 22:35:01 -0300, Horst H. von Brand wrote

>  Nobody will _run_ that, so no testing other that "it
> builds". 
if I understand Kevin correctly nobody is supposed to run it.
It's just used as a fast-fail check for compilability (if that's
a word).

It's exactly the same information you currently get from a successful
koji build, just hopefully faster.

-- 
The I.S.O. standard unit of female pulchritude is the milli-helen. This
is the amount of beauty capable of causing the launching of a single ship.

[drago01]

On Sun, Apr 8, 2012 at 2:57 AM, Kevin Kofler <kevin.kofler(a)chello.at&gt; wrote:
>  Hi,
>
>  let me suggest a new proposal for the primary architecture controversy:
>  Let's have exactly one architecture handled as primary in Koji, picked with
>  the ONLY aim of making builds as fast as possible. 
Even though fast builds are nice to have we should not make that the
primary goal and base everything around it.