Developing secondary architecture release criteria

[James Laska]

Greetings folks,

I'm interested in developing a solution to allow Fedora secondary
architectures [1] (specifically ppc64 and s390x for now) to leverage our
existing release criteria [2].  This isn't really a tremendously
difficult technical challenge, more of a content organization conundrum.
The part I'm wrestling with is how to do it in a way that it compliments
the current release criteria process.  I apologize in advance for this
rather long email.

For some background, here are some key differences between primary and
secondary architectures.  These are worth noting as I believe they'll
impact how the criteria are created and applied.

     1. Different release schedule - secondary architectures don't
        follow the same release schedule, so secondary architecture
        blocker/NTH bugs shouldn't block the main release.  This is good
        since we have enough primary architecture blocker bugs
        already :)
     2. Different target audience - Depending on the architecture,
        different outcomes may apply for certain criteria.  For example,
        on s390x ... there is no Desktop outside of using VNC.
        Therefore, the graphical boot, firstboot, login don't apply and
        many remaining desktop criteria are NTH, not blockers.
     3. Community size - I don't think this is a surprise, but the
        secondary architectures tend to have smaller communities of
        interest.  That's not to say there isn't interest, just the the
        number of people involved/contributing may be substantially
        smaller than the primary architectures.  Any solution needs to
        be sustainable by+for that community.

Okay, recognizing the differences ... it might help to outline what I'm
hoping to accomplish.  Some goals include ..

     1. Do _something_ - better than doing nothing :)
     2. Minimal criteria duplication - I don't want anyone to have to
        write/maintain the same criteria in multiple places.  That just
        seems like the worst-case-outcome.
     3. Low maintenance - somewhat related to the above goal, but not
        entirely.  Whatever solution we choose, I'm hoping it isn't a
        hard-to-document wiki challenge.
     4. Minimal impact to primary release criteria + process -
        personally I'm pretty happy with the current criteria
        +process ... I don't think that's in need of a major overhaul at
        this time.

To meet these goals, here are some solutions I've been thinking about so
far.  Note, not all of these are good ideas, I'm just thinking outloud
in the hopes of getting creative juices flowing.  If something doesn't
make sense, or is wrong ... don't hesitate to point it out.

     1. Full duplication - Draft individual wiki pages for each arch and
        each milestone (Alpha, Beta, Final).  Each page would outline
        custom criteria for that architecture+release pair.
              * PROS - are there any?  I guess this solution offers the
                most  customization for each secondary architecture
              * CONS - ugh, painful to create, maintain and keep in sync
                all that wiki content and pages.  This definitely
                competes with the "minimal duplication" goal.
     2. Highlight differences - Draft individual wiki pages for each
        arch for each milestone (Alpha, Beta, Final).  However, instead
        of duplicating all applicable primary arch content, only
        highlight the differences between the primary and secondary
        arch. 
              * PROS - A little less work than the full-duplication
                approach
              * CONS - Still a lot of wiki maintenance.  Not necessarily
                in terms of wiki content, but definitely in terms of
                wiki pages.  Having plenty of experience trying to
                figure out whether a but impacts criteria, I think this
                creates additional hoops to jump through to figure out
                if a bug impacts the criteria or not.
     3. Wiki Templates - Draft individual wiki pages for each arch for
        each milestone (Alpha, Beta, Final).  However, instead of
        creating new criteria, or highlighting exceptions, simply call
        the common template (with any optional template arguments).
        This also involves converting the existing criteria into
        official mediawiki templates (e.g.
        Template:Fedora_16_Alpha_Release_Criteria).  All existing
        criteria pages would then call the main template, providing any
        template parameters needed to display/not_display specific
        criteria.  A *small* sample of how this might work is available
        at [3].
              * PROS - One stop shopping for each architecture when
                trying to figure out if a bug impacts the criteria.  I
                don't need to reference the ppc64 criteria, and then the
                primary criteria.  It's all on the single page for each
                milestone (alpha, beta, final).  Note, templates are
                already used in the existing criteria to avoid
                duplicating the pre-amble, so this isn't too unusual.
              * CONS - a fair amount of prep work creating the template
                and parameterizing different sections.  I'm also not yet
                sure what parameters would be needed (perhaps they'll
                grow over time)?
     4. Inline exceptions - No new pages created, simply modify the
        existing criteria pages to indicate which arches criteria apply
        for (example at [4])
              * PROS - No new wiki pages to create, just adding content
                to the existing criteria pages.  *Much* less maintenance
                in terms of total number of wiki pages.
              * CONS - I like that the current criteria pages are fairly
                small in size.  I'd worry that adding criteria for all
                arches in to a single page would get too cluttered.
                Additionally, it's more than just the criteria lists
                that are impacted by the architecture (e.g. the
                contingency plan).  So some additional restructuring may
                be needed.
     5. Something else, something simple?

At this point, I'm leaning towards seeing what solution#3 looks like.
If that turns out to be horrible, perhaps choosing between #2 and #4.
Of course the reason I'm sending this email is to find better ideas or
discover any pros/cons not highlighted above.  Comments encouraged!

Thanks,
James

[1] https://fedoraproject.org/wiki/Architectures#Secondary_Architectures
[2] https://fedoraproject.org/wiki/Fedora_Release_Criteria
[3] http://fpaste.org/yM9U/raw/
[4]
https://fedoraproject.org/wiki/User:Jlaska/Draft:Fedora_16_Alpha_Release_Criteria_%28inline%29
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
Url : http://lists.fedoraproject.org/pipermail/test/attachments/20110707/6440ef1b/attachment-0001.bin