Need owner of Atomic 2 Week releases

[Michael McGrath]

On Fri, Nov 13, 2015 at 12:56 PM, Michael McGrath <mmcgrath at redhat.com> wrote:
> On Fri, Nov 13, 2015 at 10:50 AM, Josh Boyer <jwboyer at fedoraproject.org> wrote:
>> On Fri, Nov 13, 2015 at 11:35 AM, Dusty Mabe <dusty at dustymabe.com> wrote:
>>>
>>>
>>> On 11/13/2015 10:41 AM, Amanda Carter wrote:
>>>>
>>>> Hey folks, creating a separate thread for this longer term discussion.
>>>> We're getting ready to release our first 2 week atomic update on Tuesday and
>>>> Dusty Mabe has raised 2 potential release blockers that were not part of
>>>> automated testing. It's good that he caught them, but it's also a bit of a
>>>> stroke of luck. Since there is no official QE for this release, who should
>>>> own verifying that there are no release blocking bugs prior to every
>>>> automated release and escalating if there are? If no one raises the blocker,
>>>> we'll have no way to block the release.
>>>>
>>>> This is something that we need an answer to fairly quickly since we don't
>>>> even have confidence that the current release is good other than the current
>>>> reports. And we'll be taking this plunge again in just 2 weeks.
>>>>
>>>> Thanks for your attention to this,
>>>>
>>>
>>> We should really have a blocker review process for these similar to the ones
>>> we have for normal Fedora releases. Primary items that we should be
>>
>> Are these 2 week images built from updates that are currently in
>> stable, or are they built from updates-testing as well?
>>
>> I ask because it matters for things outside of atomic.  The release
>> blocker review process works because it catches things _before_ they
>> are released for general availability.  If the 2 week atomic images
>> are only composed from already stable updates, then the packages are
>> already out there in the project otherwise.
>>
>> So if you have something "blocker" in an atomic image, you are now
>> forced to wait for an update to make it through the entire fedora
>> updates process before you can ship your 2 week image.  For something
>> like the kernel, that might very well mean you don't ship a 2 week
>> image because the fix is not in stable in sufficient time.
>>
>> The atomic images might be better served by doing tests on
>> updates-testing packages that are included to ensure that blockers
>> don't otherwise show up as a surprise.  I would also recommend
>> reaching out to the package owners for each important package in
>> advance so that the atomic sig is aware of what is planned for updates
>> and such.
>>
>> josh
>
>
> I'd also point out that the nature of Atomic's rollback features make
> it better suited to less testing in that they can always roll back.
> The problem that I see comes if there's a bug that lasts through
> several releases causing a fairly bad trust scenario w/ upgrades.
>
> --
> Mike McGrath | mmcgrath at redhat.com | (312) 660-3547
> Atomic | Red Hat Chicago | http://projectatomic.io/

One other thing I forgot to mention, Dennis requested some additional
QA for this to do some basic smoke tests.  We're working on setting
something up that can be used for a basic smoke test until something
more permanent can be setup.  I've cc'd Ari who's familiar.

Ari, can you give a quick update on what your team is thinking at this
point?  Something automated would help greatly for at least one part
of the 2 week release (though bugs would still need to be handled).

-- 
Mike McGrath | mmcgrath at redhat.com | (312) 660-3547
Atomic | Red Hat Chicago | http://projectatomic.io/