Fedora 21 Alpha RC testing request

[Adam Williamson]

Hi, folks!

In case anyone didn't see it elsewhere, we put 21 Alpha RC1 out for
testing today. It still has some kinks, but we really need to make sure
we have a reasonably complete test run on it and catch any remaining
blockers. If folks could help test particularly in their own areas of
interest, that'd be great.

The images can be found at
https://dl.fedoraproject.org/pub/alt/stage/21_Alpha_RC1/ - it's a fairly
big tree and the folder layout is still a bit of a work-in-progress,
sorry if you have to click around until you find something.

The test pages are:

https://fedoraproject.org/wiki/Test_Results:Fedora_21_Alpha_RC1_Install
https://fedoraproject.org/wiki/Test_Results:Fedora_21_Alpha_RC1_Base
https://fedoraproject.org/wiki/Test_Results:Fedora_21_Alpha_RC1_Desktop
https://fedoraproject.org/wiki/Test_Results:Fedora_21_Alpha_RC1_Server

to test, basically, pick a test case from (one of) the table(s), run it,
and edit your result into the appropriate table cell using the
{{result}} macro - short version, {{result|pass|adamwill}} is a pass,
{{result|warn|adamwill|123456}} is a warning (aka the compromise between
'pass' and 'fail' - use it for things that aren't exactly critical fails
but are worth noting), and {{result|fail|adamwill|654321}} is a failure.
The number should be a Bugzilla bug #.

Please file bugs for failures; if the bug's serious, propose it as a
release blocker by setting it to block the bug 'AlphaBlocker' or using
the blocker bug webapp at
https://qa.fedoraproject.org/blockerbugs/propose_bug . Each test case
has a link to the release criteria that it enforces at the top - you can
refer to this in proposing the blocker. But please err on the side of
nominating bugs as blockers, we'd rather have a few to reject than miss
one we should have blocked for. Rejections do not go on your permanent
record :)

The test cases have milestones listed alongside them in the tables. Do
the tests marked 'Alpha' first, then the ones marked 'Beta', then the
ones marked 'Final', and the ones with no milestone come last (these are
optional tests that aren't usually expected to encounter release
blocking issues).

Later composes may follow - please do help test those too. We need to
run all the Alpha tests against at least one of the release composes
(ideally we'd run them all against all the RCs).

Thanks a lot folks!

[Haïkel]

Hello guys,

I have been testing the base & atomic images on openstack, so far, so
good ! I'll do later libvirt.
> From last meeting, I understand that jsmith will be handling the testing on AWS.

Few things:
* matrix has not been duplicated for Atomic, since Base and Atomic are
*very* different images, it should
* sshd is not a good use case for base service manipulation test case
(really ;) )

Regards,
H.

[Mike Ruckman]

On Wed, Sep 17, 2014 at 11:33:07AM +0200, Haïkel wrote:
> Hello guys,
>
> I have been testing the base & atomic images on openstack, so far, so
> good ! I'll do later libvirt.
> From last meeting, I understand that jsmith will be handling the testing on AWS.
>
> Few things:
> matrix has not been duplicated for Atomic, since Base and Atomic are
>
> *very* different images, it should
> sshd is not a good use case for base service manipulation test case
>
> (really ;) )
>
> Regards,
> H.

Are the same testcases used in base applicable to Atomic images? If they are
then we can just add a column to the Base validation page. However, if there
should be new testcases created to cover the rest of Atomic, then we should 
go ahead and write those and perhaps create a different page for Atomic like
I've created for cloud [0].

I know that we'll block on the base cloud image, but is my understanding 
correct that Atomic isn't something we'd block release on?

[Mike Ruckman]

On Wed, Sep 17, 2014 at 11:33:07AM +0200, Haïkel wrote:
> Hello guys,
>
> I have been testing the base & atomic images on openstack, so far, so
> good ! I'll do later libvirt.
> From last meeting, I understand that jsmith will be handling the testing on AWS.
>
> Few things:
> matrix has not been duplicated for Atomic, since Base and Atomic are
>
> *very* different images, it should
> sshd is not a good use case for base service manipulation test case
>
> (really ;) )
>
> Regards,
> H.

And the link I forgot to put at the bottom of the email :)

https://fedoraproject.org/wiki/Test_Results:Fedora_21_Alpha_RC1_Cloud

[Haïkel]

On a functional POV, most test cases are the same.

I'm in favour of considering Atomic as a release blocker but Joe and Colin
may have arguments not to.

H.

[Joe Brockmeier]

On 09/17/2014 10:35 AM, Haïkel wrote:
> I'm in favour of considering Atomic as a release blocker but Joe and
> Colin may have arguments not to.

As a release blocker for Alpha or final?

I don't want to block the alpha if we don't have it ready (but I think
we do) but absolutely want this to be a blocker for the final.

Best,

jzb

[Haïkel]

2014-09-17 19:05 GMT+02:00 Joe Brockmeier :
> As a release blocker for Alpha or final?

Both, as alpha release requirements are much less restrictive but I'm
open on the alpha side.
> I don't want to block the alpha if we don't have it ready (but I think
> we do) but absolutely want this to be a blocker for the final.
>
> Best,
>
> jzb

Works for me, and *YES*, it should block the final release !
> --
> Joe Brockmeier | Principal Cloud & Storage Analyst
> [e-mail] | http://community.redhat.com/
> Twitter: @jzb  | http://dissociatedpress.net/
>
> cloud mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/cloud
> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

[Mike Ruckman]

On Wed, Sep 17, 2014 at 09:00:58PM +0200, Haïkel wrote:
> 2014-09-17 19:05 GMT+02:00 Joe Brockmeier :
> > As a release blocker for Alpha or final?
>
> Both, as alpha release requirements are much less restrictive but I'm
> open on the alpha side.
> > I don't want to block the alpha if we don't have it ready (but I think
> > we do) but absolutely want this to be a blocker for the final.
> >
> > Best,
> >
> > jzb
>
> Works for me, and *YES*, it should block the final release !

We (mattdm, adamw and myself) discussed this in #cloud and then also in the 
#atomic meeting. There's already a lot of stuff to validate for the F21 release
with the current load of products to be tested. As it's written now in the
Changes page on the wiki [0] "If all fails, there simply won't be a Fedora 
Atomic Cloud Image product for F21." Trying to slip this in after the fact will
just make it harder for us to get a 2014 Fedora release.

The course of action we landed on for the Atomic image was to use the rest of 
this release cycle to come up with the docs and processes we need in order to 
have Atomic as a release blocking image for F22.

tl;dr: Atomic won't block for F21 release.

I've started a wiki space to plan this bit out [1] (feel free to move it), and 
dustymabe has offered to help come up with testcases and I'll be doing the 
same as I get time to devote to it. The more people to work on writing the docs
we need the more solid it will be for F22.

Thanks!

[0] - https://fedoraproject.org/wiki/Changes/Atomic_Cloud_Image
[1] - https://fedoraproject.org/wiki/User:Roshi/QA/Cloud_Docs#Atomic.2FDocker

[Jared K. Smith]

On Wed, Sep 17, 2014 at 5:33 AM, Haïkel  wrote:
> From last meeting, I understand that jsmith will be handling the testing
> on AWS.

I said that I'd be happy to do some testing on AWS images, but wasn't
specifically talking about the RC1 images.  I would be happy to test the
RC1 images today, as long as I can get the right AWS credentials from
Matthew to do so.

--
Jared Smith

[Haïkel]

Le 17 sept. 2014 17:25, "Jared K. Smith"  a écrit
:
> On Wed, Sep 17, 2014 at 5:33 AM, Haïkel  wrote:
> > From last meeting, I understand that jsmith will be handling the testing

on AWS.
> I said that I'd be happy to do some testing on AWS images, but wasn't

specifically talking about the RC1 images.  I would be happy to test the
RC1 images today, as long as I can get the right AWS credentials from
Matthew to do so.
>

My apologies for the misunderstanding. I would be grateful if you did when
you'll get the credentials.

H.
> --
> Jared Smith
>
> cloud mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/cloud
> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct