Release validation testers: a question

[Adam Williamson]

Hi folks! I've got a question for everyone who helps out with release
validation testing. It's related to the project to build a new system
for submitting those results. We have a ticket for designing that
system:

https://pagure.io/design/issue/483

and as part of the discussion there, we're wondering something. Given
that it's part of the new design that there's some kind of step where
you specify which particular image you're testing, does it make more
sense to you if you choose things in this order:

1. Pick a test to run
2. Pick an image to run the test on

Or this order:

1. Pick an image to test
2. Pick a test to run on the image

? It's pretty important to the design approach. Thanks!

(Please do feel free to read the ticket and mention any other thoughts
you have on the design, too.)
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Chris Murphy]

On Tue, Feb 14, 2017 at 1:02 PM, Adam Williamson
<adamwill(a)fedoraproject.org&gt; wrote:
>  Hi folks! I've got a question for everyone who helps out with
> release
>  validation testing. It's related to the project to build a new system
>  for submitting those results. We have a ticket for designing that
>  system:
>
>  https://pagure.io/design/issue/483
>
>  and as part of the discussion there, we're wondering something. Given
>  that it's part of the new design that there's some kind of step where
>  you specify which particular image you're testing, does it make more
>  sense to you if you choose things in this order:
>
>  1. Pick a test to run
>  2. Pick an image to run the test on
>
>  Or this order:
>
>  1. Pick an image to test
>  2. Pick a test to run on the image
>
>  ? It's pretty important to the design approach. Thanks!
>  
I'm ambivalent on affecting the design because I don't know what the
consequence is, and I'm naturally contrary even with myself; HOWEVER
the way I have been working is image focused first, and test focused
second. That is, the image is the primary and static variable, and
then I go through various tests. If I'm regression testing, I'd do the
reverse where the test is what gets pinned, and I vary the image.
Being a bug magnet, I end up in way more regression tests than I want.
I'd say time wise I probably spend the same amount of time image
(version) pinned vs test pinned.

I think an Aeromax 2000 might be more useful than my little anecdote though.

-- 
Chris Murphy

[Sumantro Mukherjee]

----- Original Message -----
>  From: "Adam Williamson" <adamwill(a)fedoraproject.org&gt;
>  To: "For testing and quality assurance of Fedora releases"
> <test(a)lists.fedoraproject.org&gt;
>  Sent: Wednesday, February 15, 2017 1:32:14 AM
>  Subject: Release validation testers: a question
>  
>  Hi folks! I've got a question for everyone who helps out with release
>  validation testing. It's related to the project to build a new system
>  for submitting those results. We have a ticket for designing that
>  system:
>  
>  https://pagure.io/design/issue/483
>  
>  and as part of the discussion there, we're wondering something. Given
>  that it's part of the new design that there's some kind of step where
>  you specify which particular image you're testing, does it make more
>  sense to you if you choose things in this order:
>  
>  1. Pick a test to run
>  2. Pick an image to run the test on
>  
>  Or this order:
>  
>  1. Pick an image to test
>  2. Pick a test to run on the image
>  
>  ? It's pretty important to the design approach. Thanks!
>  
>  (Please do feel free to read the ticket and mention any other thoughts
>  you have on the design, too.)
>  --
>  Adam Williamson
>  Fedora QA Community Monkey
>  IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
>  http://www.happyassassin.net
>  _______________________________________________
>  test mailing list -- test(a)lists.fedoraproject.org
>  To unsubscribe send an email to test-leave(a)lists.fedoraproject.org
>   

Hey!

Much like cmurf , I prefer image first until I complete testing the image. I generally
dont switch to another image. So, yes I would go with image first.
Thanks
Sumantro

[John Dulaney]

On Tue, Feb 14, 2017 at 12:02:14PM -0800, Adam Williamson wrote:
>  Hi folks! I've got a question for everyone who helps out with
> release
>  validation testing. It's related to the project to build a new system
>  for submitting those results. We have a ticket for designing that
>  system:
>  
>  https://pagure.io/design/issue/483
>  
>  and as part of the discussion there, we're wondering something. Given
>  that it's part of the new design that there's some kind of step where
>  you specify which particular image you're testing, does it make more
>  sense to you if you choose things in this order:
>  
>  1. Pick a test to run
>  2. Pick an image to run the test on
>  
>  Or this order:
>  
>  1. Pick an image to test
>  2. Pick a test to run on the image
>  
I grab an image and then do multiple tests with it.

John.

[Kanika Murarka]

On 15 February 2017 at 14:46, John Dulaney <jdulaney(a)fedoraproject.org&gt;
wrote:

>  On Tue, Feb 14, 2017 at 12:02:14PM -0800, Adam Williamson wrote:
>  > Hi folks! I've got a question for everyone who helps out with release
>  > validation testing. It's related to the project to build a new system
>  > for submitting those results. We have a ticket for designing that
>  > system:
>  >
>  > https://pagure.io/design/issue/483
>  >
>  > and as part of the discussion there, we're wondering something. Given
>  > that it's part of the new design that there's some kind of step where
>  > you specify which particular image you're testing, does it make more
>  > sense to you if you choose things in this order:
>  >
>  > 1. Pick a test to run
>  > 2. Pick an image to run the test on
>  >
>  > Or this order:
>  >
>  > 1. Pick an image to test
>  > 2. Pick a test to run on the image
>  >
>
>  I will prefer:- 
1. Pick an image to test
2. Pick a test to run on the image

Thanks

_______________________________________________
>  test mailing list -- test(a)lists.fedoraproject.org
>  To unsubscribe send an email to test-leave(a)lists.fedoraproject.org
>

[Tenk Tenk]

Prefer:

1 - Pick an image to test
2 - Pick a test to run on the image

Best Regards,



On Wed, Feb 15, 2017 at 9:06 PM, Kanika Murarka <murarkakanika(a)gmail.com&gt;
wrote:

>  On 15 February 2017 at 14:46, John Dulaney <jdulaney(a)fedoraproject.org&gt;
>  wrote:
>
> > On Tue, Feb 14, 2017 at 12:02:14PM -0800, Adam Williamson wrote:
> > > Hi folks! I've got a question for everyone who helps out with release
> > > validation testing. It's related to the project to build a new system
> > > for submitting those results. We have a ticket for designing that
> > > system:
> > >
> > > https://pagure.io/design/issue/483
> > >
> > > and as part of the discussion there, we're wondering something. Given
> > > that it's part of the new design that there's some kind of step where
> > > you specify which particular image you're testing, does it make more
> > > sense to you if you choose things in this order:
> > >
> > > 1. Pick a test to run
> > > 2. Pick an image to run the test on
> > >
> > > Or this order:
> > >
> > > 1. Pick an image to test
> > > 2. Pick a test to run on the image
> > >
> >
> > I will prefer:-
>
>  1. Pick an image to test
>  2. Pick a test to run on the image
>
>  Thanks
>
>  _______________________________________________
> > test mailing list -- test(a)lists.fedoraproject.org
> > To unsubscribe send an email to test-leave(a)lists.fedoraproject.org
> >
>
>
>  _______________________________________________
>  test mailing list -- test(a)lists.fedoraproject.org
>  To unsubscribe send an email to test-leave(a)lists.fedoraproject.org
>
>

[GERHARD GOETZHABER]

I think availability of images will be essential before anything else 
can be done well!

However, at present stage, there's an important question from my part, 
too: What happened about i386 images? The last one I found is from Feb. 
8th. Besides my AMD 990FX system, I've an Acer Aspire 1 which in many a 
way is a machine as well for testing and working as simple in handling. 
There may be  a crowd of other people who like keeping similar good old 
stuff for evenly getting set up with new Linux software ...


On 2/14/17 9:02 PM, Adam Williamson wrote:
>  Hi folks! I've got a question for everyone who helps out with
> release
>  validation testing. It's related to the project to build a new system
>  for submitting those results. We have a ticket for designing that
>  system:
>
>  https://pagure.io/design/issue/483
>
>  and as part of the discussion there, we're wondering something. Given
>  that it's part of the new design that there's some kind of step where
>  you specify which particular image you're testing, does it make more
>  sense to you if you choose things in this order:
>
>  1. Pick a test to run
>  2. Pick an image to run the test on
>
>  Or this order:
>
>  1. Pick an image to test
>  2. Pick a test to run on the image
>
>  ? It's pretty important to the design approach. Thanks!
>
>  (Please do feel free to read the ticket and mention any other thoughts
>  you have on the design, too.)

i386 Rawhide images (was: Re: Release validation testers: a question)

[Adam Williamson]

On Wed, 2017-02-15 at 14:31 +0100, GERHARD GOETZHABER wrote:
>  I think availability of images will be essential before anything else
>
>  can be done well!
>  
>  However, at present stage, there's an important question from my part, 
>  too: What happened about i386 images? The last one I found is from Feb. 
>  8th. Besides my AMD 990FX system, I've an Acer Aspire 1 which in many a 
>  way is a machine as well for testing and working as simple in handling. 
>  There may be  a crowd of other people who like keeping similar good old 
>  stuff for evenly getting set up with new Linux software ... 
It looks like the i386 install root creation is failing:

https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-2017021...

I think the same was happening to x86_64 until recently, so possibly
releng fixed it on x86_64 but forgot to fix it for i386 as well. I've
asked Dennis.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Mike Ruckman]

On Tue, Feb 14, 2017 at 12:02:14PM -0800, Adam Williamson wrote:
>  Hi folks! I've got a question for everyone who helps out with
> release
>  validation testing. It's related to the project to build a new system
>  for submitting those results. We have a ticket for designing that
>  system:
>  
>  https://pagure.io/design/issue/483
>  
>  and as part of the discussion there, we're wondering something. Given
>  that it's part of the new design that there's some kind of step where
>  you specify which particular image you're testing, does it make more
>  sense to you if you choose things in this order:
>  
>  1. Pick a test to run
>  2. Pick an image to run the test on
>  
>  Or this order:
>  
>  1. Pick an image to test
>  2. Pick a test to run on the image
>  
>  ? It's pretty important to the design approach. Thanks!
>  
>  (Please do feel free to read the ticket and mention any other thoughts
>  you have on the design, too.)
>  -- 
>  Adam Williamson
>  Fedora QA Community Monkey
>  IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
>  http://www.happyassassin.net 
I also pick an image and then run tests against it.

// Mike
--
Fedora QA

[Giulio E.]

Il giorno mer, 15/02/2017 alle 09.35 -0500, Mike Ruckman ha scritto:
>  On Tue, Feb 14, 2017 at 12:02:14PM -0800, Adam Williamson wrote:
>  > Hi folks! I've got a question for everyone who helps out with
>  > release
>  > validation testing. It's related to the project to build a new
>  > system
>  > for submitting those results. We have a ticket for designing that
>  > system:
>  > 
>  > https://pagure.io/design/issue/483
>  > 
>  > and as part of the discussion there, we're wondering something.
>  > Given
>  > that it's part of the new design that there's some kind of step
>  > where
>  > you specify which particular image you're testing, does it make
>  > more
>  > sense to you if you choose things in this order:
>  > 
>  > 1. Pick a test to run
>  > 2. Pick an image to run the test on
>  > 
>  > Or this order:
>  > 
>  > 1. Pick an image to test
>  > 2. Pick a test to run on the image
>  > 
>  > ? It's pretty important to the design approach. Thanks!
>  > 
>  > (Please do feel free to read the ticket and mention any other
>  > thoughts
>  > you have on the design, too.)
>  > -- 
>  > Adam Williamson
>  > Fedora QA Community Monkey
>  > IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin .
>  > net
>  > http://www.happyassassin.net
>  
>  I also pick an image and then run tests against it.
>  
>  // Mike
>  --
>  Fedora QA
>  _______________________________________________
>  test mailing list -- test(a)lists.fedoraproject.org
>  To unsubscribe send an email to test-leave(a)lists.fedoraproject.org 
I'm a bit late but... I usually pick a Desktop image (ideally, I pick
one which has not been tested yet, or misses some important results)
and then perform the tests I have time for.

---
Giulio (juliuxpigface)

[Kevin Fenzi]

On Tue, 14 Feb 2017 12:02:14 -0800
Adam Williamson <adamwill(a)fedoraproject.org&gt; wrote:

...snip...

>  Or this order:
>  
>  1. Pick an image to test
>  2. Pick a test to run on the image 
I do this. Usually because I am wanting to test a particular image I
care about like the cloud base or Xfce live. I also usually tend to
just go through all the tests I easily can while I have the image
booted/installed. 

kevin