Impact of treebuilder on AutoQA, and RATS?

[Adam Williamson]

Hey, folks. I was thinking this week that, in the context of the RATS
discussion, it's probably worth looking at the big changes to lorax
coming out of the treebuilder branch:

https://fedoraproject.org/wiki/Anaconda/Features/Lorax-TreeBuilder

I filed a ticket for the specific impact on the compose_tree test:

https://fedorahosted.org/autoqa/ticket/392

But I wondered if the change has any other impacts on AutoQA in general,
and RATS. Does it make it more or less viable that we could simply
automate the creation of boot.iso for RATS runs, and avoid asking releng
to provide it for us? Or is this still something releng should own and
we should co-ordinate with releng on?

[Kamil Paral]

> Hey, folks. I was thinking this week that, in the context of the RATS
> discussion, it's probably worth looking at the big changes to lorax
> coming out of the treebuilder branch:
>
> https://fedoraproject.org/wiki/Anaconda/Features/Lorax-TreeBuilder
>
> I filed a ticket for the specific impact on the compose_tree test:
>
> https://fedorahosted.org/autoqa/ticket/392
>
> But I wondered if the change has any other impacts on AutoQA in
> general,
> and RATS. Does it make it more or less viable that we could simply
> automate the creation of boot.iso for RATS runs, and avoid asking
> releng
> to provide it for us? Or is this still something releng should own
> and
> we should co-ordinate with releng on?

I don't mind composing the boot.iso using AutoQA. But:

1. We need to upload it somewhere so that it can be detected and used by other tests (e.g. rats_install). It would be nice keep at least 7 last composes available, but we are very low on disk space. Also I imagine that not only our tests will want to use it after it is available, but other people will regularly download it too (similarly to daily desktop composes). Releng storage would be best to use here.

2. Who's going to tackle compose issues? If the compose fails, do we really want to debug that issue, find broken component, work on resolving it? Because at least for me the compose process is a black magic. /me votes no for further responsibilities.

Ideally releng should manage the test script and solve the issues, and we should just provide the automation and error reporting for them (if they don't have their own and better capabilities). But I don't know whether they are willing to do that, because there is probably no policy in current Fedora saying that Branched boot.iso should be composeable every day, so it's not strictly part of their job. Adam, maybe you have better information here?

[James Laska]

On Mon, 2011-12-12 at 03:49 -0500, Kamil Paral wrote:
> > Hey, folks. I was thinking this week that, in the context of the RATS
> > discussion, it's probably worth looking at the big changes to lorax
> > coming out of the treebuilder branch:
> >
> > https://fedoraproject.org/wiki/Anaconda/Features/Lorax-TreeBuilder
> >
> > I filed a ticket for the specific impact on the compose_tree test:
> >
> > https://fedorahosted.org/autoqa/ticket/392
> >
> > But I wondered if the change has any other impacts on AutoQA in
> > general,
> > and RATS. Does it make it more or less viable that we could simply
> > automate the creation of boot.iso for RATS runs, and avoid asking
> > releng
> > to provide it for us? Or is this still something releng should own
> > and
> > we should co-ordinate with releng on?
>
> I don't mind composing the boot.iso using AutoQA. But:
>
> We need to upload it somewhere so that it can be detected and used by other tests (e.g. rats_install). It would be nice keep at least 7 last composes available, but we are very low on disk space. Also I imagine that not only our tests will want to use it after it is available, but other people will regularly download it too (similarly to daily desktop composes). Releng storage would be best to use here.
>
> Who's going to tackle compose issues? If the compose fails, do we really want to debug that issue, find broken component, work on resolving it? Because at least for me the compose process is a black magic. /me votes no for further responsibilities.
>
> Ideally releng should manage the test script and solve the issues, and we should just provide the automation and error reporting for them (if they don't have their own and better capabilities). But I don't know whether they are willing to do that, because there is probably no policy in current Fedora saying that Branched boot.iso should be composeable every day, so it's not strictly part of their job. Adam, maybe you have better information here?

Note, automated creation of the boot.iso is available in AutoQA as a
test now.  Given recent changes, the script may need some adjustments,
but the principle is sound.

Take a look at tests/compose_tree/* in git.

This doesn't handle some of the multi-test synchronization, solely the
testing needed to generate the boot.iso.

Thanks,
James

Thanks,
James

[Adam Williamson]

On Mon, 2011-12-12 at 10:28 -0500, James Laska wrote:
> Note, automated creation of the boot.iso is available in AutoQA as a
> test now.  Given recent changes, the script may need some adjustments,
> but the principle is sound.
>
> Take a look at tests/compose_tree/* in git.
> From the OP:

"I filed a ticket for the specific impact on the compose_tree test:

https://fedorahosted.org/autoqa/ticket/392"

[Hongqing Yang]

----- Original Message -----
> From: "Adam Williamson" 
> To: "AutoQA development" 
> Sent: Tuesday, December 13, 2011 8:23:34 AM
> Subject: Re: Impact of treebuilder on AutoQA, and RATS?
>
> On Mon, 2011-12-12 at 10:28 -0500, James Laska wrote:
> > Note, automated creation of the boot.iso is available in AutoQA as
> > a
> > test now.  Given recent changes, the script may need some
> > adjustments,
> > but the principle is sound.
> >
> > Take a look at tests/compose_tree/* in git.
>
> From the OP:
>
> "I filed a ticket for the specific impact on the compose_tree test:
> https://fedorahosted.org/autoqa/ticket/392"
>
> Adam Williamson

I do not think the rawhide acceptance test plan requires the boot.iso, but
the initrd.img and vmlinuz are required. pls see the test plan:
http://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan

Hongqing

[Hongqing Yang]

I just realize that there is not any images at download.fedoraproject.org/pub/fedora/linux/development/rawhide/[x86_64,i386]/os/ ,
The rats_install update is almost ready, but it will not work without images. 
I would like to pick up the task to build the images based on the work James has done and follow the instruction http://fedoraproject.org/wiki/How_to_create_a_Fedora_install_ISO_for_testing if none of you is doing this.

Hongqing

----- Original Message -----
> From: "Hongqing Yang" 
> To: "AutoQA development" 
> Sent: Tuesday, December 13, 2011 11:23:32 AM
> Subject: Re: Impact of treebuilder on AutoQA, and RATS?
>
> ----- Original Message -----
> > From: "Adam Williamson" 
> > To: "AutoQA development" 
> > Sent: Tuesday, December 13, 2011 8:23:34 AM
> > Subject: Re: Impact of treebuilder on AutoQA, and RATS?
> >
> > On Mon, 2011-12-12 at 10:28 -0500, James Laska wrote:
> > > Note, automated creation of the boot.iso is available in AutoQA
> > > as
> > > a
> > > test now.  Given recent changes, the script may need some
> > > adjustments,
> > > but the principle is sound.
> > >
> > > Take a look at tests/compose_tree/* in git.
> >
> > From the OP:
> >
> > "I filed a ticket for the specific impact on the compose_tree test:
> > https://fedorahosted.org/autoqa/ticket/392"
> >
> > Adam Williamson
>
> I do not think the rawhide acceptance test plan requires the
> boot.iso, but
> the initrd.img and vmlinuz are required. pls see the test plan:
> http://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
>
> Hongqing
> _______________________________________________
> autoqa-devel mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/autoqa-devel

[Kamil Paral]

> I just realize that there is not any images at
> download.fedoraproject.org/pub/fedora/linux/development/rawhide/[x86_64,i386]/os/

That's because Rawhide media composes are no longer created. Releng should start creating them once F17 is Branched. I believe we are OK with testing just Branched composes in this test cycle.
> The rats_install update is almost ready, but it will not work without
> images.
> I would like to pick up the task to build the images based on the
> work James has done and follow the instruction
> http://fedoraproject.org/wiki/How_to_create_a_Fedora_install_ISO_for_testing
> if none of you is doing this.

Read James' comment and Adam's ticket mentioned in this thread. If you want to make sure compose_tree works OK and fix any related problems, that's great.