Anaconda UI rewrite + AutoQA

[Kamil Paral]

On our last QA meeting Adam Williamson notified us that there would be an Anaconda hackfest at the next FUDCon and we should raise our concerns (if we had some) regarding AutoQA and the UI re-write. Well, I don't know much about anaconda testing inside AutoQA, I just know we execute some anaconda tests, that's all. But I decided to kick off the discussion and CC relevant people.
> From what I know, we have two kind of anaconda tests inside AutoQA:

1. older rats_install test written by Will Woods and executed James Laska in the past
2. newer anaconda_storage, anaconda_checkbot and compose_tree written by Chris Lumens and James Laska

I am not sure which of those tests provide important feedback to the anaconda team and whether they are being used at all. Can relevant people provide more details about it?

Anaconda team, do you have some specific questions about AutoQA integration and the UI re-write? Will you want to change your tests inside AutoQA in response the new UI and need to have some information from us? I'll happy to answer them if I can.

Also, if the test authors have some comments or recommendation about the new UI architecture in order to allow simple automated testing, please post them here.

Thanks,
Kamil

[Chris Lumens]

> From what I know, we have two kind of anaconda tests inside AutoQA:
> older rats_install test written by Will Woods and executed James
>
> Laska in the past
> 2. newer anaconda_storage, anaconda_checkbot and compose_tree written
> by Chris Lumens and James Laska
>
> I am not sure which of those tests provide important feedback to the
> anaconda team and whether they are being used at all. Can relevant
> people provide more details about it?

I'm currently not checking the results of any tests, but I would like
to.  Can I get emailed on just the results of the anaconda tests?

Of the ones in (2), checkbot is likely the most important, though it may
also be failing at the moment due to transifex.  Storage could
potentially be very handy.
> Anaconda team, do you have some specific questions about AutoQA
> integration and the UI re-write? Will you want to change your tests
> inside AutoQA in response the new UI and need to have some information
> from us? I'll happy to answer them if I can.

The storage test will potentially need changes since I'm working on
removing the interface requirement from storage probing.  I don't know
if we are going to have specific extra

I don't have any testing-related questions at the moment, but that's
largely because I have no idea how to even get started making sure this
UI is easily testable.  If you have any pointers, I'd be happy to
consider them.  I'm only going to write a new UI once, so we should make
sure it's testable.

- Chris

[Kamil Paral]

> > From what I know, we have two kind of anaconda tests inside AutoQA:
> > older rats_install test written by Will Woods and executed James
> >
> > Laska in the past
> > 2. newer anaconda_storage, anaconda_checkbot and compose_tree
> > written
> > by Chris Lumens and James Laska
> >
> > I am not sure which of those tests provide important feedback to
> > the
> > anaconda team and whether they are being used at all. Can relevant
> > people provide more details about it?
>
> I'm currently not checking the results of any tests, but I would like
> to.  Can I get emailed on just the results of the anaconda tests?

As of AutoQA 0.7.0, which should be deployed to production next week, all anaconda-related tests (anaconda_storage, anaconda_checkbot and compose_tree) support opt-in email subscription [1]. If you opt-in, you should get an email with results for every such test executed.

If you would like to subscribe some non-FAS account email, like a mailing list, we can hardcode that into the test (but tell me soon).

[1] http://jlaska.wordpress.com/2010/06/01/fedora-package-maintainers-want-test-...
> I don't have any testing-related questions at the moment, but that's
> largely because I have no idea how to even get started making sure
> this
> UI is easily testable.  If you have any pointers, I'd be happy to
> consider them.  I'm only going to write a new UI once, so we should
> make
> sure it's testable.

I believe we (our you) won't write GUI tests per se. Rather the underlying "engine"/library should be controllable so that we can simulate the installation by calling API instead of "clicking buttons".

[Chris Lumens]

> As of AutoQA 0.7.0, which should be deployed to production next week,
> all anaconda-related tests (anaconda_storage, anaconda_checkbot and
> compose_tree) support opt-in email subscription [1]. If you opt-in,
> you should get an email with results for every such test executed.

I will look into that after the long weekend.
> I believe we (our you) won't write GUI tests per se. Rather the
> underlying "engine"/library should be controllable so that we can
> simulate the installation by calling API instead of "clicking
> buttons".

Well, a lot of the new anaconda UI is being done in glade and using GTK
signals as control.  I don't know to what degree we can ensure that the
library results in the same behavior as clicking buttons.  I suppose
it's controlled by how I write the signal handlers.  I welcome any input
here.

- Chris

[Hongqing Yang]

----- Original Message -----
> From: [e-mail]
> To: "Kamil Paral" 
> Cc: "AutoQA development" , [e-mail]
> Sent: Thursday, November 24, 2011 2:17:51 AM
> Subject: Re: Anaconda UI rewrite + AutoQA
> > As of AutoQA 0.7.0, which should be deployed to production next
> > week,
> > all anaconda-related tests (anaconda_storage, anaconda_checkbot and
> > compose_tree) support opt-in email subscription [1]. If you opt-in,
> > you should get an email with results for every such test executed.
>
> I will look into that after the long weekend.
> > I believe we (our you) won't write GUI tests per se. Rather the
> > underlying "engine"/library should be controllable so that we can
> > simulate the installation by calling API instead of "clicking
> > buttons".
>
> Well, a lot of the new anaconda UI is being done in glade and using
> GTK
> signals as control.  I don't know to what degree we can ensure that
> the
> library results in the same behavior as clicking buttons.  I suppose
> it's controlled by how I write the signal handlers.  I welcome any
> input
> here.
> Chris

The new UI will not affect the current Fedora Installation Test Automation project.
And I do not think the rats_install still works now because the stages2 images install.img
is compressed into initrd.img after f14. I will update it later.

Hongqing

[Kamil Paral]

> I don't have any testing-related questions at the moment, but that's
> largely because I have no idea how to even get started making sure
> this
> UI is easily testable.  If you have any pointers, I'd be happy to
> consider them.  I'm only going to write a new UI once, so we should
> make
> sure it's testable.

I have gathered some basic guidelines from the Desktop team:
- use standard GTK widgets, not custom ones
- make sure everything is accessible via dogtail
- make sure anaconda can be run in "application mode", e.g. it can be started and tested in a common desktop environment

That's all I got. Will Woods also promised to put his insights in here.

[Chris Lumens]

> use standard GTK widgets, not custom ones

Too late, there's already some custom widgets.  I guess I'll have to
look into what I can do.
> make sure everything is accessible via dogtail

We used to have some sort of dogtail support in anaconda, but as much as
I asked I never heard back that anyone was using it, nor that anyone had
really ever tried to use it.  So I removed it.
> make sure anaconda can be run in "application mode", e.g. it can be
>
> started and tested in a common desktop environment

This I need to put some work into.  The main anaconda file is in pretty
terrible shape as far as this is concerned, but the UI code started out
standalone.  I should be able to make a driver for it that'll pass in a
dummy devicetree and install class (or, perhaps I can pass real ones it
- since the first half of the UI won't make any permanent changes).

- Chris

[Hongqing Yang]

----- Original Message -----
> From: [e-mail]
> To: "Kamil Paral" 
> Cc: "AutoQA development" , [e-mail]
> Sent: Friday, December 2, 2011 11:08:54 PM
> Subject: Re: Anaconda UI rewrite + AutoQA
> > use standard GTK widgets, not custom ones
>
> Too late, there's already some custom widgets.  I guess I'll have to
> look into what I can do.
> > make sure everything is accessible via dogtail
>
> We used to have some sort of dogtail support in anaconda, but as much
> as
> I asked I never heard back that anyone was using it, nor that anyone
> had
> really ever tried to use it.  So I removed it.

since the author of dogtail resigned before, so there is nobody to maintain
it. Now they have assigned new resource to it. 
Since dogtail did not work before, I switched to LDTP. I also received some
requests about dogtail. Hope it can work with dogtail in the new release.
> > make sure anaconda can be run in "application mode", e.g. it can
> >
> > be
> > started and tested in a common desktop environment
>
> This I need to put some work into.  The main anaconda file is in
> pretty
> terrible shape as far as this is concerned, but the UI code started
> out
> standalone.  I should be able to make a driver for it that'll pass in
> a
> dummy devicetree and install class (or, perhaps I can pass real ones
> it
>
> since the first half of the UI won't make any permanent changes).
>
> Chris
>
> autoqa-devel mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/autoqa-devel