Re: [PATCH] add upgradepath test

[Kamil Paral]

----- "James Laska"  wrote:
> See attached patch to convey the stand-alone idea noted above.

Ok, I see your point now. Yes, we can do it like that. We will create
"stand-alone" script check_upgradepath.py, that will depend on autoqa
(therefore be able to use autoqa library and repoinfo.conf), but can
be run alone like:

$ ./check_upgradepath.py --envrs blobby-0.9b-1.fc13 --kojitag dist-f13-updates

I'm not sure how large the benefit is over calling it through the hook,
but surely the CLI interface is a little simpler than:

$ autoqa post-bodhi-update --title blobby-0.9b-1.fc13 --target-tag dist-f13-updates blobby-0.9b-1.fc13 --local -t upgradepath

Both should achieve the same functionality.

[PATCH] add upgradepath test

[James Laska]

On Fri, 2010-08-20 at 09:38 -0400, Kamil Paral wrote:
> ----- "James Laska"  wrote:
> > See attached patch to convey the stand-alone idea noted above.
>
> Ok, I see your point now. Yes, we can do it like that. We will create
> "stand-alone" script check_upgradepath.py, that will depend on autoqa
> (therefore be able to use autoqa library and repoinfo.conf), but can
> be run alone like:
>
> $ ./check_upgradepath.py --envrs blobby-0.9b-1.fc13 --kojitag dist-f13-updates
>
> I'm not sure how large the benefit is over calling it through the hook,
> but surely the CLI interface is a little simpler than:
>
> $ autoqa post-bodhi-update --title blobby-0.9b-1.fc13 --target-tag dist-f13-updates blobby-0.9b-1.fc13 --local -t upgradepath

You know, I completely forgot about this method of executing tests
against --local.  Sounds like I need to reacquaint myself with this
approach.  If running a test is just a matter of 1) yum install autoqa,
2) autoqa $hook --args ... that's not too bad.  We already have the
argument handling/parsing being done by autoqa and the hook.  Apologies,
I never thought of this method.

Wwoods, what do you think?

Thanks,
James

[PATCH] add upgradepath test - UPDATE

[James Laska]

On Fri, 2010-08-20 at 10:23 -0400, James Laska wrote:
> On Fri, 2010-08-20 at 09:38 -0400, Kamil Paral wrote:
> > ----- "James Laska"  wrote:
> > > See attached patch to convey the stand-alone idea noted above.
> >
> > Ok, I see your point now. Yes, we can do it like that. We will create
> > "stand-alone" script check_upgradepath.py, that will depend on autoqa
> > (therefore be able to use autoqa library and repoinfo.conf), but can
> > be run alone like:
> >
> > $ ./check_upgradepath.py --envrs blobby-0.9b-1.fc13 --kojitag dist-f13-updates
> >
> > I'm not sure how large the benefit is over calling it through the hook,
> > but surely the CLI interface is a little simpler than:
> >
> > $ autoqa post-bodhi-update --title blobby-0.9b-1.fc13 --target-tag dist-f13-updates blobby-0.9b-1.fc13 --local -t upgradepath
>
> You know, I completely forgot about this method of executing tests
> against --local.  Sounds like I need to reacquaint myself with this
> approach.  If running a test is just a matter of 1) yum install autoqa,
> 2) autoqa $hook --args ... that's not too bad.  We already have the
> argument handling/parsing being done by autoqa and the hook.  Apologies,
> I never thought of this method.
>
> Wwoods, what do you think?

Just a quick update from the fedora-qa meeting today [1].  The group
agreed that the above command-line method for running tests works just
fine.  The group felt that stand-alone tests are easier for autoqa
newbies to understand, but that the above method integrates nicely with
AutoQA and does require a bit less work.  The team also concluded that
the method should be documented on the wiki somewhere.

1. Joza noted he would file a ticket to add 'How to run autoqa
        tests'
     2. Jlaska noted he would be interested in drafting some content for
        review

Thanks,
James

[1] https://fedoraproject.org/wiki/QA/Meetings/20100823

[PATCH] add upgradepath test - UPDATE

[Will Woods]

On Mon, 2010-08-23 at 15:57 -0400, James Laska wrote:
> Just a quick update from the fedora-qa meeting today [1].  The group
> agreed that the above command-line method for running tests works just
> fine.

..and since that was really the only objection to the patch, it's been
applied to master. Thanks again, Vojta!

-w

[PATCH] add upgradepath test - UPDATE

[Kamil Paral]

----- "Will Woods"  wrote:
> ..and since that was really the only objection to the patch, it's
> been
> applied to master. Thanks again, Vojta!

Thanks Will, I believe the new test case should work just fine. There are 
just 2 outstanding issues, which I believe vaschenb will patch shortly:

1. Schedule upgradepath for running with post-bodhi-hook (modify control.autoqa)
2. Improve output format a little - I requested a minor improvement in the
   test output so it's easy to see whether and which checks have failed.

[PATCH] add upgradepath test - UPDATE

[Vojtěch Aschenbrenner]

On Tue, Aug 24, 2010 at 04:08:11AM -0400, Kamil Paral wrote:
> Thanks Will, I believe the new test case should work just fine. There are 
> just 2 outstanding issues, which I believe vaschenb will patch shortly:
> Schedule upgradepath for running with post-bodhi-hook (modify control.autoqa)

Done.
> Improve output format a little - I requested a minor improvement in the
> test output so it's easy to see whether and which checks have failed.

Done.

Changes are pushed into master (sorry for not sending patch, but I'm to
lazy :-), so please don't kill me for pushing into master)