Re: Creating autoqa-watchers sub-package?

[Kamil Paral]

----- "James Laska"  wrote:
> Greetings,
>
> As autoqa is currently packaged, all tests, watchers, configs and
> cronjobs are included in the main autoqa package.  This works fine
> except for a minor annoyance.  Anytime you install 'autoqa', unless
> you
> are actually scheduling jobs, you needed comment or remove the
> watcher
> notification /etc/cron.d/autoqa.  This gets annoying the more I
> deploy
> autoqa.
>
> I'd like propose moving the watcher scripts and cronjob into a
> sub-package (see attached patch for detail).
>
> When most people install 'autoqa', they want the test library and
> tests.
> This won't change that behavior.  The only change will be for anyone
> setting up a test server.  I'll need to update the existing wiki
> documentation to note installing 'autoqa-watchers' when setting up a
> test server [1].
>
> Comments/concerns/ideas?

Splitting is a good idea. Just a remark: If you put watchers (i.e. hooks)
into a separate package but leave autoqa harness in the original package,
then the autoqa harness can't be used (it doesn't work separately, it needs
hook.py files to operate). So people won't be able to just try and run some
test, even manually.

I see several ways:
1. Let's create autotest and autotest-client packages. Autotest-client will
contain only stuff really required for clients, i.e. autotest and autoqa 
library. Autotest package will contain the rest. Easy to deploy, but people
that want to experiment with it must install all (and disable cron jobs).

2. Your idea, autotest and autotest-watchers packages. Easy to deploy, people
still can't just run a test without the full installation. But this time they
can at least have a look at the tests' source code.

3. Creating autotest and autotest-server packages. Autotest-server will 
contain just the crontab file. Silly? :)

4. What about commenting out the crontab file by default? If someone really
wants to deploy a server, he will just uncomment the file (and maybe tweak
the settings anyway). That file is left intact on upgrade, so that shouldn't
be a problem, it's a one-time task. And we will document that on the wiki
(the same way we have now documented that everyone not wanting to run a server
should comment out that file).

I would like to provide people easy way to try and play with autoqa. Although
the last solution seems to be extremely simple, it seems to work well enough.
Would that help you too? Or do you prefer another way? What do you think?

Creating autoqa-watchers sub-package?

[James Laska]

Hi Kamil,

Good ideas ... comments in-line below.

On Tue, 2010-09-14 at 03:53 -0400, Kamil Paral wrote:
> ----- "James Laska"  wrote:
> > Greetings,
> >
> > As autoqa is currently packaged, all tests, watchers, configs and
> > cronjobs are included in the main autoqa package.  This works fine
> > except for a minor annoyance.  Anytime you install 'autoqa', unless
> > you
> > are actually scheduling jobs, you needed comment or remove the
> > watcher
> > notification /etc/cron.d/autoqa.  This gets annoying the more I
> > deploy
> > autoqa.
> >
> > I'd like propose moving the watcher scripts and cronjob into a
> > sub-package (see attached patch for detail).
> >
> > When most people install 'autoqa', they want the test library and
> > tests.
> > This won't change that behavior.  The only change will be for anyone
> > setting up a test server.  I'll need to update the existing wiki
> > documentation to note installing 'autoqa-watchers' when setting up a
> > test server [1].
> >
> > Comments/concerns/ideas?
>
> Splitting is a good idea. Just a remark: If you put watchers (i.e. hooks)
> into a separate package but leave autoqa harness in the original package,
> then the autoqa harness can't be used (it doesn't work separately, it needs
> hook.py files to operate). So people won't be able to just try and run some
> test, even manually.

Exactly, in the idea proposed, if setting up an fully automated AutoQA
test server, you would install autoqa-watchers which would pull in
appropriate dependencies (really just autoqa).
> I see several ways:

For the following ideas, I assume you meant s/autotest/autoqa/.
> Let's create autotest and autotest-client packages. Autotest-client will
>
> contain only stuff really required for clients, i.e. autotest and autoqa 
> library. Autotest package will contain the rest. Easy to deploy, but people
> that want to experiment with it must install all (and disable cron jobs).
> Your idea, autotest and autotest-watchers packages. Easy to deploy, people
>
> still can't just run a test without the full installation. But this time they
> can at least have a look at the tests' source code.

Oh I see what you mean now.  I keep forgetting about the approach to run
tests using 'autoqa' directly.  Apologies, not intentional, I'm just
unfamiliar with this technique still.  I'm so used to running the tests
by hand, instead of the 'autoqa $hook-name $args' approach.
> Creating autotest and autotest-server packages. Autotest-server will
>
> contain just the crontab file. Silly? :)

Yeah, I agree ... this seems a bit silly.
> What about commenting out the crontab file by default? If someone really
>
> wants to deploy a server, he will just uncomment the file (and maybe tweak
> the settings anyway). That file is left intact on upgrade, so that shouldn't
> be a problem, it's a one-time task. And we will document that on the wiki
> (the same way we have now documented that everyone not wanting to run a server
> should comment out that file).

That's the other simple approach.  Move /etc/cron.d/autoqa over
to /usr/share/doc and list it as a recommended cron configuration.
Since really, the exact cron specification used will vary depending on
resources and admin preferences.
> I would like to provide people easy way to try and play with autoqa. Although
> the last solution seems to be extremely simple, it seems to work well enough.
> Would that help you too? Or do you prefer another way? What do you think?

I'm starting to lean towards option#4 now.  I'll let this absorb and
send an appropriate patch to the list later in the week (likely along
with an autoqa.spec update).

Thanks,
James

Creating autoqa-watchers sub-package?

[Kamil Paral]

----- "James Laska"  wrote:
> Hi Kamil,
>
> Good ideas ... comments in-line below.
>
> On Tue, 2010-09-14 at 03:53 -0400, Kamil Paral wrote:
> > ----- "James Laska"  wrote:
> > > Greetings,
> > >
> > > As autoqa is currently packaged, all tests, watchers, configs and
> > > cronjobs are included in the main autoqa package.  This works
>
> fine
> > > except for a minor annoyance.  Anytime you install 'autoqa',
>
> unless
> > > you
> > > are actually scheduling jobs, you needed comment or remove the
> > > watcher
> > > notification /etc/cron.d/autoqa.  This gets annoying the more I
> > > deploy
> > > autoqa.
> > >
> > > I'd like propose moving the watcher scripts and cronjob into a
> > > sub-package (see attached patch for detail).
> > >
> > > When most people install 'autoqa', they want the test library and
> > > tests.
> > > This won't change that behavior.  The only change will be for
>
> anyone
> > > setting up a test server.  I'll need to update the existing wiki
> > > documentation to note installing 'autoqa-watchers' when setting up
>
> a
> > > test server [1].
> > >
> > > Comments/concerns/ideas?
> >
> > Splitting is a good idea. Just a remark: If you put watchers (i.e.
>
> hooks)
> > into a separate package but leave autoqa harness in the original
>
> package,
> > then the autoqa harness can't be used (it doesn't work separately,
>
> it needs
> > hook.py files to operate). So people won't be able to just try and
>
> run some
> > test, even manually.
>
> Exactly, in the idea proposed, if setting up an fully automated
> AutoQA
> test server, you would install autoqa-watchers which would pull in
> appropriate dependencies (really just autoqa).
> > I see several ways:
>
> For the following ideas, I assume you meant s/autotest/autoqa/.

Damn, of course I did :) Sorry.
> > Let's create autotest and autotest-client packages.
>
> Autotest-client will
> > contain only stuff really required for clients, i.e. autotest and
>
> autoqa
> > library. Autotest package will contain the rest. Easy to deploy, but
>
> people
> > that want to experiment with it must install all (and disable cron
>
> jobs).
> > Your idea, autotest and autotest-watchers packages. Easy to
>
> deploy, people
> > still can't just run a test without the full installation. But this
>
> time they
> > can at least have a look at the tests' source code.
>
> Oh I see what you mean now.  I keep forgetting about the approach to
> run
> tests using 'autoqa' directly.  Apologies, not intentional, I'm just
> unfamiliar with this technique still.  I'm so used to running the
> tests
> by hand, instead of the 'autoqa $hook-name $args' approach.

We start to have some tests which can't be run standalone. E.g. upgradepath.
Well, technically, you can import it in Python to run it, but that's quite
hard. So, if we want people to be able to run it, the autoqa harness is
the only way (and --local option makes it a little easier).
> > Creating autotest and autotest-server packages. Autotest-server
>
> will
> > contain just the crontab file. Silly? :)
>
> Yeah, I agree ... this seems a bit silly.
> > What about commenting out the crontab file by default? If someone
>
> really
> > wants to deploy a server, he will just uncomment the file (and maybe
>
> tweak
> > the settings anyway). That file is left intact on upgrade, so that
>
> shouldn't
> > be a problem, it's a one-time task. And we will document that on the
>
> wiki
> > (the same way we have now documented that everyone not wanting to
>
> run a server
> > should comment out that file).
>
> That's the other simple approach.  Move /etc/cron.d/autoqa over
> to /usr/share/doc and list it as a recommended cron configuration.
> Since really, the exact cron specification used will vary depending
> on
> resources and admin preferences.

Ah, even better. Good idea.
> > I would like to provide people easy way to try and play with autoqa.
>
> Although
> > the last solution seems to be extremely simple, it seems to work
>
> well enough.
> > Would that help you too? Or do you prefer another way? What do you
>
> think?
>
> I'm starting to lean towards option#4 now.  I'll let this absorb and
> send an appropriate patch to the list later in the week (likely along
> with an autoqa.spec update).

By the way, we can combine those approaches. So maybe we can split it
into subpackages AND disable crontab file by default. Whatever helps.