FESCo wants to ban direct stable pushes in Bodhi (urgent call for feedback)

[Till Maas]

On Fri, Feb 26, 2010 at 05:28:26PM -0800, Adam Williamson wrote:
> On Fri, 2010-02-26 at 19:53 +0100, Till Maas wrote:
> > On Fri, Feb 26, 2010 at 10:01:56AM -0800, Jesse Keating wrote:
> > > On Fri, 2010-02-26 at 18:56 +0100, Till Maas wrote:
> > > > 
> > > > Something I am dreaming about is to have some infrastructure to
> > > > automatically test packages, so mabye they could build that first and
> > > > then write tests for packages. 
> > > 
> > > The AutoQA project is in full swing, developing just that, a framework
> > > to test packages in an automated fashion.
> > 
> > Ah, that's great. I thought it was only supposed to test packages
> > metadata etc, but not the behaviour of programs inside them. Is it
> > already possible to write a test that installs each packages that
> > contains a binary and ensures that it does not segfault when it is
> > called without arguments or with -h,--help,-? ideally for different
> > locales?
> 
> In addition to Jesse's reply, AutoQA is a framework for tests. You can
> theoretically run almost any type of test through AutoQA. Essentially
> it's just a little machine for running a bunch of arbitrary commands at
> specified times, and saving the results somewhere. It's by nature very
> flexible, and there's not a lot of restrictions on what the tests you
> can run within the AutoQA framework actually *do*.

Ok, maybe the question should be then: How much does AutoQA support me
writing these tests? E.g. this test is pretty simple, but afaics there
is no easy support for the common tasks that are needed to run the test,
but not really part of the test, e.g. installing the package or setting
up a machine.

The Writeing AutoQA Tests wiki page[0] says:
| I'll say it again: Write the test first. The tests don't require
| anything from autotest or autoqa. You should have a working test before
| you even start thinking about AutoQA

But this is not really supportive, because if I want to test a packages,
I need a framework that creates the initial environment, e.g. a system
of the Fedora version to be tested with the package installed and there
needs to be a way to interact with the programs.

Or is this really a test I can easily integrate into AutoQA currently?
Say we start without locales and commandline arguments, then the test
would be:

Input: Package to be tested as ${PACKAGE}
for binary in $(rpm -ql ${PACKAGE} | grep bin); do ${binary} 2>&1 | grep
"Segmentation fault" && echo "test failed" ; done

Regards
Till

[0] https://fedoraproject.org/wiki/Writing_AutoQA_Tests
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/devel/attachments/20100227/ce0fe392/attachment.bin