Testing of packages

[Michael Schwendt]

On Fri, 27 Apr 2012 13:04:56 +0200, MR (Matthias) wrote:

> On 27/04/12 12:37, Michael Schwendt wrote:
> 
> > Hardly any %check section uses installed files. Typically, files in the
> > buildroot are used, or files plus test data in the builddir. Test data
> > won't be available.
> > 
> We encourage our packagers to implement useful check-sections.

Which is a recommendation mostly because of existing "make check" or "make
test" targets. Additionally, some spec files run custom tests not
available in the upstream source tarball (e.g. automation of basic
file conversion tests).

> To make an example, I'm packaging the python-django framework. I'd like
> get python-django's check-section get executed (using a django
> installation) when python is updated. Or to get some django-requiring
> packages get checked, when django is updated. I'd like to catch not only
> API/ABI bumps, but also changed/broken behaviour, produced somewhere else.
> 
> Most of required description is already there, so it can't be to hard,
> to execute those tests from check-section.

A wrong approach?

Why do you insist on using the %check section for that? It's a build-time
feature, found only in the spec file and not packaged in the built rpms
(also due to the test data I've referred to above).

You would be better served with a run-time test-suite. A collection
of tests that would be included in a built package, could be installed
and run independently, unrelated to the src.rpm build process. Fedora's
AutoQA project sounds like the place where to look for triggers that can
be used to run tests depending on new build results.

-- 
Fedora release 17 (Beefy Miracle) - Linux 3.3.2-8.fc17.x86_64
loadavg: 0.00 0.01 0.05