Hi guys, I have identified 551 packages on the Fedora 20 source DVD which are missing a %check section in their spec files but are very likely to have a test suite. See https://github.com/atodorov/fedora-scripts/blob/master/sample-data/fedora-20...
For a few of them I already had bug open previously (will filter the list later when I run my scripts against latest Rawhide).
I have the following questions:
1) Do we consider this a bug and if yes what priority do you give it? From last week discussions it looks like most people prefer to have tests executed in %check.
2) Last week Alexander Kurtakov brought up the issue of packages executing their test suite during build. How to detect such packages? OTOH we can have some false negatives (also see 3).
3) Another proposal (sorry don't remember who proposed it) was to have %check with a comment why the test suite is not executed (e.g. requires network) or why it is executed in %build.
I support this as it makes it clear to the QA person what is happening. This comment can be later extended to explain why a particular package is missing test suite (e.g. man pages or packages containing data files).
How about making %check a packaging requirement in all cases - run tests or add a comment explaining why not, how to run them (e.g. make test) or why there are no tests for this package.
-- Alex
----- Original Message -----
From: "Alexander Todorov" atodorov@redhat.com To: "Discussion of RPM packaging standards and practices for Fedora" packaging@lists.fedoraproject.org Cc: "Development discussions related to Fedora" devel@lists.fedoraproject.org Sent: Tuesday, February 25, 2014 12:45:11 PM Subject: [Fedora-packaging] Packages with missing %check
Hi guys, I have identified 551 packages on the Fedora 20 source DVD which are missing a %check section in their spec files but are very likely to have a test suite. See https://github.com/atodorov/fedora-scripts/blob/master/sample-data/fedora-20...
For a few of them I already had bug open previously (will filter the list later when I run my scripts against latest Rawhide).
I have the following questions:
- Do we consider this a bug and if yes what priority do you give it? From
last week discussions it looks like most people prefer to have tests executed in %check.
- Last week Alexander Kurtakov brought up the issue of packages executing
their test suite during build. How to detect such packages? OTOH we can have some false negatives (also see 3).
- Another proposal (sorry don't remember who proposed it) was to have %check
with a comment why the test suite is not executed (e.g. requires network) or why it is executed in %build.
I support this as it makes it clear to the QA person what is happening. This comment can be later extended to explain why a particular package is missing test suite (e.g. man pages or packages containing data files).
How about making %check a packaging requirement in all cases - run tests or add a comment explaining why not, how to run them (e.g. make test) or why there are no tests for this package.
I would be all for it if there are enough people that want to do it, start it, make sure that >90% of the packages confirm to that and we make it a requirement after that. Making a requirement if noone steps in to do it would just upset the already heavily overloaded packagers. If there are people that want to do, they can do it now and once most things are in line it's made a requirement. Same as compiler warnings - one never enables all of them at once, enable one warning, fix instances, enable another, fix instances. To continue the analogy in current case I think it would be best if work with 1 language/ecosystem (gnome, kde, perl, python, ruby, java...) community, see things there, this things there, get it into their specific guidelines, move to next ecosystem, done with most, make it overall requirement. Such an approach would ensure understanding the needs of the different subcomunities.
Alexander Kurtakov Red Hat Eclipse team
-- Alex
-- packaging mailing list packaging@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/packaging
On Tue, Feb 25, 2014 at 12:45:11PM +0200, Alexander Todorov wrote:
Hi guys, I have identified 551 packages on the Fedora 20 source DVD which are missing a %check section in their spec files but are very likely to have a test suite. See https://github.com/atodorov/fedora-scripts/blob/master/sample-data/fedora-20...
For a few of them I already had bug open previously (will filter the list later when I run my scripts against latest Rawhide).
I have the following questions:
- Do we consider this a bug and if yes what priority do you give it?
From last week discussions it looks like most people prefer to have tests executed in %check.
https://fedoraproject.org/wiki/Packaging:Guidelines#Test_Suites
So in some set of cases this would be a bug. I don't have an estimate of how many test suites would be "practical to" execute the test suite though.
-Toshio
packaging@lists.fedoraproject.org