Packages with missing %check

[Alexander Todorov]

Hi guys,
I have identified 551 packages on the Fedora 20 source DVD which are missing a 
%check section in their spec files but are very likely to have a test suite. See
https://github.com/atodorov/fedora-scripts/blob/master/sample-data/fedora-20/srpms-with-tests-WITHOUT-check-in-fedora-20-dvd

For a few of them I already had bug open previously (will filter the list later 
when I run my scripts against latest Rawhide).

I have the following questions:

1) Do we consider this a bug and if yes what priority do you give it? From last 
week discussions it looks like most people prefer to have tests executed in %check.

2) Last week Alexander Kurtakov brought up the issue of packages executing their 
test suite during build. How to detect such packages?  OTOH we can have some 
false negatives (also see 3).


3) Another proposal (sorry don't remember who proposed it) was to have %check 
with a comment why the test suite is not executed (e.g. requires network) or why 
it is executed in %build.

I support this as it makes it clear to the QA person what is happening.
This comment can be later extended to explain why a particular package is 
missing test suite (e.g. man pages or packages containing data files).


How about making %check a packaging requirement in all cases - run tests or add 
a comment explaining why not, how to run them (e.g. make test) or why there are 
no tests for this package.


--
Alex