Packages with missing %check

[Mikolaj Izdebski]

On 02/25/2014 11:45 AM, Alexander Todorov wrote:
> Hi guys,
> I have identified 551 packages on the Fedora 20 source DVD which are
> missing a %check section in their spec files but are very likely to have
> a test suite. See
> https://github.com/atodorov/fedora-scripts/blob/master/sample-data/fedora-20/srpms-with-tests-WITHOUT-check-in-fedora-20-dvd

I can see a lot of Java packages on that list.  Most Java packages run
their test suites in %build and don't have %check section at all, so
they would be false-positives.

> 1) Do we consider this a bug and if yes what priority do you give it?
> From last week discussions it looks like most people prefer to have
> tests executed in %check.

If upstream includes test suite but it is not ran without any comment
then I consider this as a bug.  If some tests cannot be ran because of
missing dependencies or because they require network access etc. then I
try to disable these test cases and run all others.

> 2) Last week Alexander Kurtakov brought up the issue of packages
> executing their test suite during build. How to detect such packages? 
> OTOH we can have some false negatives (also see 3).

For packages using Maven (most common Java build system) you can use
heuristic implemented in the following script:
  http://mizdebsk.fedorapeople.org/scripts/wr-testsuitecheck
I used it to review my 250+ packages I maintain and it has low
false-positive ratio.

> 3) Another proposal (sorry don't remember who proposed it) was to have
> %check with a comment why the test suite is not executed (e.g. requires
> network) or why it is executed in %build.

Commenting why tests are skipped is a very good thing, but I don't like
the idea of adding empty %check sections to my 250+ packages just for
the sake of documenting that tests are ran in %build "because that's
what we do in Java world".

-- 
Mikolaj Izdebski
IRC: mizdebsk