[Fedora-packaging] How to handle tests in %check

[Ralf Corsepius]

On 09/05/2013 12:41 PM, Michael Schwendt wrote:
> On Thu, 5 Sep 2013 12:51:18 +0300, Ville Skyttä wrote:
>
>>> On Thu, 05 Sep 2013 09:21:31 +0200, Miroslav Suchy wrote:
>>>>
>>>> Mamoru disagree. For me it is blocker.
>>>> I would like to ask for your opinions. Which way is correct and should
>>>> it be blocker for review?
>>>
>>> So far, it's not a blocker.

Correct, it's a should - Why?

Because
* we want to encourage upstreams to write testsuites and not let package 
reviews fail reviews because they are supplying a testsuites and to pass 
those packages whose upstreams don't ship a testsuite.
* testsuites are of very widely ranging quality and content/coverage.
Starting with primitive API-checking linker-tests, over simple 
"self-test" testsuites up to very extensive testsuites, which 
occasionally require a huge infrastructure.

In other words: Reviewers are supposed to apply common sense and brains 
wrt. testsuites. A failing testsuite can mean nothing or everything.

Remember: Reviews are not a mere bureaucratic act. Reviewers are 
supposed to make a judgment (*review*) - This also occasionally means to 
compromise and occasionally means to reject something.

>> But reviewers can always refuse to accept something they disagree
>> with, step aside from the review, and wish the submitter good luck
>> finding someone else to take over.
>
> Well, both sides should be willing to compromise about how to continue.
Depends. There are situations, where compromises are impossible.

E.g. when a testsuite's results indicate bugs and deficienicies or even 
dysfunctionality of a package.

> The submitter cannot force the reviewer to approve the package. And the
> reviewer cannot force the submitter to change something, because the
> submitter may reply "no, thanks, then I prefer providing the package
> in my personal repo".
Depends.

> Talking about approved packages, let's not forget packagers who
> reintroduce mistakes (including disabling %check sections) some time
> later. ;-)
And even this some times is inevitable :-)

Ralf