F23: # dnf update dnf* rpm* ==> no repos configured

[Adam Williamson]

On Fri, 2015-09-25 at 18:53 +0100, Russel Winder wrote:
> On Fri, 2015-09-25 at 11:16 +0200, Joachim Backes wrote:
> > […]
> > 
> > Did you try 
> > http://koji.fedoraproject.org/koji/buildinfo?buildID=687661 
> > (dnf-1.1.2-3.fc23)?
> 
> I hadn't, but I just did a "yum-deprecated update" then "yum-
> deprecated
> update" and I got:
> 
>  	dnf.noarch 0:1.1.2-3.fc24
> 
> and it's dependencies, which I believe is the putative upgrade that
> works. Certainly "dnf list kernel" works again.
> 
> Moral of this story: dnf test suite is sub-standard and the CI and
> upload policy is in need of some adjustment

So far as I can see, no tests were actually run on this commit. Here's
the PR for it:

https://github.com/rpm-software-management/dnf/pull/352

compare to a PR where tests were run:

https://github.com/rpm-software-management/dnf/pull/360

it seems that they don't run tests automatically on PR submission,
don't know why not.

The package does have a %check which runs 'make test' though, so if the
tests were going to catch this, they should have caught it there.
However, it seems the tests indeed don't cover the problem here. The
'make test' output is identical between -2 and -3:

https://kojipkgs.fedoraproject.org//packages/dnf/1.1.2/2.fc24/data/logs/noarch/build.log
https://kojipkgs.fedoraproject.org//packages/dnf/1.1.2/3.fc24/data/logs/noarch/build.log

in both cases it seems 545 tests pass for python 2, 543 pass and 2 are
skipped for python 3.

>  – although Fedora Rawhide
> is "on the edge" and "seat of the pants" dnf is the single most
> important bit, and so the tests and CI should reflect this.

Certainly it's always good to keep improving the test suites, though I
find it difficult to assume that any failure of a test to catch a bug
means the tests are 'sub-standard'. Testing is hard, and nobody's
perfect.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net