measuring success [was Re: Bodhi 0.7.5 release]

[Adam Williamson]

On Sat, 2010-07-03 at 20:40 +0200, Michael Schwendt wrote:
 
> > That only handles a subset of the 'broken dependencies' problem. We've
> > already had an example this year of a dependency issue the proposed
> > autoqa depcheck test wouldn't catch, and Michael's script didn't - the
> > nss-softokn update 
> 
> Which one was that?
> https://bugzilla.redhat.com/596840 ?

Yep.

> > (as the broken dep is only apparent if you take into
> > consideration multilib issues and which repositories will have which
> > updated packages available).
> 
> There are multiple problems:
> 
> * Upgrades from F12 to F13. The depcheck for F13 would need to enable F12
> repos _always_ - to catch upgrade path violations that lead to dependency
> problems. I only do this a few times shortly before the release of FN+1
> (=F13).
> 
> * Yum depsolving behaviour on systems where multiarch packages are
> installed and an update isn't multiarch anymore. For example, where Yum on
> x86_64 would pull in lots of i686 packages to resolve dependencies, that
> would be considered a problem by the user but not by a depchecker, because
> there are no unresolvable dependencies to detect. Unless you teach the
> depsolver (and depchecker) that cross-arch dependencies are something
> to report as a problem. That would be more than "repository closure".
> The depchecker doesn't look for file conflicts either. There could be a
> dependency chain, which doesn't suffer from unresolvable deps but from
> implicit file conflicts.

Yep, indeed. I'm really not criticising the script. I'm just pointing
out that we know there are some pretty complex scenarios out there which
we haven't yet figured out how to test in an automated way; I'm
challenging the assumption that we can write a perfect depcheck test
which will ensure there is never a dependency issue ever again.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net