Package Update Acceptance Test Plan - final call - introspection

[Kamil Paral]

----- "James Laska" <jlaska at redhat.com> wrote:

> On Fri, 2010-04-23 at 09:06 -0400, Kamil Paral wrote:
> > > > = Introspection tests =
> > > > 
> > > > 4. Rpmlint - "no errors present" vs "no new errors present": It
> > > > is obvious we have to provide an option for package maintainers
> to
> > > > whitelist some rpmlint messages. The actual implementation of
> the
> > > > whitelist is still to be discovered, but that doesn't matter,
> it
> > > > will be there. In this respect is seems to me that "no new
> errors"
> > > > requirement has no benefit over "no errors" requirement,
> because
> > > > the latter one does everything the prior one and even more.
> When
> > > > it is possible to whitelist errors then there's no reason for
> us
> > > > to allow any errors in the output.
> > > > Implementation note: "no new errors" is more an rpmguard's task
> > > > than rpmlint's. We can implement that as an rpmguard check and
> put
> > > > it to introspection tests, that's possible. But it's not needed
> if
> > > > we agree on "no errors" requirement.
> > > 
> > > "No new errors" seems like an achievable short-term approach to
> add
> > > value while not overloading the maintainer with a potentially
> large
> > > list
> > > of rpmlint failures (/me thinking kernel) that they haven't been
> > > tracking since the package was initially incorporated into Fedora.
> 
> > > Down
> > > the road, I agree a whitelist mechanism would be ideal (as well as
> a
> > > shortcut to file a bug against rpmlint).
> > 
> > The whitelist mechanism is surely a must-have, otherwise we would
> > end up impaled on a stakes, quartered and burnt alive :)
> > 
> > "No new errors" could work for some packages, but it wouldn't work
> > for many others, especially when a version number is mentioned in
> the
> > message - it will change for every release, so we would see it
> > as a new error anyway. Kernel package is a nice example of this.
> > A few references:
> > 
> >
> https://fedorahosted.org/pipermail/autoqa-results/2010-April/013921.html
> >
> https://fedorahosted.org/pipermail/autoqa-results/2010-April/016838.html
> >
> https://fedorahosted.org/pipermail/autoqa-results/2010-April/016436.html
> 
> 
> Ah good point.  We may wish to adapt any rpmguard tests that compare
> file paths to account for the package version or release in the path.
> Same for when comparing versioned package %provides.  I have seen
> some
> implementations that globally replace any instances of the package
> version (and/or release) in all file paths with some string (e.g.
> VERREL) to avoid cases like you mention.  This seems like worthwhile
> improvement in addition to a mechanism for white listing results?

Good idea! Created tickets:
https://fedorahosted.org/autoqa/ticket/148
https://fedorahosted.org/autoqa/ticket/149