Package Update Acceptance Test Plan - final call - introspection

[James Laska]

On Fri, 2010-04-23 at 09:06 -0400, Kamil Paral wrote:
> > > = Introspection tests =
> > > 
> > > 4. Rpmlint - "no errors present" vs "no new errors present": It
> > > is obvious we have to provide an option for package maintainers to
> > > whitelist some rpmlint messages. The actual implementation of the
> > > whitelist is still to be discovered, but that doesn't matter, it
> > > will be there. In this respect is seems to me that "no new errors"
> > > requirement has no benefit over "no errors" requirement, because
> > > the latter one does everything the prior one and even more. When
> > > it is possible to whitelist errors then there's no reason for us
> > > to allow any errors in the output.
> > > Implementation note: "no new errors" is more an rpmguard's task
> > > than rpmlint's. We can implement that as an rpmguard check and put
> > > it to introspection tests, that's possible. But it's not needed if
> > > we agree on "no errors" requirement.
> > 
> > "No new errors" seems like an achievable short-term approach to add
> > value while not overloading the maintainer with a potentially large
> > list
> > of rpmlint failures (/me thinking kernel) that they haven't been
> > tracking since the package was initially incorporated into Fedora. 
> > Down
> > the road, I agree a whitelist mechanism would be ideal (as well as a
> > shortcut to file a bug against rpmlint).
> 
> The whitelist mechanism is surely a must-have, otherwise we would
> end up impaled on a stakes, quartered and burnt alive :)
> 
> "No new errors" could work for some packages, but it wouldn't work
> for many others, especially when a version number is mentioned in the
> message - it will change for every release, so we would see it
> as a new error anyway. Kernel package is a nice example of this.
> A few references:
> 
> https://fedorahosted.org/pipermail/autoqa-results/2010-April/013921.html
> https://fedorahosted.org/pipermail/autoqa-results/2010-April/016838.html
> https://fedorahosted.org/pipermail/autoqa-results/2010-April/016436.html 

Ah good point.  We may wish to adapt any rpmguard tests that compare
file paths to account for the package version or release in the path.
Same for when comparing versioned package %provides.  I have seen some
implementations that globally replace any instances of the package
version (and/or release) in all file paths with some string (e.g.
VERREL) to avoid cases like you mention.  This seems like worthwhile
improvement in addition to a mechanism for white listing results?

Thanks,
James

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
Url : http://lists.fedoraproject.org/pipermail/test/attachments/20100423/e8d59f4f/attachment.bin