Updates Criteria Summary/Brainstorming

[Till Maas]

On Sat, Nov 20, 2010 at 03:35:43PM -0700, Kevin Fenzi wrote:

> * require testing only for packages where people have signed up to be testers
> 
> Packages without 'official' testers could bypass testing or have some lower karma
> requirement. We would need for this a list of packages that have had people sign 
> up to test. 

I guess this can be somehow automated. E.g. change Bodhi to drop the
karma requirements for packages that had e.g. two subsequent updates
without any Bodhi feedback and re-enable it if they get feedback.

> * Ask maintainers to provide test cases / test cases in wiki for each package?
> 
> Test cases are not easy to make, many maintainers won't can't do so, but 
> it would be lovely to have even a base checklist of things that should work 
> in the package everytime. 

Afaik, most of the packages won't be tested and therefore the test cases
won't be read. So this should be only a requirement if there are testers
for a certain package.

> * reduced karma requirement on other releases when one has gone stable
> 
> Bodhi would need to note when a update went to stable if the exact same
> version (with dist tag differences) was in testing for other releases.
> It could then allow less karma to go stable, or add +1 from the other
> update going stable. 
> 
> Other concrete ideas?

All of this could be combined. E.g. packages with enough testers get
test cases and need to fulfill stronger criteria. Packages with not so
many testers get test cases and only need to fulfil that similar
updates need to receive good karma on one Fedora release.

Off course more automated testing is missing instead of all the manual
requirements.

Another idea would be to add a flag in bodhi to track whether a certain
testing update has been installed and then have a tool to report this
automatically back to Bodhi. Then there would be some information about
silent testers, which can be used as a criterion, too.

Also it could be made easier for maintainers to identify problematic
updates, e.g. by warning that the dependencies or provides of an update
changed when the update is created.

Regards
Till
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/devel/attachments/20101121/316a6c1b/attachment.bin