Provide more testing feedback (was: Re: Refining the update queues/process)

[Till Maas]

On Wed, Mar 03, 2010 at 02:06:33PM -0800, Adam Williamson wrote:
> On Wed, 2010-03-03 at 17:04 +0100, Till Maas wrote:
> 
> > I mind have misunderstood it, but afaics it only says that it will be
> > tested, because it spent time in updates-testing, but this is not even
> > true nowadays, even if packages stay long in updates-testing.
> 
> as we've explained several times, most packages that go to
> updates-testing for a few days *are* being tested, even if they get no
> apparent Bodhi feedback. Several QA group members run with
> updates-testing enabled and so get all packages (that they have
> installed) which go through updates-testing. They do not file positive

If it is limited on the packages the user is installed, a lot of
packages are not tested.

> feedback for every single package because there's just too many, but if
> they notice breakage, they file negative feedback.

So here is a first ugly script to easily give feedback for all installed
testing updates that were created after a certain date (I did not find
an easy way to get all testing updates, one did not yet comment on):
http://till.fedorapeople.org/tmp/give-feedback.py

It kind of works like this: Show the update, ask for karma to create a
comment, otherwise it is skipped. Then a comment needs to be entered.
Sometimes it will ask for the FAS password using the bodhi client.

Update: rpmdevtools-7.8-1.fc12
Comment? -1/0/1 ->karma, other -> skip> 1
Comment> "rpmdev-newspec foo" still works

The prompt has readline support, so if you just want to use the same
comment again and again, you can use cursor-up to get the comment again.

Btw. the script fails in case only subpackages of a package are
installed. But if it is used, I will create a saner version.
Nevertheless, there is also now the Fedora Engineering Service, so if
you want to improve your testing tools, you might want to ask there.

> So - for the third time - a package being in updates-testing for a few
> days and getting no negative feedback is a moderate strength indicator
> that it's not egregiously broken. Not a super-strong indicator, but
> better than a kick in the teeth.

It probably only means that the meta-data of the installed package is
not broken, but if they do not use all packages installed daily, then
there is not much test coverage.

Regards
Till
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/devel/attachments/20100304/3b7944b7/attachment-0001.bin