Update testing policy: how to use Bodhi
by Adam Williamson
Hi, folks. At the last QA meeting, I volunteered (dumb of me!) to draft
a policy for testing updates - basically, a policy for what kind of
feedback should be posted in Bodhi for candidate updates.
This turns out to be pretty hard. =) Thinking about it from an
high-level perspective like this, I think it becomes pretty clear that
the current system is just broken.
The major problem is it attempts to balance things that don't really
balance. It lets you say 'works for me' or 'doesn't work' and then sums
the two and subtracts the second from the first to give you a 'rating'
for the update.
This doesn't really mean anything. As has been rehashed many times,
there are situations where an update with a positive rating shouldn't go
out, and situations where an update with a negative rating should. So
the current system isn't really that great.
I can't think of a way to draft a policy to guide the use of the current
system in such a way that it will be really reliable. I think it'd be
much more productive to revise the Bodhi feedback system alongside
producing a policy.
So, here's a summary of what the new system should aim for.
At the high level, what is this system for? It's there for three
purposes:
1) to provide maintainers with information they can use in deciding
whether to push updates.
2) to provide a mechanism for mandating a certain minimum level of
manual testing for 'important' packages, under Bill Nottingham's current
update acceptance criteria proposal.
3) to provide an 'audit trail' we can use to look back on how the
release of a particular update was handled, in the case where there are
problems.
Given the above, we need to capture the following types of feedback, as
far as I can tell. I don't think there is any sensible way to assign
numeric values to any of this feedback. I think we have to trust people
to make sensible decisions as long as it's provided, in accordance with
any policy we decide to implement on what character updates should have.
1. I have tried this update in my regular day-to-day use and seen no
regressions.
2. I have tried this update in my regular day-to-day use and seen a
regression: bug #XXXXXX.
3. (Where the update claims to fix bug #XXXXXX) I have tried this update
and found that it does fix bug #XXXXXX.
4. (Where the update claims to fix bug #XXXXXX) I have tried this update
and found that it does not fix bug #XXXXXX.
5. I have performed the following planned testing on the update: (link
to test case / test plan) and it passes.
6. I have performed the following planned testing on the update: (link
to test case / test plan) and it fails: bug #XXXXXX.
Testers should be able to file multiple types of feedback in one
operation - for instance, 4+1 (the update didn't fix the bug it claimed
to, but doesn't seem to cause any regressions either). Ideally, the
input of feedback should be 'guided' with a freeform element, so there's
a space to enter bug numbers, for instance.
There is one type of feedback we don't really want or need to capture:
"I have tried this update and it doesn't fix bug #XXXXXX", where the
update doesn't claim to fix that bug. This is a quite common '-1' in the
current system, and one we should eliminate.
I think Bill's proposed policy can be modified quite easily to fit this.
All it would need to say is that for 'important' updates to be accepted,
they would need to have one 'type 1' feedback from a proven tester, and
no 'type 2' feedback from anyone (or something along those lines; this
isn't the main thrust of my post, please don't sidetrack it too
much :>).
The system could do a count of how many of each type of feedback any
given update has received, but I don't think there's any way we can
sensibly do some kind of mathematical operation on those numbers and
have a 'rating' for the update. Such a system would always give odd /
undesirable results in some cases, I think (just as the current one
does). I believe the above system would be sufficiently clear that there
would be no need for such a number, and we would be able to evaluate
updates properly based just on the information listed.
What are everyone's thoughts on this? Thanks!
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
14 years
F13-problem with graphical boot: box hangs before gdm greeter appears
by Joachim Backes
Hi,
Having a video card of type Nvidia GeForce FX5200 (PCI), and running my
F13 with nouveau.
1. If booting graphically (RHGB), then my box *hangs* immediately before
the gdm greeter should appear. No input possible, only Ctrl+Alt+Del for
reboot (If pressing ESC during the RHGB boot, I can get rid from this
behaviour, similarly to boot without RHGB). This happens since the
yesterday's (friday) updates.
2. No such behaviour if booting completely without RHGB. Then the gdm
login runs properly.
Anybody has an explanation for this? Any comments are welcome.
--
Joachim Backes <joachim.backes(a)rhrk.uni-kl.de>
http://www.rhrk.uni-kl.de/~backes
14 years
New BugZapper Introduction
by Gowrishankar Rajaiyan
Hi All,
I Gowrishankar from India, am very happy to join the Fedora Bug Zappers
Team. I have been working with fedora since 2004 (basically on the
server side of it). Now since a year have also started using Fedora on
the desktop side too. I feel its high time for me to contribute to the
Fedora project and do my bit.
Recently joined RedHat's QE team and you can find me as "shanks" AT
irc.freenode.net or can reach me at gsr(a)redhat.com
Happy testing :)
--
Gowrishankar Rajaiyan <gsr(a)redhat.com>
14 years
Graphical Desktop Install Fails Fedora 13 Beta RC#1 i386
by Bob Lightfoot
Fedora-13-Beta.RC1-i386-DVD missing dependancy for Graphical Desktop
Install.
gnome-packagekit-2.29.91-3.fc13.i686 requires libpackagekit-glib2.so.13
Installing to VirtualBox-OSE 3.1.4 Guest Drive on F12_x86_64 host system
Bob Lightfoot
14 years
rsync for serverbeach1?
by Andre Robatino
Would it be possible to set up rsync for serverbeach1? The reason I ask is that
it could be used to very efficiently convert between a DVD and the corresponding
CD set, which might encourage more testing of the CDs (which seem to be having
problems lately). I verified this by using one of the regular Fedora mirrors
with rsync to convert in both directions between the F12 DVD and the F12 CD set.
It would also give someone with a corrupted download a better way to fix it
than downloading from scratch. (I've personally never had it happen so far, but
it could.)
14 years
F-13 Branched report: 20100330 changes
by Fedora compose checker
Compose started at Tue Mar 30 09:15:05 UTC 2010
Broken deps for i386
----------------------------------------------------------
gnome-web-photo-0.9-4.fc13.i686 requires gecko-libs = 0:1.9.2.1
hornsey-1.5.2-0.1.fc13.i686 requires libclutter-gst-0.10.so.0
libnodeupdown-backend-openib-1.9-5.fc13.i686 requires libosmcomp.so.3(OSMCOMP_2.3)
libnodeupdown-backend-openib-1.9-5.fc13.i686 requires libosmcomp.so.3
moblin-app-installer-0.4.0-0.7.fc13.i686 requires libpackagekit-glib2.so.13
pyclutter-gst-0.9.2-1.fc12.i686 requires libclutter-gst-0.10.so.0
sepostgresql-8.4.2-2488.fc13.i686 requires postgresql-server = 0:8.4.2
totem-youtube-2.29.92-1.fc13.i686 requires libgdata.so.7
zikula-module-menutree-2.2-1.fc13.noarch requires zikula >= 0:1.2
Broken deps for x86_64
----------------------------------------------------------
gnome-web-photo-0.9-4.fc13.x86_64 requires gecko-libs = 0:1.9.2.1
hornsey-1.5.2-0.1.fc13.x86_64 requires libclutter-gst-0.10.so.0()(64bit)
libnodeupdown-backend-openib-1.9-5.fc13.x86_64 requires libosmcomp.so.3(OSMCOMP_2.3)(64bit)
libnodeupdown-backend-openib-1.9-5.fc13.x86_64 requires libosmcomp.so.3()(64bit)
moblin-app-installer-0.4.0-0.7.fc13.x86_64 requires libpackagekit-glib2.so.13()(64bit)
pyclutter-gst-0.9.2-1.fc12.x86_64 requires libclutter-gst-0.10.so.0()(64bit)
sepostgresql-8.4.2-2488.fc13.x86_64 requires postgresql-server = 0:8.4.2
totem-youtube-2.29.92-1.fc13.x86_64 requires libgdata.so.7()(64bit)
zikula-module-menutree-2.2-1.fc13.noarch requires zikula >= 0:1.2
New package 4ti2
A software package for problems on linear spaces
New package jsmath-fonts
A collection of Math symbol fonts
New package mpir
A library for arbitrary precision arithmetic
New package normaliz
A tool for mathematical computations
New package rsvndump
Remote Subversion repository dumping tool
New package rubygem-bunny
Another synchronous Ruby AMQP client
New package rubygem-haml
An elegant, structured XHTML/XML templating engine
New package rubygem-moneta
A unified interface to key/value stores
New package rubygem-ohai
Profiles your system and emits JSON
New package spill
A segregated package install logical linker
New package sugar-view-slides
Image serie viewer for Sugar
Summary:
Added Packages: 11
Removed Packages: 0
Modified Packages: 0
14 years
xorg-x11-fonts-ISO8859-1-100dpi not found by yum
by Chuck Anderson
Fedora 12 Everything/x86_64 repo has xorg-x11-fonts-ISO8859-1-100dpi
in it, but I can't seem to install it:
#yum --disablerepo=\* --enablerepo='fedora' --noplugins install xorg-x11-fonts-ISO8859-1-100dpi
Setting up Install Process
No package xorg-x11-fonts-ISO8859-1-100dpi available.
Nothing to do
#yum --noplugins info xorg-x11-fonts-ISO8859-1-100dpi
Error: No matching Packages to list
Anyone else having this same issue? The package is there in the repo
and seems to be in the repodata (but I'm not sure how to verify the
sqlite version of the repodata).
14 years, 1 month
VirtualBox in F13
by Joachim Backes
Hi,
anybody knows when VirtualBox-OSE will be available in F13? I think, in
F12 it is available from the rpmfusion-??? repos.
Kind regards
Joachim Backes <joachim.backes(a)rhrk.uni-kl.de>
http://www.rhrk.uni-kl.de/~backes
14 years, 1 month
