Proven tester special testing procedures: PackageKit?

[Adam Williamson]

We've talked about special proven testers testing procedures for
kernels, and now another candidate for special test procedures has
emerged: PackageKit. A bug in PackageKit got out in a Fedora 13 update
which broke update notification, which is obviously a big problem.
PackageKit isn't actually on the critical path for F13, but it is for
F14 and up.

So, I'd like to propose a special testing procedure for updates to
PackageKit (actually, any package management-related critpath update
should be tested in this way). As well as testing package installation,
we should test update notification and installation for such updates. We
can consider different ways of doing this, but an easy way would be
simply to 'yum downgrade' an installed package, then wait: within a
short time (I'll have to check how long with Richard, I think 30 minutes
or an hour), an update notification should appear to let you know an
update is available to that package. If not, we have problems.

If everyone thinks this makes sense, I'll propose a wiki draft which
sets up a system for package-specific testing procedures and drafts such
a procedure for PackageKit. What does everyone think of this? Thanks!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

[Adam Miller]

+1

Makes perfect sense to me and I think it would be a wise move.

-AdamM

-- 
http://maxamillion.googlepages.com
---------------------------------------------------------
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments

[James Cassell]

On Thu, 29 Jul 2010 17:01:36 -0400, Adam Williamson <awilliam(a)redhat.com&gt;  
wrote:

>  As well as testing package installation,
>  we should test update notification and installation for such updates. We
>  can consider different ways of doing this, but an easy way would be
>  simply to 'yum downgrade' an installed package, then wait: within a
>  short time (I'll have to check how long with Richard, I think 30 minutes
>  or an hour), an update notification should appear to let you know an
>  update is available to that package. If not, we have problems. 
+1

Sounds good to me, and preventing breakage is a Good Thing (TM)


-- 
James Cassell

[Rares Aioanei]

On Thu, 29 Jul 2010 17:13:50 -0400
"James Cassell" <fedoraproject(a)cyberpear.com&gt; wrote:

>  On Thu, 29 Jul 2010 17:01:36 -0400, Adam Williamson
> <awilliam(a)redhat.com&gt;  
>  wrote:
>  
>  > As well as testing package installation,
>  > we should test update notification and installation for such updates. We
>  > can consider different ways of doing this, but an easy way would be
>  > simply to 'yum downgrade' an installed package, then wait: within a
>  > short time (I'll have to check how long with Richard, I think 30 minutes
>  > or an hour), an update notification should appear to let you know an
>  > update is available to that package. If not, we have problems.
>   
+1

-- 
Rares Aioanei <fedora.listen(a)gmail.com&gt;

[John Dulaney]

Good idea, me thinks.


>  Subject: Proven tester special testing procedures: PackageKit?
>  From: awilliam(a)redhat.com
>  To: test(a)lists.fedoraproject.org
>  Date: Thu, 29 Jul 2010 14:01:36 -0700
>  
>  We've talked about special proven testers testing procedures for
>  kernels, and now another candidate for special test procedures has
>  emerged: PackageKit. A bug in PackageKit got out in a Fedora 13 update
>  which broke update notification, which is obviously a big problem.
>  PackageKit isn't actually on the critical path for F13, but it is for
>  F14 and up.
>  
>  So, I'd like to propose a special testing procedure for updates to
>  PackageKit (actually, any package management-related critpath update
>  should be tested in this way). As well as testing package installation,
>  we should test update notification and installation for such updates. We
>  can consider different ways of doing this, but an easy way would be
>  simply to 'yum downgrade' an installed package, then wait: within a
>  short time (I'll have to check how long with Richard, I think 30 minutes
>  or an hour), an update notification should appear to let you know an
>  update is available to that package. If not, we have problems.
>  
>  If everyone thinks this makes sense, I'll propose a wiki draft which
>  sets up a system for package-specific testing procedures and drafts such
>  a procedure for PackageKit. What does everyone think of this? Thanks!
>  -- 
>  Adam Williamson
>  Fedora QA Community Monkey
>  IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
>  http://www.happyassassin.net
>  
>  -- 
>  test mailing list
>  test(a)lists.fedoraproject.org
>  To unsubscribe: 
>  https://admin.fedoraproject.org/mailman/listinfo/test

[Richard Hughes]

On 29 July 2010 23:01, Adam Williamson <awilliam(a)redhat.com&gt; wrote:
>  If everyone thinks this makes sense, I'll propose a wiki draft
> which
>  sets up a system for package-specific testing procedures and drafts such
>  a procedure for PackageKit. What does everyone think of this? Thanks! 
Makes sense to me, I certainly need all the help I can get. For the
next release and onwards I'm going to try and keep the session updater
more in sync with the daemon version (with an artificially high BR) to
try and keep the testing matrix of old versions to new versions sane.
Thanks guys.

Richard.

[Masami Ichikawa]

On Fri, Jul 30, 2010 at 6:01 AM, Adam Williamson <awilliam(a)redhat.com&gt; wrote:
>  If everyone thinks this makes sense, I'll propose a wiki draft
> which
>  sets up a system for package-specific testing procedures and drafts such
>  a procedure for PackageKit. What does everyone think of this? Thanks! 
+1
Sounds good to me. It's good idea for keeping Fedora better!

-- 
/*
 * Masami Ichikawa
 * mailto: masami256(a)gmail.com
 */

[Jon Stanley]

On Thu, Jul 29, 2010 at 5:01 PM, Adam Williamson <awilliam(a)redhat.com&gt; wrote:
>  . We
>  can consider different ways of doing this, but an easy way would be 
I think that a reasonable way of accomplishing this (I may be insane
though :) ) is to have a repo that has a newer version of a package
that everyone has installed (say fedora-release) that could be enabled
by the tester (perhaps the repo definition could even be in a
tester-setup package).

This package could also contain scripts to validate the test setup of
the machine in various ways, and to remediate if required. Not sure
what a good set of "tests for the tester" is, but food for thought.

[Rahul Sundaram]

 On 07/30/2010 02:31 AM, Adam Williamson wrote:
>  We've talked about special proven testers testing procedures for
>  kernels, and now another candidate for special test procedures has
>  emerged: PackageKit. A bug in PackageKit got out in a Fedora 13 update
>  which broke update notification, which is obviously a big problem. 
Would sending a note to fedora-announce with more details and the
followup from QA on what is being done to avoid this problem in the
future as well.

Rahul

[Michael Cronenworth]

  On 07/29/2010 04:01 PM, Adam Williamson wrote:
>  A bug in PackageKit got out in a Fedora 13 update
>  which broke update notification, which is obviously a big problem.
>  PackageKit isn't actually on the critical path for F13, but it is for
>  F14 and up. 
Is there a big tracking this? Was it discussed on IRC only? I see 
nothing in the PackageKit bodhi update or devel/testing lists mentioning 
this until your e-mail, which is not very detailed.

I just ran today's updates with gnome-packagekit successfully. On one of 
my home systems, where I do a majority of my QA time, I saw no issue and 
reported positive karma. I only use gnome-packagekit on that system 
specifically to catch any bugs.

Thanks,
Michael

[drago01]

On Fri, Jul 30, 2010 at 10:32 PM, Michael Cronenworth <mike(a)cchtml.com&gt; wrote:
>   On 07/29/2010 04:01 PM, Adam Williamson wrote:
> > A bug in PackageKit got out in a Fedora 13 update
> > which broke update notification, which is obviously a big problem.
> > PackageKit isn't actually on the critical path for F13, but it is for
> > F14 and up.
>
>  Is there a big tracking this? Was it discussed on IRC only? 
https://fedorahosted.org/fesco/ticket/440

[Adam Williamson]

On Thu, 2010-07-29 at 14:01 -0700, Adam Williamson wrote:
>  We've talked about special proven testers testing procedures for
>  kernels, and now another candidate for special test procedures has
>  emerged: PackageKit. A bug in PackageKit got out in a Fedora 13 update
>  which broke update notification, which is obviously a big problem.
>  PackageKit isn't actually on the critical path for F13, but it is for
>  F14 and up.
>  
>  So, I'd like to propose a special testing procedure for updates to
>  PackageKit (actually, any package management-related critpath update
>  should be tested in this way). As well as testing package installation,
>  we should test update notification and installation for such updates. We
>  can consider different ways of doing this, but an easy way would be
>  simply to 'yum downgrade' an installed package, then wait: within a
>  short time (I'll have to check how long with Richard, I think 30 minutes
>  or an hour), an update notification should appear to let you know an
>  update is available to that package. If not, we have problems.
>  
>  If everyone thinks this makes sense, I'll propose a wiki draft which
>  sets up a system for package-specific testing procedures and drafts such
>  a procedure for PackageKit. What does everyone think of this? Thanks! 
Quick update on this - I've been ambushed by F14 Alpha stuff. I
certainly intend to move this forward when I have time, but if anyone
else feels inclined to draft something up, please don't wait for me, go
right ahead and do it! Thanks.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net