Proposals for the Updates Testing Procedure

[Warren Togami]

Michael K. Johnson wrote:
> On Wed, Nov 12, 2003 at 09:48:15PM -1000, Warren Togami wrote:
> 
>>Along with each announced test update, I believe it would be crucial to 
>>include a link to a corresponding Bugzilla report.  While longer 
> 
> 
> This is probably a good idea, even if we need to create an
> empty report for discussion.
> 

Or fill out the empty report with content from a simple template.  I'd 
say information about why an upgrade, and perhaps the patch, etc.

>>* Time-limit to publish where no negative comments are posted within the 
>>Bugzilla report.  Senior developers reserve the right to hold an update 
>>  if a good technical reason can be stated.  (Insert more details here.)
> SNIP
> 
> Time doesn't really tell us anything.  I'd like us to be able to get
> some information on downloads, and then consider the time since some
> reasonable number of downloads have been done.  The amount of testing
> that is going to be needed will depend on the package, and I'd think
> that the package maintainer is going to be a good judge of how much
> testing is needed.  So I'd say we should come up with guidelines, but
> be flexible.  We need to look into providing the data on which to
> base good decisions.
> 
> Also, security updates are clearly going to need to be handled differently
> from bugfix updates, and new feature updates also differently.  Security
> updates need the most expeditious handling; bugfix less so, and new feature
> updates will tend to need less-expeditious handling *and* a larger amount
> of testing.

I guess I was mainly concerned about update packages sitting forever in 
testing only because too few people care enough to test & comment on it.
https://bugzilla.fedora.us/show_bug.cgi?id=35
I'd really hate to avoid cases like this... where we have a package 
sitting in limbo for almost 9 months.

I'd say that 1-2 months would be a fair time limit for update testing. 
At some point, maybe a week before the expiration date, there should be 
warnings posted for a last call for testing.  This would be an important 
safeguard for the community so updates cannot be snuck in by default easily.

In most cases however packages would receive enough testing to make 
publication far quicker and never necessitate invoking the expiration date.

Warren