updates improvements/changes ideas

[Matthias Runge]

On 30/11/10 10:51, Michael Schwendt wrote:
> On Tue, 30 Nov 2010 09:31:31 +0100, Matthias wrote:
> 
>> On 29/11/10 19:08, Adam Williamson wrote:
>>> On Mon, 2010-11-29 at 12:40 -0500, James Laska wrote:
>>
>>> Things that some people see as problematic are:
>>>
>>> * Having to wait a week to push an update if you can't find testing
>>> * Testing being required for packages with automated test suites
>>> * The delay to security updates which is introduced by the testing
>>> requirements
>>
>> Testing would get much easier, if packagers could provide some test
>> cases. The packager could send mail to -devel or -testing to get some
>> testers.
> 
> Sounds backwards to me. Given the life cycle of a bug, there is activity
> in bugzilla prior to the maintainer developing a fix. Plus, bodhi adds
> update notifications to bugzilla. If I were to expect someone to test the
> fix, it would be the bug reporter to be the additional tester.
> Even more so if the fix is released by upstream. In that case, coming
> up with test-cases would be a lot of extra(/duplicate?) work for
> package maintainers, especially for version upgrades that contain
> plenty of fixes and changes.

The bug reporter will probably verify the fixture for his bug. But,
given the fact, this new version breaks other things, this could be
possibly covered by a test case (in best case).

IMHO you should write test cases only once (a general sheet, what to test).
I see, we need to get more testers. But those people need to know what
to test, sometimes even: how to test.... If we don't provide those
information, we will get a lot of feedback: +1, "works for me". What do
we really know then? What was tested? Does it prevent us from shipping
broken updates? Definitely not.