A Glorious Vision of Our Shared Update Feedback Future (bodhi, karma, and proventesters, oh my)

["Jóhann B. Guðmundsson"]

On 11/22/2011 09:53 PM, Adam Williamson wrote:
> On Tue, 2011-11-22 at 21:31 +0000, "Jóhann B. Guðmundsson" wrote:
>> On 11/22/2011 09:03 PM, Adam Williamson wrote:
>>> 2. Any update marked as 'critpath breaking' by a proven tester would be
>>> blocked from being pushed stable at all - automatically or manually -
>>> until the PT modified the feedback or it was overridden by someone with
>>> appropriately godlike powers (TBD, but probably not just the maintainer)
>> Hum I thought the proven tester concept was being dropped anyway.
> At present that seems like where we're going, but I ignored it for the
> purposes of the Glorious Vision. The Glorious Vision does, I think,
> provide a reasonable suggestion as to how PTs could be useful if we had
> more detailed karma possibilities.
>
>> A) Have maintainers for critical path component provided test cases for
>> proven tester to use to test to differentiate them from fas-tester and
>> the "I installed this update and continued to use my system as normal
>> and did not note any regressions'"?
> In many cases no, but the nice thing about critpath is that in most
> cases we don't really need that. For some tricky components it'd be nice
> to have test cases that effectively explain exactly how that component
> intersects with critpath, but for almost all critpath components, I
> can't see that it would ever be wrong to hit the 'panic button' if you
> installed the update, rebooted, and the system blew up, for instance.
> The nice thing about a system based on *negative* feedback is that while
> it can be less than optimally efficient, it's almost always useful to at
> least some degree even when it's not perfect.
>
>> B) Has the scalability problem be solved as in the required minimum
>> amount of active proven testers has been met for this to actually work?
> See above: again the nice thing here is we don't _need_ a critical mass
> for a mechanism based on negative rather than positive results. Of
> course, the more testers we have the more issues we will likely
> identify, but a panic button is useful even if it only ever gets pressed
> once (as long as the press isn't a false one). Even if there were 99
> other times when it _could_ have been pressed and wasn't, 1/100 success
> rate is better than 0/100.

With the above information what benefits/value will we have by having 
proven tester over fas tester hitting the panic button
( since no addinal testing is being performed by the proven tester over 
fas-tester thus it makes no difference if fas-tester or proven tester 
hits the panic button)

JBG