I went to work a bit on jdulaney's fork of the proven tester page to make the mentorship-merging stuff fit a little more smoothly, and I sort of got carried away. I'd normally just add this as a new revision for Proven_testers, but it's pretty substantially revised. It probably needs to get re-reviewed:
https://fedoraproject.org/wiki/User:Dafrito/Proven_tester
I can write up a summary of the major differences, but I figure I should only toss out one wall-of-text at a time. ;) Basically, I took the current one, and tried to expand on places that were shallow (like testing criteria) and made other sections flow a lot better (like the old Feedback section).
I feel the need to mention I refer to critpath packages as just critical packages and critical updates, instead of critical path packages and critical path updates. This is purely a stylistic change: when I was reading it out loud, I trip over the alliteration of "path packages" a lot. Let me know if this (or anything else) is a mortal sin.
-- Aaron Faanes
I like it J. H. Dulaney
From: dafrito@gmail.com Date: Mon, 5 Jul 2010 19:33:35 -0500 Subject: Proven tester wiki love To: test@lists.fedoraproject.org
I went to work a bit on jdulaney's fork of the proven tester page to make the mentorship-merging stuff fit a little more smoothly, and I sort of got carried away. I'd normally just add this as a new revision for Proven_testers, but it's pretty substantially revised. It probably needs to get re-reviewed:
https://fedoraproject.org/wiki/User:Dafrito/Proven_tester
I can write up a summary of the major differences, but I figure I should only toss out one wall-of-text at a time. ;) Basically, I took the current one, and tried to expand on places that were shallow (like testing criteria) and made other sections flow a lot better (like the old Feedback section).
I feel the need to mention I refer to critpath packages as just critical packages and critical updates, instead of critical path packages and critical path updates. This is purely a stylistic change: when I was reading it out loud, I trip over the alliteration of "path packages" a lot. Let me know if this (or anything else) is a mortal sin.
-- Aaron Faanes
test mailing list test@lists.fedoraproject.org To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
_________________________________________________________________ The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=P...
On Mon, 2010-07-05 at 21:12 -0400, John Dulaney wrote:
I like it
J. H. Dulaney
From: dafrito@gmail.com Date: Mon, 5 Jul 2010 19:33:35 -0500 Subject: Proven tester wiki love To: test@lists.fedoraproject.org
I went to work a bit on jdulaney's fork of the proven tester page to make the mentorship-merging stuff fit a little more smoothly, and I sort of got carried away. I'd normally just add this as a new
revision
for Proven_testers, but it's pretty substantially revised. It
probably
needs to get re-reviewed:
https://fedoraproject.org/wiki/User:Dafrito/Proven_tester
I can write up a summary of the major differences, but I figure I should only toss out one wall-of-text at a time. ;) Basically, I
took
the current one, and tried to expand on places that were shallow
(like
testing criteria) and made other sections flow a lot better (like
the
old Feedback section).
I feel the need to mention I refer to critpath packages as just critical packages and critical updates, instead of critical path packages and critical path updates. This is purely a stylistic
change:
when I was reading it out loud, I trip over the alliteration of
"path
packages" a lot. Let me know if this (or anything else) is a mortal sin.
Thanks for drafting.
I agree with John. I like the updates draft. I made a few *minor* wiki changes, feel free to revert if desired. This new draft feels less like two distinct pages, and more like something that developed on its own. This might be more reflective of my reading style, but I'm trying to think of ways to organize the "Providing feedback" content in a way that is easy to scan/reference content.
I was thinking about organizing your existing "Providing feedback" content into 3 sub-sections:
== Providing feedback == ... === Negative karma === ... === Neutral karma === ... === Positive karma === ...
You have some good examples for the appropriate type of feedback. It might seem obvious to some, but it's not always clear what type of feedback to give. Does this clutter your draft too much? You are more adept at organizing wiki content, so I trust your instincts.
Thanks, James
On Mon, 2010-07-05 at 19:33 -0500, Aaron Faanes wrote:
I went to work a bit on jdulaney's fork of the proven tester page to make the mentorship-merging stuff fit a little more smoothly, and I sort of got carried away. I'd normally just add this as a new revision for Proven_testers, but it's pretty substantially revised. It probably needs to get re-reviewed:
https://fedoraproject.org/wiki/User:Dafrito/Proven_tester
I can write up a summary of the major differences, but I figure I should only toss out one wall-of-text at a time. ;) Basically, I took the current one, and tried to expand on places that were shallow (like testing criteria) and made other sections flow a lot better (like the old Feedback section).
I feel the need to mention I refer to critpath packages as just critical packages and critical updates, instead of critical path packages and critical path updates. This is purely a stylistic change: when I was reading it out loud, I trip over the alliteration of "path packages" a lot. Let me know if this (or anything else) is a mortal sin.
I understand - I've had the same feeling - but I think we may want to stick with the 'critical path' naming, as it's what's used elsewhere in Fedora-land.
I'm not absolutely in love with the 'stable update' concept introduced here, to be honest. Personally I'd prefer to stick with the concept of critical path functionality, and whether or not the update violates it. I'll try and submit a proposed revision for this soon.
I quite like the overall layout of the page. But it's a lot wordier than the existing version, and reads less like a clear set of instructions on how to actually perform the proven tester function and more like an abstract discussion of concepts that requires some work and interpretation to actually put into practice. I'm not convinced that, if I were a brand new proven tester applicant, I'd actually feel particularly confident about exactly what it is I should be doing after reading that page. In technical language terms, it switches voices quite a lot. Using the passive voice tends to make sentences longer and harder to interpret; I'd want to avoid stuff like:
"Neutral karma should be left in cases of an untestable or insufficiently tested update." It just feels...squiggly, as someone reading the page for instructions on what to do. I really like the short sentences, active voice and use of 'you' that we have in the current version of the page. It makes things a lot more direct.
Honestly, I have to say, I still prefer the current version of the page over this proposed re-draft.
Do any of the current / recent proven tester applicants have a preference?
On Wed, Jul 7, 2010 at 1:44 AM, Adam Williamson awilliam@redhat.com wrote:
On Mon, 2010-07-05 at 19:33 -0500, Aaron Faanes wrote:
I went to work a bit on jdulaney's fork of the proven tester page to make the mentorship-merging stuff fit a little more smoothly, and I sort of got carried away. I'd normally just add this as a new revision for Proven_testers, but it's pretty substantially revised. It probably needs to get re-reviewed:
https://fedoraproject.org/wiki/User:Dafrito/Proven_tester
I can write up a summary of the major differences, but I figure I should only toss out one wall-of-text at a time. ;) Basically, I took the current one, and tried to expand on places that were shallow (like testing criteria) and made other sections flow a lot better (like the old Feedback section).
I feel the need to mention I refer to critpath packages as just critical packages and critical updates, instead of critical path packages and critical path updates. This is purely a stylistic change: when I was reading it out loud, I trip over the alliteration of "path packages" a lot. Let me know if this (or anything else) is a mortal sin.
I understand - I've had the same feeling - but I think we may want to stick with the 'critical path' naming, as it's what's used elsewhere in Fedora-land.
In general I am happy with the draft - however there is one area that I am a little uncertain about, and maybe others could clarify. This is the situation where for example there is currently a kernel in updates-testing for both f12 and f13, and each of these packages has a series of bug fixes. I am perfectly happy to install and test but I may not be able to test all the bugs reported as fixed. In this instance if the kernel installs without issue and the machine boots, with normal logins, normal networking for my particular wireless card on a laptop, and no issues with any tests that I conduct, should I report +ve karma if all the tests I can do pass without incident, or should I as proventester not add karma (i.e. neutral) since I can't check that all the bugs reported are not problematic.
In the current case I see that for https://admin.fedoraproject.org/updates/kernel-2.6.32.16-141.fc12?_csrf_toke... I can check 559153 but not 558002 for example. So should I add a report and have neutral karma or +ve? The instructions would indicate neutral karma is appropriate here. (Prior to becoming a proventester I would have given +ve karma in this situation.
Similarly for https://admin.fedoraproject.org/updates/kernel-2.6.33.6-147.fc13?_csrf_token... where there is an even longer list of bugs fixed. Clearly I can only test on the hardware that I am installed on for either f12 or f13 ....
Would be nice to have this clarified - I guess that reading the proventester instructions again this case should have neutral feedback. It seems that the only situation for giving positive karma as a proventester is if previously reported negative karma is demonstrated to have been wrong - please correct me if I am wrong.
Thanks Mike
On Wed, 2010-07-07 at 19:22 +0100, mike cloaked wrote:
On Wed, Jul 7, 2010 at 1:44 AM, Adam Williamson awilliam@redhat.com wrote:
On Mon, 2010-07-05 at 19:33 -0500, Aaron Faanes wrote:
I went to work a bit on jdulaney's fork of the proven tester page to make the mentorship-merging stuff fit a little more smoothly, and I sort of got carried away. I'd normally just add this as a new revision for Proven_testers, but it's pretty substantially revised. It probably needs to get re-reviewed:
https://fedoraproject.org/wiki/User:Dafrito/Proven_tester
I can write up a summary of the major differences, but I figure I should only toss out one wall-of-text at a time. ;) Basically, I took the current one, and tried to expand on places that were shallow (like testing criteria) and made other sections flow a lot better (like the old Feedback section).
I feel the need to mention I refer to critpath packages as just critical packages and critical updates, instead of critical path packages and critical path updates. This is purely a stylistic change: when I was reading it out loud, I trip over the alliteration of "path packages" a lot. Let me know if this (or anything else) is a mortal sin.
I understand - I've had the same feeling - but I think we may want to stick with the 'critical path' naming, as it's what's used elsewhere in Fedora-land.
In general I am happy with the draft - however there is one area that I am a little uncertain about, and maybe others could clarify. This is the situation where for example there is currently a kernel in updates-testing for both f12 and f13, and each of these packages has a series of bug fixes. I am perfectly happy to install and test but I may not be able to test all the bugs reported as fixed. In this instance if the kernel installs without issue and the machine boots, with normal logins, normal networking for my particular wireless card on a laptop, and no issues with any tests that I conduct, should I report +ve karma if all the tests I can do pass without incident, or should I as proventester not add karma (i.e. neutral) since I can't check that all the bugs reported are not problematic.
In the current case I see that for https://admin.fedoraproject.org/updates/kernel-2.6.32.16-141.fc12?_csrf_toke... I can check 559153 but not 558002 for example. So should I add a report and have neutral karma or +ve? The instructions would indicate neutral karma is appropriate here. (Prior to becoming a proventester I would have given +ve karma in this situation.
Similarly for https://admin.fedoraproject.org/updates/kernel-2.6.33.6-147.fc13?_csrf_token... where there is an even longer list of bugs fixed. Clearly I can only test on the hardware that I am installed on for either f12 or f13 ....
Would be nice to have this clarified - I guess that reading the proventester instructions again this case should have neutral feedback. It seems that the only situation for giving positive karma as a proventester is if previously reported negative karma is demonstrated to have been wrong - please correct me if I am wrong.
This certainly isn't the intention. In general if you use it and find it works for you - in that it doesn't break the critical path actions - you should file +ve feedback.
Actually, re-reading them, both versions tend to emphasize the negative feedback scenarios more than the positive. We should fix that. Positive feedback is in a way more important than negative, because we *need* proven testers to leave at least +1 on each critpath update (that doesn't have a problem, obviously) or it will not get published. We should probably explicitly call out that, in general, if you don't hit any of the negative or neutral feedback scenarios discussed, you should leave positive feedback.
Thinking about it, though, we could consider a slightly different process for the kernel, as it's a component that's *extremely* subject to different experiences for different users. I'm not sure the workflow we've designed will work terribly well for kernels. I suspect it'll be all too easy for a kernel which actually contains a major regression to be approved; all it needs is for a proventester who doesn't happen to own the hardware concerned to find it works fine on their system, and file a +1, and anyone else to file a +1 too, and it'd be approved, even though someone who does own the hardware might come by and test an hour later and find the problem...
we might want to design a system for the kernel where all proventesters hold off posting positive feedback for a day or two, until several proventesters and regular testers have had the chance to check for regressions.
On Wed, Jul 7, 2010 at 8:16 PM, Adam Williamson awilliam@redhat.com wrote:
Thinking about it, though, we could consider a slightly different process for the kernel, as it's a component that's *extremely* subject to different experiences for different users. I'm not sure the workflow we've designed will work terribly well for kernels. I suspect it'll be all too easy for a kernel which actually contains a major regression to be approved; all it needs is for a proventester who doesn't happen to own the hardware concerned to find it works fine on their system, and file a +1, and anyone else to file a +1 too, and it'd be approved, even though someone who does own the hardware might come by and test an hour later and find the problem...
we might want to design a system for the kernel where all proventesters hold off posting positive feedback for a day or two, until several proventesters and regular testers have had the chance to check for regressions.
That was exactly my thought too - I saw these kernel updates were there but thought that to satisfy the current criteria as best I could I would wait and see what comments that came in to bodhi over the next day or so looked like and then install and test. If I then saw no negatives, and my own tests found no problems then I felt +1 would be valid, but I wanted re-assurance from people here first. It would seem that in this situation neutral karma from a proventester would not be particularly useful as the package would not get the necessary push to stable unless a proventester gives +1. If this is acceptable as a way forward I would be happy with that but as you say for the kernel perhaps an additional paragraph in the draft would be useful.
On 07/07/2010 12:29 PM, mike cloaked wrote:
On Wed, Jul 7, 2010 at 8:16 PM, Adam Williamsonawilliam@redhat.com wrote:
Thinking about it, though, we could consider a slightly different process for the kernel, as it's a component that's *extremely* subject to different experiences for different users. I'm not sure the workflow we've designed will work terribly well for kernels. I suspect it'll be all too easy for a kernel which actually contains a major regression to be approved; all it needs is for a proventester who doesn't happen to own the hardware concerned to find it works fine on their system, and file a +1, and anyone else to file a +1 too, and it'd be approved, even though someone who does own the hardware might come by and test an hour later and find the problem...
we might want to design a system for the kernel where all proventesters hold off posting positive feedback for a day or two, until several proventesters and regular testers have had the chance to check for regressions.
That was exactly my thought too - I saw these kernel updates were there but thought that to satisfy the current criteria as best I could I would wait and see what comments that came in to bodhi over the next day or so looked like and then install and test. If I then saw no negatives, and my own tests found no problems then I felt +1 would be valid, but I wanted re-assurance from people here first. It would seem that in this situation neutral karma from a proventester would not be particularly useful as the package would not get the necessary push to stable unless a proventester gives +1. If this is acceptable as a way forward I would be happy with that but as you say for the kernel perhaps an additional paragraph in the draft would be useful.
Silly idea...how about assigning a requirement that each bugfix (or at least a majority of them) in a kernel require positive karma before the kernel is blessed? The idea is that most bugfixes are just that--fixes. Otherwise it'd be silly to release it, not so?
It's just a thought. It's the criteria I use when I release code. ---------------------------------------------------------------------- - Rick Stevens, Systems Engineer, C2 Hosting ricks@nerd.com - - AIM/Skype: therps2 ICQ: 22643734 Yahoo: origrps2 - - - - You possess a mind not merely twisted, but actually sprained. - ----------------------------------------------------------------------
On Wed, 2010-07-07 at 14:19 -0700, Rick Stevens wrote:
Silly idea...how about assigning a requirement that each bugfix (or at least a majority of them) in a kernel require positive karma before the kernel is blessed? The idea is that most bugfixes are just that--fixes. Otherwise it'd be silly to release it, not so?
It's just a thought. It's the criteria I use when I release code.
It's not silly, but it's currently impossible. =) There's no mechanism to attach a piece of feedback or a karma vote to a specific bug, programmatically. You can indicate this in a comment, but we really can't plan around that. Bodhi 2.0 should address this, with the refined karma system.
On Wed, 7 Jul 2010 20:29:27 +0100 mike cloaked mike.cloaked@gmail.com wrote:
On Wed, Jul 7, 2010 at 8:16 PM, Adam Williamson awilliam@redhat.com wrote:
Thinking about it, though, we could consider a slightly different process for the kernel, as it's a component that's *extremely* subject to different experiences for different users.
+1
test@lists.fedoraproject.org To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
I am wondering, how hard would it be to modify Bodhi to require extra Karma for kernels? Who should I talk to about this possibility? John, Son of Clay.
Subject: Re: Proven tester wiki love From: awilliam@redhat.com To: test@lists.fedoraproject.org Date: Wed, 7 Jul 2010 12:16:47 -0700
Actually, re-reading them, both versions tend to emphasize the negative feedback scenarios more than the positive. We should fix that. Positive feedback is in a way more important than negative, because we *need* proven testers to leave at least +1 on each critpath update (that doesn't have a problem, obviously) or it will not get published. We should probably explicitly call out that, in general, if you don't hit any of the negative or neutral feedback scenarios discussed, you should leave positive feedback.
Thinking about it, though, we could consider a slightly different process for the kernel, as it's a component that's *extremely* subject to different experiences for different users. I'm not sure the workflow we've designed will work terribly well for kernels. I suspect it'll be all too easy for a kernel which actually contains a major regression to be approved; all it needs is for a proventester who doesn't happen to own the hardware concerned to find it works fine on their system, and file a +1, and anyone else to file a +1 too, and it'd be approved, even though someone who does own the hardware might come by and test an hour later and find the problem...
we might want to design a system for the kernel where all proventesters hold off posting positive feedback for a day or two, until several proventesters and regular testers have had the chance to check for regressions. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net
-- test mailing list test@lists.fedoraproject.org To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
_________________________________________________________________ Hotmail is redefining busy with tools for the New Busy. Get more from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:W...
-
Aaron Faanes -
Adam Williamson -
James Laska -
John Dulaney -
mike cloaked -
Rares Aioanei -
Rick Stevens