2:02 p.m.
===========================================
#fedora-meeting: proventesters (2011-10-05)
===========================================
Meeting started by nirik at 18:00:01 UTC. The full logs are available at
http://meetbot.fedoraproject.org/fedora-meeting/2011-10-05/proventesters....
Meeting summary
---------------
* Intro/Gather more agenda (nirik, 18:00:15)
* LINK:
http://fedoraproject.org/wiki/Proven_tester#Joining_the_proven_testers
(nirik, 18:01:53)
* Recruitment (nirik, 18:05:35)
* LINK:
https://picasaweb.google.com/103451550643082159750/B#5576235128261512338
<--- I doodled up a t-shirt idea a while back (maxamillion,
18:08:01)
* IDEA: short term here: mine existing karma for non proventesters and
ask them to join, more press (blog posts, point people on test
list), and look at f-e-k improvements (nirik, 18:24:17)
* IDEA: longer term: gui for f-e-k and having pk or something like it
offer updates-testing (nirik, 18:24:26)
* IDEA: longer term: tie into abrt (nirik, 18:24:33)
* One stop page for updates testing resources (nirik, 18:25:26)
* will try and create a package updates tester page with links to
resources. (nirik, 18:29:44)
* Pending updates (nirik, 18:31:39)
* 11 critpath updates pending in f14 (nirik, 18:35:41)
* 19 critpath updates pending in f15 (nirik, 18:35:49)
* Open Floor (nirik, 18:35:56)
Meeting ended at 19:00:41 UTC.
Action Items
------------
Action Items, by person
-----------------------
* **UNASSIGNED**
* (none)
People Present (lines said)
---------------------------
* nirik (95)
* tflink (30)
* maxamillion (18)
* mike-c (12)
* Cerlyn (8)
* abadger1999 (7)
* Southern_Gentlem (4)
* satellit_ (4)
* zodbot (3)
* jwb (2)
--
18:00:01 <nirik> #startmeeting proventesters (2011-10-05)
18:00:01 <zodbot> Meeting started Wed Oct 5 18:00:01 2011 UTC. The chair is nirik.
Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:01 <nirik> #meetingname proventesters
18:00:01 <zodbot> The meeting name has been set to 'proventesters'
18:00:15 <nirik> #topic Intro/Gather more agenda
18:00:23 <nirik> any folks around for a proventesters meetup?
18:00:31 * satellit_ listening
18:00:38 <jwb> decided to show up to figure out how to become one
18:01:25 <nirik> jwb: cool. ;) it's easy.
18:01:35 * tflink is around
18:01:53 <nirik>
http://fedoraproject.org/wiki/Proven_tester#Joining_the_proven_testers
18:02:16 <nirik> basically apply, read the docs as to what you are supposed to do,
get approved
18:02:28 * maxamillion is here-ish
18:02:57 <jwb> nirik, thx
18:03:30 <nirik> I had 2 items on my plan... recruitment ideas, and one stop
shopping page for resources... did any other folks have items?
18:03:44 <tflink> nothing here
18:04:12 <maxamillion> none here
18:04:15 * nirik will wait a min more for folks.
18:04:19 <maxamillion> rgr
18:05:26 <nirik> ok, lets go ahead then...
18:05:35 <nirik> #topic Recruitment
18:05:57 <nirik> so, any ideas on how we can get more proventesters involved? and/or
more testers for updates in general?
18:06:37 <nirik> I had a few random ideas:
18:06:44 <tflink> I forget, do we have any step-by-step instructions for installing
updates-testing and giving karma?
18:06:53 <nirik> could we mine old updates for people who have given lots of karma
but are not proventesters?
18:06:55 <maxamillion> I've actually thought about this in the past ... I think
maybe getting a little funding to offer swag of some sort would be good ... like a
"Fedora QA Community Member" t-shirt or something (likely a less expensive
option to start out with)
18:06:58 * tflink will wait for nirik's ideas before giving mine
18:07:31 <nirik> well, all we have currently is
http://fedoraproject.org/wiki/Proven_tester
18:07:40 <nirik> which lists some things, but not sure in how much detail.
18:07:44 <tflink> I remember an interesting conversation about metrics and their
human effect
18:08:00 <nirik> I really think most of it is that people don't know how easy it
is to become a proventester
18:08:01 <maxamillion>
https://picasaweb.google.com/103451550643082159750/B#5576235128261512338 <--- I doodled
up a t-shirt idea a while back
18:08:09 <tflink> as soon as there is some sort of reward for doing X based on
measurement Y, people will generally start behaving differently
18:08:20 <nirik> maxamillion: that would be cool. dunno if we have funds for such a
thing tho
18:08:22 <maxamillion> tflink: well ... true
18:08:25 <maxamillion> nirik: yeah
18:08:25 <tflink> nirik: yeah, and how much they're needed
18:09:03 <nirik> ideally, everyone who provides karma on any kind of regular basis
should take the few minutes to become a proventester, IMHO.
18:09:54 <tflink> I wonder if stuff like step by step instructions would help
18:10:00 <tflink> maybe screencasts, too?
18:10:14 <tflink> the other two things that I can think of are:
18:10:25 <tflink> people worried about borking their system and losing data/time
18:10:46 <tflink> make f-e-k a little more user friendly
18:11:12 <tflink> I've thought about trying to make a gui for f-e-k before
18:11:38 <nirik> I'm not sure how active the developer is these days... but he
was around not long back
18:11:40 <tflink> to make it easier to find test cases, related packages and skip
over stuff that you don't feel right karma-izing
18:12:05 <tflink> I'd be interested in doing it but the issue is available time
18:12:13 <nirik> yeah.
18:12:57 <tflink> is it something that we really think would help?
18:13:04 <nirik> also, folks providing feedback on list could be pointed at becoming
proventesters...
18:13:38 <Cerlyn> Would a GUI avoid the "skip" confusion? Or can that be
solved at the command line?
18:13:49 <nirik> not sure. I think a bigger win might be to setup something so PK
would say "hey, there's an update of foo you have installed in updates-testing,
would you like to test it?"
18:14:01 <tflink> just leaving karma would be a start, not even becoming a proven
tester
18:14:21 <maxamillion> nirik: that's a cool idea
18:14:34 <tflink> nirik: yeah, that would be good. Give a list of testing updates
available and let the user select which ones
18:14:40 <nirik> it's been floated many times before, but has noone working on
it. ;)
18:14:41 <tflink> to install
18:14:55 <nirik> a yum plugin might be easier. ;)
18:15:55 <nirik> I don't think there's a magic bullet...
18:15:58 <Cerlyn> More precisely needed is a way to identify which packages a user
actually used and for libraries, how they got there
18:16:07 <Cerlyn> A lot of packages are installed by default I would never use
18:16:27 <nirik> Cerlyn: yeah, sometimes its a bit of a puzzle...
18:16:37 <tflink> and updated libraries that come down and I have no idea what they
do
18:16:39 <nirik> libfoo is an update. What the heck do I use libfoo for?
18:16:49 * tflink was wondering if a dependency tree + description might help
18:17:14 <tflink> appbar -> libbar -> libfoo (appbar does magical stuff)
18:17:19 <nirik> yeah, it does give summary now, but tree would be nice.
18:17:39 <nirik> not sure how hard that would be to implement.
18:17:52 <Cerlyn> It also makes no sense to tell me about a new ATI Xorg driver is
available, but I have nvidia/Intel/unknown-to-me instead. Saying video still works could
be useless.
18:17:56 <tflink> me neither, I've never even looked @ f-e-k code
18:18:29 <nirik> Cerlyn: yeah, all the x drivers are installed by default, since
they are small and you could move your install to another box and want it to work.
18:18:44 <nirik> but it means you have a number that aren't actually in use.
18:20:15 <nirik> so, I think short term here: mine existing karma for non
proventesters and ask them to join, more press (blog posts, point people on test list),
and look at f-e-k improvements?
18:21:10 <tflink> yeah
18:21:42 <nirik> longer term: gui for f-e-k and having pk or something like it offer
updates-testing
18:21:50 <nirik> another idea was tie it into abrt. ;)
18:22:09 <nirik> foo has just crashed and you reported a bug on it... but foo has a
updates-testing update, would you like to test?
18:22:28 <tflink> that's a cool idea
18:24:17 <nirik> #idea short term here: mine existing karma for non proventesters
and ask them to join, more press (blog posts, point people on test list), and look at
f-e-k improvements
18:24:26 <nirik> #idea longer term: gui for f-e-k and having pk or something like it
offer updates-testing
18:24:33 <nirik> #idea longer term: tie into abrt
18:24:40 <nirik> anything more on this? or shall we move on?
18:25:26 <nirik> #topic One stop page for updates testing resources
18:25:41 <nirik> Another idea I had was to try and create a short page with all the
resources.
18:25:47 <nirik> we have stuff all over the place now.
18:25:50 <maxamillion> that's probably a good one
18:25:58 <maxamillion> I thought we had that at one point ... or atleast a mockup of
one
18:26:00 <tflink> yeah, especially for the short term
18:26:01 <maxamillion> on the wiki somewhere
18:26:04 <nirik> bodhi, fedora-easy-karma, rss feeds, reports to mailing lists, etc
18:26:51 <nirik> would anyone be able to put in some time to make one? ;)
18:27:01 <nirik> I could help out, but not sure how much time I have to devote to
it.
18:27:48 <nirik> I can try and make something tho.. or ask on the list. ;)
18:28:10 <maxamillion> oh ... like all in wonder page for QA efforts, that's
different than what I had in mind but I like it
18:28:46 <nirik> one of the complaints has been that it's hard to see all the
stuff you need...
18:28:57 <nirik> if we can get all the links in one place it might help
18:29:18 <tflink> yeah
18:29:19 <maxamillion> yeah
18:29:40 <maxamillion> and for long term, see if we can't get some of that stuff
merged into bits of the revamp to fedora community?
18:29:44 <nirik> #info will try and create a package updates tester page with links
to resources.
18:30:03 <nirik> maxamillion: possibly. Not sure if thats a focus for it, but it
could be nice for that.
18:30:56 <tflink> lets figure out what's really being used before trying to
integrate it :)
18:31:08 <tflink> or what would be the most useful
18:31:10 <nirik> fair enough
18:31:23 <maxamillion> rgr
18:31:39 <nirik> #topic Pending updates
18:32:00 <nirik> so, did we want to try and look at the updates pending in 15/14
currently and see if we could move along the ancient ones?
18:32:06 <nirik> or is that better done out of the meeting?
18:32:53 <nirik> 11 critpath ones in f14
18:33:12 <nirik> 19 in f15
18:33:53 <nirik> some of them are ancient due to almost no one having the hardware.
18:34:00 <nirik> some are just unclear how to test.
18:35:30 <nirik> I'm hoping to have an old laptop setup to help test 14/15
soon...
18:35:41 <nirik> #info 11 critpath updates pending in f14
18:35:49 <nirik> #info 19 critpath updates pending in f15
18:35:56 <nirik> #topic Open Floor
18:36:01 <nirik> ok, anything for open floor?
18:36:15 <nirik> should we continue meeting? or does it seem to be doing any good?
18:36:37 <mike-c> nirik: I came in late - so trying to get the gist of the
discussion...
18:36:44 <satellit_> would there be a limited proventester category for testers who
mainly do installs to Hardware?
18:36:59 <nirik> mike-c: basically we are trying to increase updates-testing package
testing. ;)
18:37:07 <tflink> satellit_: what would the benefit of that be?
18:37:37 <nirik> mike-c: get more proventesters, or general testers involved so
packages don't get stuck in updates-testing.
18:37:37 <satellit_> I am not sure I have the skills to do some of the other kinds
of testing....
18:38:00 <mike-c> Any value in asking testers what main kinds of testing and which
versions of Fedora they are mainly doing?
18:38:30 <nirik> satellit_: then just don't add karma where you don't feel
you have tested enough... ;)
18:38:34 <Southern_Gentlem> satellit_, if you are running a vm to test a package
that is good as well
18:38:37 <satellit_> ok
18:38:49 <nirik> mike-c: well, not sure... we want more people particularly for the
older releases...
18:38:57 <Southern_Gentlem> i am running f14 so i do test f14 native and f15 in a
vm
18:39:49 <mike-c> For example I am very keen to test kernels, nouveau and similar,
as well as mail server and similar but also test other things where I can - mainly f16
currently with a little f15
18:40:16 <Southern_Gentlem> thats why i think a proven-testers mailing list that on
monday and thursday spam with a list that needs tested would be great
18:40:27 <nirik> in general I think we have more testers on newer releases... fedora
is so fast moving, most of the folks involved tend to want to move fast with it.
18:40:58 * nirik has a f14 and f15 vms, and a bunch of f15 machines and a few f16 machines
where I do most of my testing.
18:41:31 <mike-c> Makes sense if testers do run older versions as servers and where
stability is essential but newer versions where they want to test upcoming performance?
18:42:23 <nirik> Southern_Gentlem: would a web page with links to the current stuff
meet your needs?
18:42:34 <nirik> mike-c: yeah, I think some of that is happening.
18:43:13 <mike-c> Southern_Gentlem: I am happy with being spammed on the test list
with the list of packages that need testing with one spam shot per fedora version....
18:44:22 <nirik> Southern_Gentlem: I floated the idea of another list last week, and
the reaction was pretty negative on the test list. Do chime in there if you like...
18:45:43 <mike-c> I guess also that for most people availability to run tests varies
in time depending on other commitments... for example I do try to join the graphics test
days but can't always be free then - similarly during the normal cycle of listed
packages for test from the announcements on the list
18:46:48 <nirik> yeah, absoutely.
18:47:13 <nirik> I tend to run fedora-easy-karma once a week or so to collect all
the feedback from using the machine...
18:47:22 <nirik> or more often if there's a specific update that needs testing
18:47:26 <mike-c> Would there be any value in asking people who join proventester
what their main interest is i.e. testing packages for released N, N and N-1, N and N+1 etc
so you could have a list of what the pool of testers are likely to be doing?
18:47:31 <Southern_Gentlem> mike thats why i think another list would be good thing
18:48:12 <nirik> mike-c: well, I'm not sure how that helps us... I guess we
would know more who we have out there, but it doesn't help us grow or provide better
coverage...
18:49:38 <mike-c> I guess - it would be nice to have a balanced division of effort -
though I know that is why this discussion is happening! At the moment it is pot luck what
gets tested in any given period...
18:50:03 <Cerlyn> It probably wouldn't be too hard to push a survey to everyone
already in the group as well
18:50:13 <nirik> sure, but we can't easily say: 'hey, you, go test f14'
(well, we could, but they could say 'no, I don't care about f14')
18:51:30 <mike-c> Sure - testing is dependent on voluntary effort - and there have
been several attempts to encourage testers to do things - Adam Will occasionally makes a
specific request to the test list
18:51:56 <nirik> yeah, although those are almost always for fN-branched.
18:52:08 <nirik> occasionally a maintainer will ask for testing...
18:52:13 <mike-c> I guess that it only needs one or two people to test any one
package so provided the pool of testers is reasonable it "should" work!
18:52:35 <nirik> I guess this is related to another idea I read somewhere (can't
recall where):
18:52:58 <nirik> revive the 'qa-contact' field for packages... so people
could sign up for specific packages.
18:53:29 <nirik> pkgdb has a 'Initial qa contact' field... which I don't
think we currently use for anything.
18:54:00 <mike-c> Interesting! That may well work for some testers who have
specific interests and do test the ones they have a specific love for on a regular basis -
perhaps across several fN
18:54:07 <maxamillion> nirik: should qa-contact tag everyone in proventesters for
all critpath packages?
18:54:28 <nirik> well, I am not sure what actual effect it has. It might be
cc'ed on all bugs only.
18:54:29 <maxamillion> or just divvy it up accordingly?
18:54:34 <maxamillion> rgr
18:54:35 <nirik> abadger1999: you happen to be around?
18:54:42 <abadger1999> nirik: yep
18:54:52 <abadger1999> qa-contact is for bugzilla
18:54:57 <abadger1999> per-package
18:55:09 <abadger1999> takes a single fas username
18:55:26 <nirik> ok, so thats not any different than inital cc really?
18:55:32 <abadger1999> yeah
18:55:42 <abadger1999> Just a different field in bugzilla
18:55:52 <abadger1999> (qa contact field instead of CC field)
18:56:11 <Cerlyn> I think it has some purpose if Bugzilla is setup to use a
test-in-build style of operation
18:56:37 <mike-c> Perhaps it could be offered for testers to sign up on the bz to
"take" the "qa-contact" field?
18:56:40 <nirik> we could perhaps co-opt it some in bodhi to send those people info
about updates to that package I guess.
18:56:46 <Cerlyn> I had another idea - could Bodhi notify people who tested the last
N versions of a package and gave non-zero karma points that a new one has been released?
18:56:54 <nirik> but I think it's going to require more thought/design.
18:58:11 <Cerlyn> (The QA contact role has its own set of email preferences for
users in bugzilla)
18:58:35 <nirik> Cerlyn: we could, but I bet people would complain.
18:59:19 <nirik> ok, we are running low on time.
18:59:36 <nirik> any last minute items? do folks want to try and meet next week?
19:00:31 <nirik> lets continue on list...
19:00:38 <nirik> Thanks for coming everyone!
19:00:41 <nirik> #endmeeting