10:17 a.m.
Hey everyone! I happened by chance to meet someone at Red Hat who works on
Red Hat's Net Promoter Score program. NPS, if you're not familiar, is a
metric based on the simple question "How likely are you to recommend this?"
on a scale from 0-10.
I think this can be very useful, especially as tracked over time. And it
turns out that the group at Red Hat who does this would be happy to provide
this to Fedora as a service.
I'd like to make a link to this survey available on some of the various
Fedora websites. (Get Fedora, Fedora Magazine, Ask Fedora...)
We'll ask that basic question and a couple of followups like "Do you
participate in the Fedora community" and "What are your top challenges?"
(This is different from the demographic / contributor survey and won't ask
any of those sorts of questions.)
For, like, a hotel chain, you usually get this survey as an email to some
customer list. While we have email addresses of Fedora contributors, for
obvious reasons we aren't gonna be using those for this in bulk. I also
don't feel good about using the announce list — I will probably
include a link to the survey in future release announcements, but just
sending mail about the survey itself doesn't feel like what people signed up
to that list for.
I am thinking, though, of creating a "please survey me" mailing list that
people can opt-in to — if you sign up, we'll email you once every three
months or six months or so.
What do you think of this idea overall? We also have the option of creating
more specific surveys focused on an edition or spin, if your team would be
interested in such a thing.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
1:04 p.m.
+1, the thing, the release notes, and the survey list. I like surveys.
P
On Tue, Nov 12, 2019 at 5:18 PM Matthew Miller <mattdm(a)fedoraproject.org> wrote:
Hey everyone! I happened by chance to meet someone at Red Hat who works on
Red Hat's Net Promoter Score program. NPS, if you're not familiar, is a
metric based on the simple question "How likely are you to recommend this?"
on a scale from 0-10.
I think this can be very useful, especially as tracked over time. And it
turns out that the group at Red Hat who does this would be happy to provide
this to Fedora as a service.
I'd like to make a link to this survey available on some of the various
Fedora websites. (Get Fedora, Fedora Magazine, Ask Fedora...)
We'll ask that basic question and a couple of followups like "Do you
participate in the Fedora community" and "What are your top challenges?"
(This is different from the demographic / contributor survey and won't ask
any of those sorts of questions.)
For, like, a hotel chain, you usually get this survey as an email to some
customer list. While we have email addresses of Fedora contributors, for
obvious reasons we aren't gonna be using those for this in bulk. I also
don't feel good about using the announce list — I will probably
include a link to the survey in future release announcements, but just
sending mail about the survey itself doesn't feel like what people signed up
to that list for.
I am thinking, though, of creating a "please survey me" mailing list that
people can opt-in to — if you sign up, we'll email you once every three
months or six months or so.
What do you think of this idea overall? We also have the option of creating
more specific surveys focused on an edition or spin, if your team would be
interested in such a thing.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
_______________________________________________
council-discuss mailing list -- council-discuss(a)lists.fedoraproject.org
To unsubscribe send an email to council-discuss-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/council-discuss@lists.fedor...
1:11 p.m.
On Tue, Nov 12, 2019, 14:06 Petr Šabata <psabata(a)redhat.com> wrote:
+1, the thing, the release notes, and the survey list. I like
surveys.
https://youtu.be/Bwi96ZRtc_8
+1 to the general idea. We'll want to give some thought about how to put it
in front of people such that we get meaningful answers. Which I suppose
means we should think about what it is that we're asking. Essentially NPS
comes down to either "yay Fedora" or "boo Fedora", but where we put
the
survey will depend on what we're specifically trying to find out.
1:45 p.m.
On Tue, Nov 12, 2019 at 02:11:42PM -0500, Ben Cotton wrote:
+1 to the general idea. We'll want to give some thought about how
to put it
in front of people such that we get meaningful answers. Which I suppose
means we should think about what it is that we're asking. Essentially NPS
comes down to either "yay Fedora" or "boo Fedora", but where we put
the
survey will depend on what we're specifically trying to find out.
We will also be able to categorize the responses by where they came from, so
we can look at, for example, different response from Ask Fedora vs. the
opt-in mailing list.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
5:15 a.m.
Le 2019-11-12 17:17, Matthew Miller a écrit :
I think this can be very useful, especially as tracked over time. And
it
turns out that the group at Red Hat who does this would be happy to
provide
this to Fedora as a service.
Hello Matthew,
I'm not a huge fan of surveys as I doubt the value we can get from it.
Is our survey need so big we need a complex tool requiring to depend on
an external team?
In terms of tooling, wouldn't our vote tooling do the job here? We'll
have control over the data and opt in.
In terms of governance, it's the responsibility of Commops to deliver
such insight of our community.
https://docs.fedoraproject.org/en-US/commops/
Use metrics and data to improve understanding of Fedora community
> Support development of metrics tooling
Can't we do this inside our community?
In addition, how will this team validate the statistical validity of the
survey without using data collected for other means? Which will also
leads us to GDPR issues.
Jean-Baptiste
7:37 a.m.
On Wed, Nov 13, 2019 at 11:15 AM Jean-Baptiste Holcroft
<jean-baptiste(a)holcroft.fr> wrote:
Le 2019-11-12 17:17, Matthew Miller a écrit :
> I think this can be very useful, especially as tracked over time. And
> it
> turns out that the group at Red Hat who does this would be happy to
> provide
> this to Fedora as a service.
Hello Matthew,
I'm not a huge fan of surveys as I doubt the value we can get from it.
Is our survey need so big we need a complex tool requiring to depend on
an external team?
In terms of tooling, wouldn't our vote tooling do the job here? We'll
have control over the data and opt in.
I would actually prefer a third party to do this, the community as a
whole has too many tools and related things to manage and third
parties have good process for dealing with things like GDPR. I feel if
it's deemed that this is necessary I think a specialist tool is the
better way to get the value rather than putting the load onto already
overloaded teams.
> In terms of governance, it's the responsibility of Commops to deliver
> such insight of our community.
> https://docs.fedoraproject.org/en-US/commops/
> > Use metrics and data to improve understanding of Fedora community
> >> Support development of metrics tooling
>
> Can't we do this inside our community?
>
> In addition, how will this team validate the statistical validity of the
> survey without using data collected for other means? Which will also
> leads us to GDPR issues.
>
> Jean-Baptiste
> _______________________________________________
> council-discuss mailing list -- council-discuss(a)lists.fedoraproject.org
> To unsubscribe send an email to council-discuss-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/council-discuss@lists.fedor...
5:51 a.m.
Matthew Miller <mattdm(a)fedoraproject.org> wrote:
...
Hey everyone! I happened by chance to meet someone at Red Hat who
works on
Red Hat's Net Promoter Score program. NPS, if you're not familiar, is a
metric based on the simple question "How likely are you to recommend this?"
on a scale from 0-10.
I think this can be very useful, especially as tracked over time.
How do you think that it might be useful? This seems like the most
important question to answer: if we know what we want to do with the
data, that will enable us to effectively design the research exercise.
Allan
10:46 a.m.
On Wed, Nov 13, 2019 at 11:51:45AM +0000, Allan Day wrote:
> I think this can be very useful, especially as tracked over
time.
How do you think that it might be useful? This seems like the most
important question to answer: if we know what we want to do with the
data, that will enable us to effectively design the research exercise.
Net Promoter Score is a gauge of loyalty and engagement. Having that as a
running metric lets us know in a broad sense whether our outreach and
mindshare efforts are working.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
8:40 a.m.
Matthew Miller <mattdm(a)fedoraproject.org> wrote:
...
Net Promoter Score is a gauge of loyalty and engagement. Having that
as a
running metric lets us know in a broad sense whether our outreach and
mindshare efforts are working.
Personally I'd have expected NPS to be more strongly influenced by
someone's experience of using Fedora, rather than outreach and
mindshare efforts. If someone recommends a product, isn't it usually
because they've used it and liked it?
If measuring outreach and mindshare is the goal, wouldn't more direct
measures be more effective, like social media metrics, website data,
press coverage and user numbers? Better still would be research
targeted at new users, to find out what drew them to Fedora.
Allan
1:43 p.m.
On Thu, Nov 14, 2019 at 02:40:23PM +0000, Allan Day wrote:
> Net Promoter Score is a gauge of loyalty and engagement. Having
that as a
> running metric lets us know in a broad sense whether our outreach and
> mindshare efforts are working.
Personally I'd have expected NPS to be more strongly influenced by
someone's experience of using Fedora, rather than outreach and
mindshare efforts. If someone recommends a product, isn't it usually
because they've used it and liked it?
That's an excellent question and the answer is: I don't know. But in our
current draft of the survey, the question _after_ the NPS one asks why the
respondant chose that value.
If measuring outreach and mindshare is the goal, wouldn't more
direct
measures be more effective, like social media metrics, website data,
press coverage and user numbers? Better still would be research
targeted at new users, to find out what drew them to Fedora.
Those are all valuable too. We can definitely also use this platform to run
specific targeted surveys, maybe even with a link at the end of GNOME
initial setup or something like that.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
11:32 a.m.
Matthew Miller <mattdm(a)fedoraproject.org> wrote:
...
> > Net Promoter Score is a gauge of loyalty and engagement.
Having that as a
> > running metric lets us know in a broad sense whether our outreach and
> > mindshare efforts are working.
>
> Personally I'd have expected NPS to be more strongly influenced by
> someone's experience of using Fedora, rather than outreach and
> mindshare efforts. If someone recommends a product, isn't it usually
> because they've used it and liked it?
That's an excellent question and the answer is: I don't know. But in our
current draft of the survey, the question _after_ the NPS one asks why the
respondant chose that value.
I'd personally be careful about asking "why": I think the common
research view is that you don't ask respondents to supply the analysis
themselves (peoples' self-understandings are often wrong!)
From a research design perspective, it would likely be better to ask a
range of more specific questions, rather than having an open question
which you then have to try and interpret afterwards. This would make
it simpler to analyse results, and we could be more confident in how
the respondents understand the questions.
Other things I wonder about NPS:
- Will it help us understand why activities are successful or
unsuccessful, and how to improve them?
- How long will it take to see trends over time, and by the time we
see them, will it be too late to change the outcome?
...
We can definitely also use this platform to run
specific targeted surveys, maybe even with a link at the end of GNOME
initial setup or something like that.
Yep, I'm definitely interested in that!
Allan
10:36 a.m.
Hi,
Fedora is lucky to have great metrics and data about our community;
however, as others mentioned, we have not used it as well as we could.
A good goal for community metrics to get direct feedback from the Fedora
contributor community. This was the goal of the demographic /
contributor survey planning in 2017 because the Diversity & Inclusion
Team recognized the importance of creating more bidirectional feedback
loops between the Fedora community and Fedora leadership.
On 11/12/19 11:17 AM, Matthew Miller wrote:
I am thinking, though, of creating a "please survey me"
mailing list that
people can opt-in to — if you sign up, we'll email you once every three
months or six months or so.
Note there is sampling bias (convenience sampling) with this idea. Those
who opt in to this list and participate are people who are especially
motivated to subscribe and share feedback with Fedora leadership. This
will get some feedback, but you will likely get feedback from a specific
subset (perhaps an already vocal subset) of the community.
If directly emailing is off the table (which personally does not
irritate me as a community contributor), then I think we could be more
creative about where to get people's attention. For example, Elections
are an activity typically done by an active, dedicated group of
contributors. Embedding a link to the survey in the Elections site might
solicit feedback from more types of contributors than an opt-in list.
--
Cheers,
Justin W. Flory
justinwflory.com
TZ=America/New_York
Pronouns: he/him/his
9:05 a.m.
On Mon, Dec 02, 2019 at 11:36:09AM -0500, Justin W. Flory wrote:
A good goal for community metrics to get direct feedback from the
Fedora
contributor community. This was the goal of the demographic /
contributor survey planning in 2017 because the Diversity & Inclusion
Team recognized the importance of creating more bidirectional feedback
loops between the Fedora community and Fedora leadership.
Yeah, and I still would love that. But I also understand the difficulties it
presents. This new approach sidesteps two things: first, we're avoiding
demographic info for now, and second, since it is Red Hat's NPS team running
the survey and managing the data, compliance and GDPR and etc. are handled
for us.
> people can opt-in to — if you sign up, we'll email you once
every three
> months or six months or so.
Note there is sampling bias (convenience sampling) with this idea. Those
who opt in to this list and participate are people who are especially
motivated to subscribe and share feedback with Fedora leadership. This
will get some feedback, but you will likely get feedback from a specific
subset (perhaps an already vocal subset) of the community.
Oh, absolutely. We will have the ability to see where people came to the
survey from, and can take this into account.
If directly emailing is off the table (which personally does not
irritate me as a community contributor), then I think we could be more
creative about where to get people's attention. For example, Elections
are an activity typically done by an active, dedicated group of
contributors. Embedding a link to the survey in the Elections site might
solicit feedback from more types of contributors than an opt-in list.
I agree -- that's a great idea. Thanks!
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
1:42 p.m.
The idea was something like this?
https://ubuntu.com/blog/the-ubuntu-20-04-lts-pre-release-survey
9:07 a.m.
On Thu, Dec 19, 2019 at 07:42:53PM -0000, Alessio Ciregia wrote:
The idea was something like this?
https://ubuntu.com/blog/the-ubuntu-20-04-lts-pre-release-survey
We could do some surveys like that, although we plan to start with something
much more general.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
865
days inactive
920
days old
council-discuss@lists.fedoraproject.org
14 comments
8 participants
participants (8)
-
Alessio Ciregia -
Allan Day -
Ben Cotton -
Jean-Baptiste Holcroft -
Justin W. Flory -
Matthew Miller -
Peter Robinson -
Petr Šabata