skamath reported a new issue against the project: `fedora-commops` that you are following: `` ## Problem
Right now, metrics collection in CommOps is not very efficient and requires a lot of manual work. Metrics for various events/FAS groups/users are collected using scripts which query [datagrepper](https://apps.fedoraproject.org/datagrepper/) and return results. This process is very time consuming and writing scripts each time is a very tedious process. Also, querying the datagrepper to get data everytime is redundant and time-consuming.
Example of current statistics generation : https://github.com/bee2502/fedora-stats-tools (Lots of hack-y scripts)
## Proposed Solution
Hack on [statscache](https://github.com/fedora-infra/statscache) to build a central metrics generation system for Fedora with handy features to pull statistics. Statscache consumes all the messages and does not query datagrepper every single time thereby increasing the efficiency. By building on top of statscache, we can significantly reduce the number of scripts required to gather metrics to almost 0.
Nice to have features :
* Statistics by FAS group. * Per user statistics. * Statistics of users holding a badge (Useful for event statistics) * Exporting of stats in various formats (JSON, HTML, CSV, etc) * Date based filtering for all statistics (Useful for generating reports) * Graphs for statistics
I added the above-mentioned features based on datagrepper in the [tool](https://pagure.io/gsoc-stats) I had developed last summer. This can be used as a base for statscache integration.
Sample data generated using the tool : [fedstats-data](https://github.com/sachinkamath/fedstats-data)
## Final Deliverables
* A webapp to run queries (with a nice, minimal interface - based on statscache) * API for all the queries functions * Well-Documented code.
This is my initial proposal and we can definitely build upon this. Thoughts? CC: @bee2502 @bex
``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
bee2502 added a new comment to an issue you are following: `` Is statscache developed enough to support a webapp ? Also, who will mentor this project ?
Instead, a GSoC project to build plugins for basic metrics in statscache and statscache related development along with documentation could be an alternative. Someone from Infra team and hopefully @pingou can provide more info about this. ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
skamath added a new comment to an issue you are following: ``
Is statscache developed enough to support a webapp ? Also, who will mentor this project ?
Yes. The whole point of statscache is to make metrics easier. Quoting the README : It is cool, but insufficient for some more advanced reporting and analysis that we would like to do. Due to some confusions in my last years project, I couldn't work on statscache but I have set it up, have explored the plugins and the platform quite a bit. I believe what I mentioned is quite doable. If not for the webapp. It is very possible to add API features for the features I had mentioned.
Re : Mentorship, people working on metrics should be able to mentor this project. @bt0dotninja said he could help with the metrics part of stuff. I pinged sayan about this a couple of days back but looks like he is occupied with other projects atm. @bee2502, are you planning to mentor this time as well?
Instead, a GSoC project to build plugins for basic metrics in statscache and statscache related development along with documentation could be an alternative. Someone from Infra team and hopefully @pingou can provide more info about this.
``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
bex added a new comment to an issue you are following: `` I'd like to see the proposed mentors weigh in on this. I agree that we need better statistics, and in fact was having a conversation about this yesterday. I wonder if a better place to start might be to work on recreating the work that @mattdm does for his presentation in a stable tool like this? He has some sample code and defined metrics. I would not like to see this GSoC project get bogged down defining metrics instead of writing code. ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
skamath added a new comment to an issue you are following: `` After discussing this idea with @bee2502 , we decided to split the idea based on the priority of tasks :
###High Priority
**Custom statscache plugins for metrics**
Post event metrics from badges. Metrics for FAS groups. Integrate [fedora-stats-tool](https://github.com/mattdm/fedora-stats-tools) into statscache? Ability to generate weekly/monthly/quarterly/yearly reports. Ability to export statistics in different formats. Well-documented code.
### Medium Priority
A nice web interface for generating the statistics (using Flask/Bottle? ) Automatic statistics generation and storage. Deploy the final deliverable on infracloud.
@bee2502 Please add to this if I missed something :) ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
linuxmodder added a new comment to an issue you are following: `` While I'm not labeled as a statscache mentor I'd be willing to help with this project as well seeing as this year we have an abundance of python talent in the mentor pool. @skamath @bex ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
sayanchowdhury added a new comment to an issue you are following: ``
After discussing this idea with bee2502 , we decided to split the idea based on the priority of tasks : High Priority Custom statscache plugins for metrics
Which all plugins?
Post event metrics from badges. Metrics for FAS groups. Integrate fedora-stats-tool into statscache? Ability to generate weekly/monthly/quarterly/yearly reports. Ability to export statistics in different formats. Well-documented code. Medium Priority A nice web interface for generating the statistics (using Flask/Bottle? )
An interface for displaying the stats in statscache already exists. Is the plan to build project specific dashboard here on top of statscache?
Automatic statistics generation and storage.
Can you elaborate this?
Deploy the final deliverable on infracloud.
Why deploy it to infracloud rather production?
The plan seems too abstract to me. Can write a detail write of the implementation on what/how are you planning? ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
skamath added a new comment to an issue you are following: ``
After discussing this idea with bee2502 , we decided to split the idea based on the priority of tasks : High Priority Custom statscache plugins for metrics
Which all plugins?
As mentioned earlier, some of the plugins required would be : * Stats by FAS group : We need to be able to generate statistics of a FAS group as a whole. This can be used to generate reports of a SIG/group easily when necessary.
* Stats by badge : This can be useful while generating event reports. This plugin needs to find all the recipients of, say badge X and should show activity of the users, given the time delta.
* FAS Account trends : This plugin will be really useful to track the new comer trends to Fedora. This can also help CommOps/Join track newcomer retention rates. Please take a look at Slide 6 of the presentation [1] by @bee2502 for more information on this.
Also, we should be integrating the stats required by @mattdm, as mentioned by you on the statscache ticket[2]
Post event metrics from badges. Metrics for FAS groups. Integrate fedora-stats-tool into statscache? Ability to generate weekly/monthly/quarterly/yearly reports. Ability to export statistics in different formats. Well-documented code. Medium Priority A nice web interface for generating the statistics (using Flask/Bottle? )
An interface for displaying the stats in statscache already exists. Is the plan to build project specific dashboard here on top of statscache?
Yes, but IMO the interface is really basic and needs some work. We can either build a dashboard on top of the statscache or revamp statscache to include interactive features such as filtering by date, ability to generate graphs, etc.
Automatic statistics generation and storage.
Can you elaborate this?
Sorry for not elaborating on that one. By that, I meant the ability to export the metrics in different formats for using it elsewhere.
Deploy the final deliverable on infracloud.
Why deploy it to infracloud rather production?
If the project is ready for production this summer, we can definitely go ahead and deploy it to production. If not, we can host it on commops Infracloud for testing it.
The plan seems too abstract to me. Can write a detail write of the implementation on what/how are you planning?
From what I see, the following should should be the action plan : * Figure out how statscache codebase can be expanded to accomodate the new requirements. * Tweak/Add modules to statscache * Revamp the statscache interface. * Write the required plugins required for statistics. * Add interactivity to the interface. ( This can either be a direct around statscache or a separate dashboard for the project build on top of statscache).
@sayanchowdhury Your thoughts on this? :) ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
z2s8 added a new comment to an issue you are following: `` I'd like to take on the high priority task as a GSoC student, and I was thinking of getting started doing a part of the task to get to know the codebase better.
Do you think starting with creating the 'Stats by badge' plugin would be a good idea?
I sent my introduction mail here [0], I've setup statscache yet, and I'm excited to work on this!
[0] https://lists.fedoraproject.org/archives/list/summer-coding@lists.fedoraproj... ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
bee2502 added a new comment to an issue you are following: ``
Here are some metrics suggestions for Atomic and Cockpit as suggested by @jberkus and me :
* Knowing about contribution activity ( Issues posting and responses, Pull Requests, wiki edits) We'll also want to span the systems used by the various projects, so Pagure for Fedora Atomic, GitHub for Cockpit, etc.
* Statistics about contributors activity tenure participation in FADs/vFADS participation in IRC meetings if possible, Trello activity wiki edits, but this needs more thought - small/large contributions and areas.
We also want to collect stats on Cockpit as a a project. ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
bee2502 added a new comment to an issue you are following: `` One of the possible metrics : #42 Activity (mailing list or otherwise) by day of week and time of day in each contributor's local timezone ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
bee2502 added a new comment to an issue you are following: `` @skamath would love your help on this ticket to finalize the list of metrics (since we have had numerous suggestions recently). ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
The issue: `GSoC Idea - Centralized metrics for Fedora` of project: `fedora-commops` has been assigned to `skamath` by skamath.
mattdm added a new comment to an issue you are following: `` So, maybe this will help? This is scripts I used to generate contributor stats as presented at Flock and DevConf:
https://github.com/mattdm/fedora-stats-tools/tree/develop/contributor-trends
They're pretty ugly, and in particular hit the server harder than they should, and they cache nothing and stupidly have to start over at the beginning just to add data to the end. ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
mattdm added a new comment to an issue you are following: `` In general, big things I'm interested in are the onboarding and retention numbers that Bee has been looking at. Things like:
* Number of active contributors (as segmented by area as possible) over time * Number of new contributors each month (or other period of time, but month is nice) * Percent of work done by relatively new contributors * Percent of work done by "old hands" * Retention of new contributors * and retention of long-time contributors, too
These are basically relevant and interesting as indicators of project health.
It would also be interesting to map onboarding growth (going from mailing list posts to irc participation to pagure commits, say).
Another interesting thing is categorization of work patterns. Do some people tend to do a ton of work all at once, and then go dormant? Do some people do some small thing every week, adding up to a lot? This stuff isn't project health per se, but still is interesting to know and we might do something with it. ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-commops/issue/105
commops@lists.fedoraproject.org
-
Bhagyashree Padalkar -
Brian Exelbierd (bex)
-
Corey W Sheldon
-
Matthew Miller
-
Sachin S Kamath
-
Sayan Chowdhury
-
Zsombor Szántó