recording more metrics and doing more data mining about Fedora

[Jeff Spaleta]

On Wed, Jun 17, 2009 at 7:43 PM, Ian Weller<ian at ianweller.org> wrote:
> The idea that's currently in my head is to create a repository of a
> bunch of Python scripts that return a specific type of data -- perhaps a
> count of something over time, etc., that can be called via a specific
> method and told to generate data, which would then get sucked into
> whatever visualization backend we find.

My understanding is EKG has been ported to python... you should look
into that as the basis for mailinglist data mining.

I've been sitting on a python based apache access log parser that Ive
been meaning to push into fedorahosted git as "EEG"
It was meant as a pythonic extensible replacement for the map making
so we can eventually do crap like hook it into the turbogears based
infrastructure.
I can punt that over for you and let you run with it and see if you
can build it up further.

But honestly TG is way over my head at the monent. What I do my heavy
analysis lifting with is matplotlib, but that might not be most
appropriate for scripted Fedora data mining long term...not for
visualization at least.

You might want to ping luke macken and see how he creates the graphs
for bodhi's web interface and what the plans are for data
visualization widgets for Fedora Community.  If he can spin up  simple
line,bar, and pie graphs widgets that we can plug data into that
should cover most trending needs.  I'd like a better solution for maps
than matplotlib that integrates with the TG infrastructure but I don't
know what that would be.  I'm willing to help implement Fedora
Community widget sets once someone more experienced than myself points
me to reasonable candidate visualization toolkits....i know enough
about matplotlib to know its too heavy of a hammer for this.

Beyond that, we should definitely look at using code swarm and make
videos of different types of contribution. Like a code swarm of wiki
edits or mailinglist activity and other such contribution where we can
hang a username off the activity.

-jef