cockpit - pcp interop ideas

[Frank Ch. Eigler]

Hi -

A few hours ago, some of the pcp[1] / cockpit[2] teams' folks talked about
possible interfacing scenarios.

[2] http://www.cockpit-project.org/
   [1] http://www.pcp.io/

(For those with only experience of one, fedora rawhide is a good place
to get modern versions of the other.)

There appear to be three distinct interop cases.

1) replacing some cockpitd agent-side monitoring code with pcp

See cockpit.git src/daemon/*monitor.[ch]
   and pcp.git     e.g. src/pmval/pmval.c, %man pmapi.

This could consist of a small (<100 lines) amount of PMAPI code to
   fetch statistics from a local pmcd or even (if the choice of stats
   is suitable) via non-daemon PM_CONTEXT_LOCAL, replacing cockpit
   code for polling kernel /proc files.

The RH perftools/pcp folks would be glad to quickly prototype this;
   can the cockpit folks nominate one or two of the monitors?

PCP currently lacks good cgroup-level stats summarization, so that
   part would need to be retained in cockpit, until pcp catches up.

2) using a pcp service for remote log archive access

The driving idea here is to make it possible for cockpit graphs
   (normally updated at 1 Hz) to go back in time - to browse
   historical stats.

PCP can already do central/remote logging (via pmlogger/pmmgr), and
   discovery/scanning of pmcd targets (via pmmgr), whereby pmcd's that
   start nearby are found and monitoring is automatically maintained.
   Log archives may be rotated, subsampled, etc.

One complication is the service of that log archive data back out
   on the network.  (NFS-serving the archive files probably doesn't
   count.)  We have plans for a PMAPI wire-protocol level server for
   archive data (like time-traveling pmcd), but that's some way out.

Shorter term, it is possible to use pmwebd to provide web access to
   archive files.  Its pmwebapi http/json api layer can feed data from
   archives (and/or live pmcds), but is quite low level.  Its graphite
   http/json api emulation layer [3][4] can aggregate data from many
   archives, but is quite new, inherently slower, and lacks
   authentication.  It could be a start though.

[3] http://graphite.readthedocs.org/en/1.0/url-api.html
   [4] https://web.elastic.org/~fche/blog3/archive/2014/06/16/pcp-and-graphite-back...

Even shorter term, one could decree that a canonical cockpit/pcp
   installation is one where the cockpit-ws console and
   pcp-central-logging services are colocated, and thus the archive
   files for all target hosts are immediately available.

3) teaching cockpit how to administer pcp installations

3.1) For pcp-dependent cockpit installations, the minimum would be
              # yum install pcp
        If a live pmcd is deemed necessary,
              # {chkconfig,service} pmcd {on,start}

For remote access, firewalls ports (44321/tcp) to be opened;
        optionally, pcp native acl files tweaked, and/or SASL
        authentication rules setup.

3.2) management of central pcp logger

For canonical setups, a few more # yum/chkconfig/service type
        operations will do the job, plus a few one-time config file
        modifications.  For the full range of control, long story.

- FChE