rsyslog/epylog reporting
seth vidal
skvidal at fedoraproject.org
Mon Jun 20 15:31:00 UTC 2011
On Mon, 2011-06-20 at 09:12 -0600, Kevin Fenzi wrote:
> Possibly:
>
> Staging
> *stg*
I've been thinking *Stg* should just go to /dev/null considering all
the crap they spewed this weekend. :)
> Build/Rel-eng
> x86* ppc* releng* relepel* sign* compose* koji*
> rest of everything
+1
> > - the mail/postfix module in epylog needs some more cleanup -
> > alternatively we could not do mail log parsing and let pflogsumm
> > handle it.
>
> Perhaps pflogsumm on the machines that normally process lots of emails
> and have epylog do something with the logs on machines that do not
> normally process lots of emails.
smtp*, collab*, bastion* would cover it for maillogs - but in terms of
the parser - pflogsumm shouldn't have any issue dealing with our daily
mail logs globally.
Mainly what we're looking for there is anomalies and outliers.
> > - an rsync module I wrote years ago would be useful for our epylog
> > instance
>
> Yes. Stats from download* and hosted would be nice. Would show if one
> of the machines was processing less, etc.
I need to go find that rsync module. I think I know where it is.
> I wonder if it would be worthwhile to expand on the Total messages
> weeded thing. Ie, have a number of lines weeded by each regex?
> Or would that get too big? That way we could more easily work to reduce
> the number of lines of logs that we really don't care about. ;)
I don't think that information is collected in epylog - it would take
some structural changes to store it.
> I'm not sure the "SSH scan from" lines are of use. Are we intending to
> take any action from them? Perhaps a total # so we could see if it was
> increasing/decreasing? Related: perhaps a denyhosts module?
the ssh scan is particular to the env that epylog was first developed
in. Mainly we were on the lookout for systems within a certain ip range
that were scanning. It meant they had been cracked.
> Kernel oopses might be nice to capture in their own section somehow.
> That way we can note and investigate them.
I was thinking about the oopses some. What we really need is some way of
saying:
box1: 12 oopses - same module(s) - see unparsed logs for complete pastes
box2: 4 oopses - different modules(s) - ...
not sure how much pain that will be in the modules or not. I just don't
think having a huge dump of the oops/trace in the middle of the
formatted log report is going to help much. Then again, in theory, the
oopses shouldn't happen often I would hope.
>
> Rkhunter logs to it's own log, worth adding a module for it and adding
> it in here?
afaict rkhunter doesn't send anythign to syslog or the central log host
at all. Not sure how I can get to the data from epylog.
> Would it be possible to expand to httpd logs? I know we have awstats
> (which needs work), but I was thinking of things like tracebacks from
> our wsgi applications or other error conditions, not stats related.
Not sure - mainly getting the logs into one place so epylog can parse
them will be the tricky part. the modules make some assumptions about
file paths which can be a challenge.
-sv
More information about the infrastructure
mailing list