Log management

[Ray Van Dolson]

On Tue, Jan 12, 2010 at 08:29:51AM -0600, Mike McGrath wrote:
> Personally I'd like to get general metrics from the logs and list errors /
> warnings that we would care about.  The problem is we never really know
> the format of some errors we get.  We had recently gotten some memory
> errors from fedorahosted and no one noticed it until we happened to log in
> and see it.
> 
> I think I like the idea of a single nightly report that is easy to read
> through.  The trick is figuring out what should be in that report I guess.
> 
> What are others using for log analysis?
> 
> 	-Mike

Splunk :)

But in the non-commercial realm, there's a lot of stuff listed here[1].
In pre-splunk days, we were using swatch[2] quite heavily.  It's not
pretty to configure, but did its job.  Wouldn't be surprised if there
are some Python-ish tools out there that do the same.

We used it in tandem with syslog-ng (which we still use) and a FIFO.

Ray

[1] http://www.loganalysis.org/log-parsers-generic/
[2] http://www.oit.ucsb.edu/~eta/swatch/swatch.html