feedback needed: modules which match logs but don't remove the entries from the parsed list

[Seth Vidal]

A log module I've been noodling with would need to match every line of
every log. Take the line, extract the hostname and collect a list  of
all the hosts that have logged something. The goal is to compare that to
a known hosts list and report any host which has reported no logs in the
last time segment.

the problem, of course, is if you match a line it gets removed and can't
be used by other modules or unparsed lines. So that's obviously not
gonna work.

Looking at the code it seems like modules should be able to hand back
None as a result which supposedly is to say "this looked like a match
but it wasn't, we don't need this, give it to unparsed".

However, testing that code seems to bear out that it, in fact, doesn't
get handed over to unparsed.

So my options are:

1. fix that so None == no match and hand them back

2. make the 'report a list of hosts which logged nothing in the last
time segment' a core feature that isn't in a module at all. 

Not sure how I feel about 2 b/c it feels like something you could safely
do in a module.

So - thoughts?
-sv

[Paul Stauffer]

On Wed, Sep 07, 2011 at 10:37:38AM -0400, seth vidal wrote:
>  A log module I've been noodling with would need to match every
> line of
>  every log. Take the line, extract the hostname and collect a list  of
>  all the hosts that have logged something. The goal is to compare that to
>  a known hosts list and report any host which has reported no logs in the
>  last time segment. 
Excellent idea.

>  1. fix that so None == no match and hand them back
>  
>  2. make the 'report a list of hosts which logged nothing in the last
>  time segment' a core feature that isn't in a module at all. 
>  
>  Not sure how I feel about 2 b/c it feels like something you could safely
>  do in a module. 
Agreed that this should be a module.

I haven't looked at the code in ages, but if the "None" response is
definitely not behaving as designed, and fixing it would solve this problem,
then I think the correct solution would be to fix it. :)

cheers,
- Paul

-- 
Paul Stauffer <paulds(a)bu.edu&gt;
Manager of Systems Administration
Computer Science Department
Boston University