What is the future of logwatch?

[Yaakov Nemoy]

Hey List,

In the org that i work for, we use logwatch for log monitoring. Since
puppet is too new to have a module in logwatch, i've had the 'joy'
recently of attempting to write a functional module. In doing so, i
have a number of issues i would like to bring up about logwatch in
Fedora (and RHEL/CentOS).

To begin with, like a good netizen, i sent a message to the mailing
list upstream. So far i've heard no reply. There's the chance that it
got caught in my spam filter, but i've gotten one message from the ML
already. It feels like upstream is dead. There has also been no
release since 2007. Since it still 'just works' i won't bring out the
fire and brimstone and demand it be removed, but i would like to know,
how are we going to support this?

This question is important in the context of the issues i had
programming for logwatch. A well programmed tool maintains orthogonal
features in an orthogonal way. I ran into some serious bugs while
trying to develop a robust module. Logwatch has two modes, one for a
machine with a single set of logs, and one for generating reports for
a machine with multiple logs, namely your centralised logging box. The
snippets of code that filter the logs you want to analyse allow you to
control for the date range and the machine the logged event was
generated on. When trying to code for the puppet logs, my code
ultimately only worked on the single nodes, but crapped out on our
logging box. After debugging the issue, it's obvious that it's doing
some prefiltering per host that is occluding all the puppet logs, when
the 'splithosts' feature is enabled. Extending and maintaining this
package is going to be difficult without a willing maintainer. If this
is just something that needs a bug report, i would appreciate if the
package maintainer, according to zodbot Karel Klíč, could speak up on
this.

If no one is willing to maintain logwatch, i would like to ask why is
it enabled by default? For most environments, logwatch is
ignored/disabled in lieu of other solutions. People who use logwatch
seriously are already aware of its presence and will enable it when
disabled. They are also generaly experts, and from my experience been
around the sysadmin block quite a few times. For the non-expert user,
discussions of target audience aside, either they know about logwatch,
and perhaps keep an eye on it, or will never find it. I refer to the
fact that the only way to know about logwatch on a running system is
that innocous "you've got mail in /var/spool/mail/root' message. Then
you have to know to open up your mbox, which in my case means
installing mutt which is not installed by default, and read it, and
then know it's logwatch, and realize what it's there for. There are
'expert tips' i've seen that remind newbs to check logwatch on their
own machines, but if this is something important, it should be more
obvious, and if it's not so important, why is it enabled by default?

I'm not going to bike shed, so i'm not going to ask for any particular
action. We don't have the time in our organisation to take up
maintenance of logwatch, so we clearly are going to look for another
solution. Still, i would like to ask the persons who made the decision
to enable this by default, is it still relevant? Or is it something
that's in danger of becoming cruft? Am i missing a certain perspective
on why it's enabled by default?

-Yaakov

DISCLAIMER: I will not entertain bike shedding on this thread. It
sounds so simple that everyone could imagine having one's own opinion
on it. If you waste everyone's time with a 'me too', or anything that
tries to convert a mailing list discussion into a survey, i will be
unhappy. And you know don't want to find out what that means.

For every message sent to a ML, it requires a person 10 seconds
approximately to read and/or ignore it. Considering the number of
people subscribed here, you will waste a considerable number of man
hours with a useless reply, so before you hit send, take 10 seconds to
decide if you want to reply. Please.

[Karel Klic]

Hi Yaakov,

please see below.

On 03/16/2010 10:31 PM, Yaakov Nemoy wrote:
>  Hey List,
>
>  In the org that i work for, we use logwatch for log monitoring. Since
>  puppet is too new to have a module in logwatch, i've had the 'joy'
>  recently of attempting to write a functional module. In doing so, i
>  have a number of issues i would like to bring up about logwatch in
>  Fedora (and RHEL/CentOS).
>
>  To begin with, like a good netizen, i sent a message to the mailing
>  list upstream. So far i've heard no reply. There's the chance that it
>  got caught in my spam filter, but i've gotten one message from the ML
>  already. It feels like upstream is dead. There has also been no
>  release since 2007. Since it still 'just works' i won't bring out the
>  fire and brimstone and demand it be removed, but i would like to know,
>  how are we going to support this? 
Yes, the upstream is almost dead. Nevertheless I
believe if someone 
(anyone) starts to be active on their mailing list, it will make things 
move again in a good direction. If it does not, it makes sense to do a 
fork, because a _lot_ of people use the package, send patches, and 
improve logwatch in a long run.

I have had this in my todo list since 5 months or so, even promising it 
to a colleague of mine that time. Unfortunately there are still things 
with higher priority to be done. If you can help and devote some time to 
logwatch, then we should cooperate.

>  This question is important in the context of the issues i had
>  programming for logwatch. A well programmed tool maintains orthogonal
>  features in an orthogonal way. I ran into some serious bugs while
>  trying to develop a robust module. Logwatch has two modes, one for a
>  machine with a single set of logs, and one for generating reports for
>  a machine with multiple logs, namely your centralised logging box. The
>  snippets of code that filter the logs you want to analyse allow you to
>  control for the date range and the machine the logged event was
>  generated on. When trying to code for the puppet logs, my code
>  ultimately only worked on the single nodes, but crapped out on our
>  logging box. After debugging the issue, it's obvious that it's doing
>  some prefiltering per host that is occluding all the puppet logs, when
>  the 'splithosts' feature is enabled. Extending and maintaining this
>  package is going to be difficult without a willing maintainer. If this
>  is just something that needs a bug report, i would appreciate if the
>  package maintainer, according to zodbot Karel Klíč, could speak up on
>  this. 
Extending and maintaining a package are two different things.

I believe that discussing some programming issues and source code design 
is out of scope for Fedora maintainer role. It would be too much work to 
do it for every package.

>  If no one is willing to maintain logwatch, i would like to ask why
> is
>  it enabled by default? For most environments, logwatch is
>  ignored/disabled in lieu of other solutions.People who use logwatch
>  seriously are already aware of its presence and will enable it when
>  disabled. They are also generaly experts, and from my experience been
>  around the sysadmin block quite a few times. For the non-expert user,
>  discussions of target audience aside, either they know about logwatch,
>  and perhaps keep an eye on it, or will never find it. I refer to the
>  fact that the only way to know about logwatch on a running system is
>  that innocous "you've got mail in /var/spool/mail/root' message. Then
>  you have to know to open up your mbox, which in my case means
>  installing mutt which is not installed by default, and read it, and
>  then know it's logwatch, and realize what it's there for. There are
>  'expert tips' i've seen that remind newbs to check logwatch on their
>  own machines, but if this is something important, it should be more
>  obvious, and if it's not so important, why is it enabled by default? 
Is it
really enabled by default on Fedora? It should not be enabled in a 
desktop spin, and Fedora currently has no server spin.

If logwatch is installed and enabled on some spin one day, this should 
be documented in the deployment guide, so inexperienced users can read 
about the presence of logwatch, and how to use it. I am willing to write 
something if others feel it's necessary.

>  I'm not going to bike shed, so i'm not going to ask for any
> particular
>  action. We don't have the time in our organisation to take up
>  maintenance of logwatch, so we clearly are going to look for another
>  solution. Still, i would like to ask the persons who made the decision
>  to enable this by default, is it still relevant? Or is it something
>  that's in danger of becoming cruft? Am i missing a certain perspective
>  on why it's enabled by default?
>
>  -Yaakov 
Best regards,
Karel

[Frank Murphy]

On 17/03/10 17:22, Karel Klic wrote:

--snipped--

>  Is it "logwatch" really enabled by default on Fedora?
Currently you have to "yum install logwatch" with @live CD's install
not sure about the DVD.

--snipped--

-- 
Regards,

Frank Murphy
UTF_8 Encoded, Fedora 12, 13, Rawhide: x86_64

[Paul Howarth]

On Wed, 17 Mar 2010 17:50:58 +0000
Frank Murphy <frankly3d(a)gmail.com&gt; wrote:

>  On 17/03/10 17:22, Karel Klic wrote:
>  
>  --snipped--
>  
>  > Is it "logwatch" really enabled by default on Fedora?
>  
>  Currently you have to "yum install logwatch" with @live CD's install
>  not sure about the DVD. 
The default install from the DVD installed it prior to F-12 but not in
F-12.

Paul.

[Simo Sorce]

On Wed, 17 Mar 2010 19:02:26 +0000
Paul Howarth <paul(a)city-fan.org&gt; wrote:

>  On Wed, 17 Mar 2010 17:50:58 +0000
>  Frank Murphy <frankly3d(a)gmail.com&gt; wrote:
>  
>  > On 17/03/10 17:22, Karel Klic wrote:
>  > 
>  > --snipped--
>  > 
>  > > Is it "logwatch" really enabled by default on Fedora?
>  > 
>  > Currently you have to "yum install logwatch" with @live CD's install
>  > not sure about the DVD.
>  
>  The default install from the DVD installed it prior to F-12 but not in
>  F-12. 
Indeed, I gad to go and search for it explicitly when I realized I
stopped seeing the reports in my inbox after the upgrade ...

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

[Yaakov Nemoy]

2010/3/17 Paul Howarth <paul(a)city-fan.org&gt;:
>  On Wed, 17 Mar 2010 17:50:58 +0000
>  Frank Murphy <frankly3d(a)gmail.com&gt; wrote:
>
> > On 17/03/10 17:22, Karel Klic wrote:
> >
> > --snipped--
> >
> > > Is it "logwatch" really enabled by default on Fedora?
> >
> > Currently you have to "yum install logwatch" with @live CD's install
> > not sure about the DVD.
>
>  The default install from the DVD installed it prior to F-12 but not in
>  F-12. 
Ah great. I was basing my complaints off older experience and daily
work with RHEL/CentOS machines too. If we have moved away from by
default, then that answers alot of questions.

-Yaakov

[Yaakov Nemoy]

2010/3/17 Karel Klic <kklic(a)redhat.com&gt;:
>  Hi Yaakov,
>
>  please see below.
>
>  On 03/16/2010 10:31 PM, Yaakov Nemoy wrote:
> > Hey List,
> >
> > In the org that i work for, we use logwatch for log monitoring. Since
> > puppet is too new to have a module in logwatch, i've had the 'joy'
> > recently of attempting to write a functional module. In doing so, i
> > have a number of issues i would like to bring up about logwatch in
> > Fedora (and RHEL/CentOS).
> >
> > To begin with, like a good netizen, i sent a message to the mailing
> > list upstream. So far i've heard no reply. There's the chance that it
> > got caught in my spam filter, but i've gotten one message from the ML
> > already. It feels like upstream is dead. There has also been no
> > release since 2007. Since it still 'just works' i won't bring out the
> > fire and brimstone and demand it be removed, but i would like to know,
> > how are we going to support this?
>  Yes, the upstream is almost dead. Nevertheless I believe if someone
>  (anyone) starts to be active on their mailing list, it will make things
>  move again in a good direction. If it does not, it makes sense to do a
>  fork, because a _lot_ of people use the package, send patches, and
>  improve logwatch in a long run. 
I would like to think so, but it takes more than just a single email
asking them to accept something upstream. Let alone the issues i had
getting it to work, and it still isn't perfect. If the community does
perk up, this would be great, but i'm not going to base any decision
myself on that assumption.

>  I have had this in my todo list since 5 months or so, even promising
> it
>  to a colleague of mine that time. Unfortunately there are still things
>  with higher priority to be done. If you can help and devote some time to
>  logwatch, then we should cooperate. 
I don't have the time at all myself. I'd rather work with a solution
that works today for our needs.

> > This question is important in the context of the issues i had
> > programming for logwatch. A well programmed tool maintains orthogonal
> > features in an orthogonal way. I ran into some serious bugs while
> > trying to develop a robust module. Logwatch has two modes, one for a
> > machine with a single set of logs, and one for generating reports for
> > a machine with multiple logs, namely your centralised logging box. The
> > snippets of code that filter the logs you want to analyse allow you to
> > control for the date range and the machine the logged event was
> > generated on. When trying to code for the puppet logs, my code
> > ultimately only worked on the single nodes, but crapped out on our
> > logging box. After debugging the issue, it's obvious that it's doing
> > some prefiltering per host that is occluding all the puppet logs, when
> > the 'splithosts' feature is enabled. Extending and maintaining this
> > package is going to be difficult without a willing maintainer. If this
> > is just something that needs a bug report, i would appreciate if the
> > package maintainer, according to zodbot Karel Klíč, could speak up on
> > this.
>  Extending and maintaining a package are two different things.
>
>  I believe that discussing some programming issues and source code design
>  is out of scope for Fedora maintainer role. It would be too much work to
>  do it for every package. 
As long as it currently works, it has every right to be in fedora.
Still, i see it as part of our mission to help shape how people use
open source, so concentrating on the big picture is definitely in
scope. If this means discussing potential programming issues users may
have, and the relevance it has to the distro and our big picture
focus, then it's also within scope. Since programming for it is
fundamentally challenging, i think people are going to move to other
solutions or already have, and that's relevant to discuss. This is
namely my question.

-Yaakov