Blocking Spam

[Todd Simi]

Hi,

I'm getting spammed to death.  I'm running sendmail and ClamAV.

Is there a way of filtering mail by keyword?  I seem to get 
a few particular emails from different sources, but they are
all the same messages.

Thanks
Todd

[Terry Polzin]

On Monday December 25 2006 19:48, Todd Simi wrote:
> Hi,
>
> I'm getting spammed to death.  I'm running sendmail and ClamAV.
>
> Is there a way of filtering mail by keyword?  I seem to get
> a few particular emails from different sources, but they are
> all the same messages.

Install spamassassin it will help.  Another thing to do would be remove or 
obscure your e-mail address from any web pages so bots can't scan for it.

[Res]

On Mon, 25 Dec 2006, Todd Simi wrote:
> Hi,
>
> I'm getting spammed to death.  I'm running sendmail and ClamAV.
>
> Is there a way of filtering mail by keyword?  I seem to get
> a few particular emails from different sources, but they are
> all the same messages.

www.mailscanner.info
> Thanks
> Todd

[Jeffrey Ross]

Todd Simi wrote:
> Hi,
>
> I'm getting spammed to death.  I'm running sendmail and ClamAV.
>
> Is there a way of filtering mail by keyword?  I seem to get 
> a few particular emails from different sources, but they are
> all the same messages.
>
> Thanks
> Todd

Spamassassin is the way to go, but make sure you use sa-update to update 
the rules on a daily basis.  In addition you may want to consider the  
"FuzzyOCR" plugin which will (help?) stop the annoying image spam (I 
haven't seen one of those in months).

Be forewarned,  spamassassin can be a memory pig, I start it with the 
following command:

/usr/bin/spamd -c -m 20 -d -r /var/run/spamd.pid

Right now there are three spamd processes running each using roughly 
80Meg of memory for a total of ~240Meg.  If I start getting a lot of 
incoming email the number of processes will climb and so will the memory 
usage.

Jeff

[Mike Wohlgemuth]

Todd Simi wrote:
> Hi,
>
> I'm getting spammed to death.  I'm running sendmail and ClamAV.

By far the most effective anti-spam measure I've ever used is 
greylisting.  I don't use sendmail, but milter-greylist is in extras.  
That, with spamassassin with nightly runs of sa-update is about the best 
combination I've been able to find.

Mike

[jdow]

From: "Jeffrey Ross"
> Todd Simi wrote:
> > Hi,
> >
> > I'm getting spammed to death.  I'm running sendmail and ClamAV.
> >
> > Is there a way of filtering mail by keyword?  I seem to get 
> > a few particular emails from different sources, but they are
> > all the same messages.
> >
> > Thanks
> > Todd
>
> Spamassassin is the way to go, but make sure you use sa-update to update 
> the rules on a daily basis.  In addition you may want to consider the  
> "FuzzyOCR" plugin which will (help?) stop the annoying image spam (I 
> haven't seen one of those in months).
>
> Be forewarned,  spamassassin can be a memory pig, I start it with the 
> following command:
>
> /usr/bin/spamd -c -m 20 -d -r /var/run/spamd.pid
>
> Right now there are three spamd processes running each using roughly 
> 80Meg of memory for a total of ~240Meg.  If I start getting a lot of 
> incoming email the number of processes will climb and so will the memory 
> usage.

Also make sure you run up a suitable collection of SARE rules for
SpamAssassin. http://www.rulesemporium.com/. Read the descriptions of
the various sets of rules and pick what is appropriate for your setup.
(The FuzzyOCR plugin can be found through the SpamAssassin wiki if you
go to the plugins page. It is in development. 3.41 is supposed to be
the current one recommended for users rather than testers. (I REALLY
need to update from the version I am using. But the pressure's low.
Even the old version I am using does excellently.))

It reads like Todd is running a personal mail service. So tools like
greylisting may and may not help him mostly modulo his patience with
slightly delayed email messages and whether he is using fetchmail or
a real smtp installation.

{^_^}   Joanne

[Tim]

On Mon, 2006-12-25 at 20:14 -0500, Jeffrey Ross wrote:
> Spamassassin is the way to go, but make sure you use sa-update to
> update the rules on a daily basis.

I hadn't really played much with spam assassin.  Never been overly
impressed by it, nor many other alternatives, nor keen on mail programs
which have "this is junk" buttons without any description of what that
will actually do (play silly games with to/from addresses, analyse it
some other way, etc.).  Though have turned on the junk filtering options
on Evolution over the last few weeks, been marking junk as junk, though
haven't seen it automatically detect ANY junk, at all.

So, not having heard of using sa-update before, I decided to see what it
can do.  It does this:

Can't locate Archive/Tar.pm in @INC (@INC contains: /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.7/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.4/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.3/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.7 /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5 /usr/lib/perl5/site_perl/5.8.4 /usr/lib/perl5/site_perl/5.8.3 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.4/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.3/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.7 /usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/vendor_perl/5.8.4 /usr/lib/perl5/vendor_perl/5.8.3 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi /usr/lib/perl5/5.8.8) at /usr/bin/sa-update line 94.
BEGIN failed--compilation aborted at /usr/bin/sa-update line 94.

When all was installed over the last few weeks/months, there have been
no complaints about missing dependencies, and no forced installations.
I remain unimpressed.

[Terry Polzin]

On Monday December 25 2006 20:58, Tim wrote:
> On Mon, 2006-12-25 at 20:14 -0500, Jeffrey Ross wrote:
> > Spamassassin is the way to go, but make sure you use sa-update to
> > update the rules on a daily basis.
>
> I hadn't really played much with spam assassin.  Never been overly
> impressed by it, nor many other alternatives, nor keen on mail programs
> which have "this is junk" buttons without any description of what that
> will actually do (play silly games with to/from addresses, analyse it
> some other way, etc.).  Though have turned on the junk filtering options
> on Evolution over the last few weeks, been marking junk as junk, though
> haven't seen it automatically detect ANY junk, at all.
>
> So, not having heard of using sa-update before, I decided to see what it
> can do.  It does this:
>
> Can't locate Archive/Tar.pm in @INC (@INC contains:
> /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi
> /usr/lib/perl5/vendor_perl/5.8.8
> /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.7/i386-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.4/i386-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.3/i386-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.7
> /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5
> /usr/lib/perl5/site_perl/5.8.4 /usr/lib/perl5/site_perl/5.8.3
> /usr/lib/perl5/site_perl
> /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi
> /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi
> /usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi
> /usr/lib/perl5/vendor_perl/5.8.4/i386-linux-thread-multi
> /usr/lib/perl5/vendor_perl/5.8.3/i386-linux-thread-multi
> /usr/lib/perl5/vendor_ perl/5.8.7 /usr/lib/perl5/vendor_perl/5.8.6
> /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/vendor_perl/5.8.4
> /usr/lib/perl5/vendor_perl/5.8.3 /usr/lib/perl5/vendor_perl
> /usr/lib/perl5/5.8.8/i386-linux-thread-multi /usr/lib/perl5/5.8.8) at
> /usr/bin/sa-update line 94. BEGIN failed--compilation aborted at
> /usr/bin/sa-update line 94.

I just had the same issue with sa-update;
    
    yum install perl-Archive-Tar
will fix that.  In kmail you can configure that the " this is junk button" 
exactly does.  It works great for me.

[Jeffrey Ross]

Mike Wohlgemuth wrote:
> Todd Simi wrote:
> > Hi,
> >
> > I'm getting spammed to death.  I'm running sendmail and ClamAV.
>
> By far the most effective anti-spam measure I've ever used is 
> greylisting.  I don't use sendmail, but milter-greylist is in extras.  
> That, with spamassassin with nightly runs of sa-update is about the 
> best combination I've been able to find.
>
> Mike

I tried greylisting in combination with spamassassin and it really 
didn't buy me much, many of the spammers are starting to get intelligent 
and will retry the message in an attempt to get around the greylisting.

Spamassassin by far has worked very very well for me, and yes in 
addition to the sa-updates I have a handful of sare rules that get 
updated daily.

I'm down to 1 to 2 spam emails a day from a 50+ a day about 2 years ago 
before spamassassin.

Jeff

[Todd Zullinger]

Tim wrote:
> So, not having heard of using sa-update before, I decided to see
> what it can do.  It does this:
>
> Can't locate Archive/Tar.pm in @INC

[...]
> BEGIN failed--compilation aborted at /usr/bin/sa-update line 94.

This is a packaging bug (#193100).  It was fixed in FC6 back in June
but for whatever reason the fix wasn't made to the FC5 packages.   I
just added a comment to bugzilla to see if this can get fixed for FC5
since it is still supported.

Anyway, the fix for this one is simple, install perl-Archive-Tar.
> I remain unimpressed.

I've been running spamassassin on my desktop and servers for a few
years now and it catches a boatload of spam with a pretty out of the
box setup.  I can't imagine trying to deal with my mail if I didn't
have it.

[Tim]

Tim:
> > So, not having heard of using sa-update before, I decided to see what it
> > can do.  It does this:
> >
> > Can't locate Archive/Tar.pm in @INC (@INC contains:
> > /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi
> > /usr/lib/perl5/vendor_perl/5.8.8   ... and so on ...

Terry Polzin:
> I just had the same issue with sa-update;
>     yum install perl-Archive-Tar
>
> will fix that.

Thanks for that, one wonders why it wasn't pulled in with
install/updates for spam assassin, in the past, automatically.  Also
which sadist decided the the archive name, unneccessarily, needed a few
capital letters.  So I've installed it, and we'll see if it makes any
difference to how things work.
> In kmail you can configure that the "this is junk button" exactly
> does.  It works great for me.

Unfortunately, I don't use that program.  I'm not a KDE fan, on the
whole (including kmail, the last time I tried it).  I am using
Evolution, which so far, is the least annoying GUI client on Linux that
I've tried, though it's documentation leaves an awful lot to be desired.

Yes, documentation authors, I can work out what some "check incoming
messages for junk" tick box enables and disables, you don't need to
redescribe *that* in the manual.  What I want to know is in what manner
the thing does its job...  I want to know things like, does the junk
mail folder automatically clear out old mail, can I manually remove
things from it, and so on...  It's scant on useful information.

[jdow]

From: "Tim"
> On Mon, 2006-12-25 at 20:14 -0500, Jeffrey Ross wrote:
> > Spamassassin is the way to go, but make sure you use sa-update to
> > update the rules on a daily basis.
>
> I hadn't really played much with spam assassin.  Never been overly
> impressed by it, nor many other alternatives, nor keen on mail programs
> which have "this is junk" buttons without any description of what that
> will actually do (play silly games with to/from addresses, analyse it
> some other way, etc.).  Though have turned on the junk filtering options
> on Evolution over the last few weeks, been marking junk as junk, though
> haven't seen it automatically detect ANY junk, at all.
>
> So, not having heard of using sa-update before, I decided to see what it
> can do.  It does this:
>
> Can't locate Archive/Tar.pm in @INC (@INC

It needs Archive::Tar?

How are you installing it? (I jettison the Fedora RPM and use cpan. That
gives you a "canonical" install. Before removing the RPM save the file
/etc/init.d/spamassassin. It is useful when it comes time to make spamd
run.)

I have a dummy spamassassin RPM that installs nothing other than the
knowledge that a dummy spamassassin is present to satisfy the YUM monster.

{^_^}

[Tim]

On Mon, 2006-12-25 at 18:39 -0800, jdow wrote:
> It needs Archive::Tar?

Apparently so...  ["It" being part of spam assassin.]
> How are you installing it?

It was the default installation that came with FC5, plus any updates
that might have happened in the meantime.

On other boxes I had decided not to bother installing spam assassin, and
until recently didn't use it.  Evolution is painfully slow when it does
*ANY* filtering, and five spams a day versus nearly 200 non-spams a day,
meant that manual deletion isn't much of a chore.  And there's something
rather satisfying about saying, "Die you spam, die!," as you hit delete.
I just decided to try it out, to see how it does its tricks.  And to see
if it's going to be a practical solution for a friend of mine who seems
to have the opposite spam:ham ratio.
> (I jettison the Fedora RPM and use cpan. That gives you a "canonical"
> install. Before removing the RPM save the
> file /etc/init.d/spamassassin. It is useful when it comes time to make
> spamd run.)

Hmm, I've not had the best of luck with Perl, in the past.  I can
remember all the messing around I had to do to get the WDG HTML
validator installed, long ago (this needs that, ad infinitum, and *YOU*,
poor sod, had to manage it all by yourself).
> I have a dummy spamassassin RPM that installs nothing other than the
> knowledge that a dummy spamassassin is present to satisfy the YUM
> monster.

Hmm, I had thought of that approach with one or two other stupidly
*REQUIRED* but unused RPMs in Fedora.

[jdow]

From: "Tim"
> On Mon, 2006-12-25 at 18:39 -0800, jdow wrote:
> > It needs Archive::Tar?
>
> Apparently so...  ["It" being part of spam assassin.]
> > How are you installing it?
>
> It was the default installation that came with FC5, plus any updates
> that might have happened in the meantime.
>
> On other boxes I had decided not to bother installing spam assassin, and
> until recently didn't use it.  Evolution is painfully slow when it does
> *ANY* filtering, and five spams a day versus nearly 200 non-spams a day,
> meant that manual deletion isn't much of a chore.  And there's something
> rather satisfying about saying, "Die you spam, die!," as you hit delete.
> I just decided to try it out, to see how it does its tricks.  And to see
> if it's going to be a practical solution for a friend of mine who seems
> to have the opposite spam:ham ratio.

NEVER EVER use SpamAssain as a filter for a mail reading agent.
NEVER EVER use SpamAssain as a filter for a mail reading agent.
NEVER EVER use SpamAssain as a filter for a mail reading agent.

I have told you three times therefore it must be true.

That said read up on fetchmail. It can deliver direct to your machine.
Then evolution can read directly from the mail directory or, if you
start dovecot or something like that, it can read from any machine on
your network. (That is how Loren and I handle our email. It is collected
on one machine and read on another.)

Fetchmail does the filtering via procmail here. This line in .fetchmailrc
does it. (We each run our own fetchmail with individual accounts we
fetch from.)

defaults mda "/usr/bin/procmail -d XXX"
(Replace XXX with the username. It is probably not necessary but "it works";
and I've not tried to fix it.

Procmail calls spamassassin with these lines in ".procmailrc".

SHELL=/bin/sh
DROPPRIVS=yes

:0
* < 500000
{
   :0 fw: spamassassin.lock
   | /usr/bin/spamc -t 150 -u XXX
}

(XXX == username again. Perhaps it's overkill. It works. I'm not fixing it.)

The timeout parameter, -t 150, is about right here with a 1GHz Athlonish
type machine and a lot of network tests enabled.

In the init.d file you can tune the number of children spawned by spamd.
(It's the same algorithm as Apache httpd.) You can also tune how many
times a child gets used before it gets killed and a new one is created.

I note my .procmailrc file is much fancier than the above. I have a lot
of simple "anti-pest" stuff in it such as trolls blacklisted with
extreme prejudice to /dev/null. Bug me too many times, like the UOL.com.br
challenge response "thing" and /dev/null happens. {^_-}

{^_^}

[jdow]

From: "jdow"
> NEVER EVER use SpamAssain as a filter for a mail reading agent.
> NEVER EVER use SpamAssain as a filter for a mail reading agent.
> NEVER EVER use SpamAssain as a filter for a mail reading agent.
>
> I have told you three times therefore it must be true.
>
> That said read up on fetchmail. It can deliver direct to your machine.
> Then evolution can read directly from the mail directory or, if you
> start dovecot or something like that, it can read from any machine on
> your network. (That is how Loren and I handle our email. It is collected
> on one machine and read on another.)
>
> Fetchmail does the filtering via procmail here. This line in .fetchmailrc
> does it. (We each run our own fetchmail with individual accounts we
> fetch from.)
>
> defaults mda "/usr/bin/procmail -d XXX"
> (Replace XXX with the username. It is probably not necessary but "it 
> works";
> and I've not tried to fix it.
>
> Procmail calls spamassassin with these lines in ".procmailrc".
>
> SHELL=/bin/sh
> DROPPRIVS=yes
>
> :0
> < 500000
>
> {
>   :0 fw: spamassassin.lock
>   | /usr/bin/spamc -t 150 -u XXX
> }
>
> (XXX == username again. Perhaps it's overkill. It works. I'm not fixing 
> it.)
>
> The timeout parameter, -t 150, is about right here with a 1GHz Athlonish
> type machine and a lot of network tests enabled.
>
> In the init.d file you can tune the number of children spawned by spamd.
> (It's the same algorithm as Apache httpd.) You can also tune how many
> times a child gets used before it gets killed and a new one is created.
>
> I note my .procmailrc file is much fancier than the above. I have a lot
> of simple "anti-pest" stuff in it such as trolls blacklisted with
> extreme prejudice to /dev/null. Bug me too many times, like the UOL.com.br
> challenge response "thing" and /dev/null happens. {^_-}

One other thing - NEVER EVER use spamassassin or spamc as root. That is
a very bad thing.

{^_^}

[Todd Zullinger]

jdow wrote:
> It needs Archive::Tar?

Yep.  See the comments in sa-update around line 89:

# These are the non-standard required modules
# Use the evals to avoid the annoying RPM requirement check
eval { use Net::DNS; };
eval { use LWP::UserAgent; };
eval { use HTTP::Date qw(time2str); };
eval { use Archive::Tar 1.23; };
eval { use IO::Zlib 1.04; };

I'm not sure precisely what annoying RPM requirement checks they're
trying to avoid, but it worked to prevent rpm from picking up the
dependencies automatically at package creation time.
> How are you installing it? (I jettison the Fedora RPM and use cpan.
> That gives you a "canonical" install. Before removing the RPM save
> the file /etc/init.d/spamassassin. It is useful when it comes time
> to make spamd run.)

I've had good times with the Fedora packages, but I don't stress SA
too much I suppose.  I am avidly opposed to letting CPAN (or
configure; make; make install) drop stuff into my system with only a
hope that it will clean it up.  It's amazing how many badly written
install/uninstall routines you find when trolling through people's
source code. :)

Using rpm packages is much more reliable and accountable for me,
generally.  Of course, it was busted in this case, but it's similarly
busted for a tarball install as well.  It's just that doing the manual
install you may be more likely to catch the warning:

NOTE: the optional Archive::Tar (version 1.23) module is not
    installed.

The "sa-update" script requires this module to access tar update
      archive files.

[Tim]

On Mon, 2006-12-25 at 19:24 -0800, jdow wrote:
> NEVER EVER use SpamAssain as a filter for a mail reading agent.
> NEVER EVER use SpamAssain as a filter for a mail reading agent.
> NEVER EVER use SpamAssain as a filter for a mail reading agent.
>
> I have told you three times therefore it must be true.

;-)  Wouldn't be my choice, either.  Though various clients have that
integration feature that *ought* to do something useful, but...
> That said read up on fetchmail. It can deliver direct to your machine.

I already do.  I use fetchmail to collect from various remote POP and
IMAP server accounts.  It drops it in my mail spool (/var/mail/tim).  I
use dovecot as an IMAP server on my LAN so that I can check mail from
any terminal on the premises.

So, for me, it'd be relatively easy (I presume, looking at the examples
you provided) to jury rig spam assassin somewhere into there.  But it
hasn't seemed worth the effort for my typical daily 5:200 spam:ham
ratio.  Even more so, if I don't want to worry about accidental false
positives.

My friend, on the other hand, has ham vastly outnumbered by spam, but
would like to run his system simply:  Hitting send/receive on the mail
client gets the mail, there and then.  No delay as fetchmail, etc., goes
through a background run every few minutes.  No clicking on anything
else to spurn it into instant action.  I don't see an easy solution for
that, other than hoping the mail client could do the trick.

We do have a remote mail service that uses spam assassin, but we don't
like its configuration, nor is it easy to remotely configure, and it
becomes a DOS in itself (its bayes and auto-whitelist files fill up all
the userspace for the account before the spam does).  Ugh!

Changing host isn't an option, not at least for several months.

I'd love to learn how to use a chainsaw by practicing on a spammer's
computer.

[James Wilkinson]

Jeffrey Ross wrote:
> Spamassassin is the way to go, but make sure you use sa-update to
> update the rules on a daily basis. In addition you may want to
> consider the "FuzzyOCR" plugin which will (help?) stop the annoying
> image spam (I haven't seen one of those in months).

Also, I'd strongly recommend training SA's Bayesian analysis, using the
sa-learn program. SpamAssassin won't use Bayesian analysis until it has
learnt 200 good ("ham") e-mails and 200 spams.

Bayesian analysis continues to be a *very* good way of analysing e-mail,
in my experience.

Hope this helps,

James.

[James Wilkinson]

jdow wrote:
> How are you installing it? (I jettison the Fedora RPM and use cpan. That
> gives you a "canonical" install. Before removing the RPM save the file
> /etc/init.d/spamassassin. It is useful when it comes time to make spamd
> run.)
>
> I have a dummy spamassassin RPM that installs nothing other than the
> knowledge that a dummy spamassassin is present to satisfy the YUM monster.

That's "canonical" as in "direct from http://spamassassin.apache.org/%22?

The download page says
  RPM: SpamAssassin RPMs can be built directly from the tar file.

* Simply run:

rpmbuild -tb Mail-SpamAssassin-3.1.7.tar.gz

If necessary use the following option if you grab the bzip2 tar
      file instead of the gzip version:

--define "srcext .bz2"

You end up with two RPMs you need to install, instead of Fedora's one,
but you end up with canonical SpamAssassin managed by RPM.

Hope this helps,

James.

[Tim]

On Tue, 2006-12-26 at 09:08 +0000, James Wilkinson wrote:
> Also, I'd strongly recommend training SA's Bayesian analysis, using
> the sa-learn program. SpamAssassin won't use Bayesian analysis until
> it has learnt 200 good ("ham") e-mails and 200 spams.

Isn't that supposed to be the point of the junk/not-junk buttons on mail
clients?
> Bayesian analysis continues to be a *very* good way of analysing
> e-mail, in my experience.

Back when I was still using Windows, I used to use the in-built one that
came with The Bat! mail client.  It seemed to do a reasonable job, and
it was damn quick (unlike the speed of any kind of mail filtering in
Evolution).  Though, before that, I'd knocked most spam off, without any
false positives, with about 12 mail rules.

[Tim]

Tim:
> > So, not having heard of using sa-update before, I decided to see
> > what it can do.  It does this:
> >
> > Can't locate Archive/Tar.pm in @INC
>
> [...]
> > BEGIN failed--compilation aborted at /usr/bin/sa-update line 94.

Todd Zullinger:
> This is a packaging bug (#193100).  It was fixed in FC6 back in June
> but for whatever reason the fix wasn't made to the FC5 packages.   I
> just added a comment to bugzilla to see if this can get fixed for FC5
> since it is still supported.
>
> Anyway, the fix for this one is simple, install perl-Archive-Tar.

Hmm, well that fixes the installation on FC5.  FC6 has a different
error:

Can't locate LWP/UserAgent.pm in @INC (@INC
contains: /usr/lib/perl5/vendor_perl /5.8.8/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/s ite_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.7/i386-linux -thread-multi /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi /usr/lib/pe rl5/site_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/ lib/perl5/site_perl/5.8.7 /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_per l/5.8.5 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thr ead-multi /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl 5/vendor_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.7 /us r/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/ve ndor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi /usr/lib/perl5/5.8.8) at /usr/bin/sa-update line 92.
BEGIN failed--compilation aborted at /usr/bin/sa-update line 92.

How do you work out what package it wants to fulfill its needs?

[Charles Curley]

On Wed, Dec 27, 2006 at 12:46:46AM +1030, Tim wrote:
> Tim:
> > Anyway, the fix for this one is simple, install perl-Archive-Tar.
>
> Hmm, well that fixes the installation on FC5.  FC6 has a different
> error:
>
> Can't locate LWP/UserAgent.pm in @INC (@INC
> contains: /usr/lib/perl5/vendor_perl /5.8.8/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/s ite_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.7/i386-linux -thread-multi /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi /usr/lib/pe rl5/site_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/ lib/perl5/site_perl/5.8.7 /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_per l/5.8.5 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thr ead-multi /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl 5/vendor_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.7 /us r/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/ve ndor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi /usr/lib/perl5/5.8.8) at /usr/bin/sa-update line 92.
> BEGIN failed--compilation aborted at /usr/bin/sa-update line 92.
>
> How do you work out what package it wants to fulfill its needs?

It says, "Can't locate LWP/UserAgent.pm in @INC..." So it's looking
for a perl module called "LWP::UserAgent" somewhere in the list of
directories that constitute the array @INC (short for "include"). A
quick look at line 92 of sa-update shows that it says:

eval { use LWP::UserAgent; };

And the comment just above indicates that the reason the author used
the eval trick was to bypass RPM dependency checking.

Next question: do you really need it? I don't have it installed, and
sa-update runs and gets updates without complaining. But I don't see
how sa-update can get updates without it.

Another question is, how do you get it? A bit of checking with yum
indicates that it isn't available from core, extras or livna. So you
need cpan2rpm so you can pull it in from cpan.org and RPM-ize it for
installation.

[Dotan Cohen]

On 26/12/06, Tim  wrote:
> On Tue, 2006-12-26 at 09:08 +0000, James Wilkinson wrote:
> > Also, I'd strongly recommend training SA's Bayesian analysis, using
> > the sa-learn program. SpamAssassin won't use Bayesian analysis until
> > it has learnt 200 good ("ham") e-mails and 200 spams.
>
> Isn't that supposed to be the point of the junk/not-junk buttons on mail
> clients?
> > Bayesian analysis continues to be a *very* good way of analysing
> > e-mail, in my experience.
>
> Back when I was still using Windows, I used to use the in-built one that
> came with The Bat! mail client.  It seemed to do a reasonable job, and
> it was damn quick (unlike the speed of any kind of mail filtering in
> Evolution).  Though, before that, I'd knocked most spam off, without any
> false positives, with about 12 mail rules.

Kmail has a very decent bayesian filter called bogomail. I had to
teach it for about two weeks before it started doing any filtering
itself, with about 70 spams a day to 20 or so hams. At first it
started filtering out only the most obvious spam, but now (about two
months after first install) it only lets 2-3 spams through a day, with
no false positives so far.

But Bogofilter is my second line of defence. On the POP3 server I've
got spammassasin running, it traps about 300 spams a day, with two or
three false positives that I've discovered in the past few months.

So my 400:20 spam:ham ratio gets down to 70:20 at the server, and
further reduced to 3:20 at the Inbox. I can live with that.

Dotan Cohen

http://lyricslist.com/lyrics/artist_albums/265/imx.php
http://technology-sleuth.com/short_answer/what_is_hdtv.html

[James Wilkinson]

I wrote:
> Also, I'd strongly recommend training SA's Bayesian analysis, using
> the sa-learn program. SpamAssassin won't use Bayesian analysis until
> it has learnt 200 good ("ham") e-mails and 200 spams.

Tim wrote:
> Isn't that supposed to be the point of the junk/not-junk buttons on mail
> clients?

Usually, that trains the mail clients *own* spam filter.

Unlike many other things, it's not really ideal to have multiple
different spam filters. This is because if one thinks an e-mail is a bit
iffy, but probably OK, it will let it through. But if multiple separate
spam tests all think that an e-mail is dodgy, one can reject it with a
lot more confidence.

SpamAssassin is designed to incorporate different styles of checks --
bayesian, DNSBL checks on the sending mail server, and *lots* of fixed
rules -- and come up with one overall spam score. The more checks that
SpamAssassin can do reliably (which in the case of Bayesian analysis,
means training SA's own Bayesian engine), the more accurately it can
spot spam and let through good e-mail.

Hope this helps,

James.

[jdow]

From: "James Wilkinson"
> jdow wrote:
> > How are you installing it? (I jettison the Fedora RPM and use cpan. That
> > gives you a "canonical" install. Before removing the RPM save the file
> > /etc/init.d/spamassassin. It is useful when it comes time to make spamd
> > run.)
> >
> > I have a dummy spamassassin RPM that installs nothing other than the
> > knowledge that a dummy spamassassin is present to satisfy the YUM 
> > monster.
>
> That's "canonical" as in "direct from http://spamassassin.apache.org/%22?

Effectively.
> The download page says
>  RPM: SpamAssassin RPMs can be built directly from the tar file.
>
> Simply run:
>
> rpmbuild -tb Mail-SpamAssassin-3.1.7.tar.gz
>
> If necessary use the following option if you grab the bzip2 tar
> file instead of the gzip version:
>
> --define "srcext .bz2"
>
> You end up with two RPMs you need to install, instead of Fedora's one,
> but you end up with canonical SpamAssassin managed by RPM.

The missing files that are in that second RPM are the main reason I went
to CPAN installs. It also means I can add the perl features that can make
SpamAssassin somewhat stronger.

RPM management seems to leave you too much at the mercy of the Fedora
Core folks who seem to be allergic to including features from time to time.

{^_^}

[jdow]

From: "Tim"
> On Tue, 2006-12-26 at 09:08 +0000, James Wilkinson wrote:
> > Also, I'd strongly recommend training SA's Bayesian analysis, using
> > the sa-learn program. SpamAssassin won't use Bayesian analysis until
> > it has learnt 200 good ("ham") e-mails and 200 spams.
>
> Isn't that supposed to be the point of the junk/not-junk buttons on mail
> clients?

In general those are put there by spammers who are hopping you'll push
the wrong button. {O,o}

Actually what it does and how it works with YOUR usage probably depend
on your "global" versus "per user" Bayes files, scores, and rules.
> > Bayesian analysis continues to be a *very* good way of analysing
> > e-mail, in my experience.
>
> Back when I was still using Windows, I used to use the in-built one that
> came with The Bat! mail client.  It seemed to do a reasonable job, and
> it was damn quick (unlike the speed of any kind of mail filtering in
> Evolution).  Though, before that, I'd knocked most spam off, without any
> false positives, with about 12 mail rules.

Quick - maybe. But Bayesian analysis without rules is like a bicycle with
only one pedal.

{^_^}

[jdow]

From: "Dotan Cohen"
> On 26/12/06, Tim  wrote:
> > On Tue, 2006-12-26 at 09:08 +0000, James Wilkinson wrote:
> > > Also, I'd strongly recommend training SA's Bayesian analysis, using
> > > the sa-learn program. SpamAssassin won't use Bayesian analysis until
> > > it has learnt 200 good ("ham") e-mails and 200 spams.
> >
> > Isn't that supposed to be the point of the junk/not-junk buttons on mail
> > clients?
> > > Bayesian analysis continues to be a *very* good way of analysing
> > > e-mail, in my experience.
> >
> > Back when I was still using Windows, I used to use the in-built one that
> > came with The Bat! mail client.  It seemed to do a reasonable job, and
> > it was damn quick (unlike the speed of any kind of mail filtering in
> > Evolution).  Though, before that, I'd knocked most spam off, without any
> > false positives, with about 12 mail rules.
>
> Kmail has a very decent bayesian filter called bogomail. I had to
> teach it for about two weeks before it started doing any filtering
> itself, with about 70 spams a day to 20 or so hams. At first it
> started filtering out only the most obvious spam, but now (about two
> months after first install) it only lets 2-3 spams through a day, with
> no false positives so far.
>
> But Bogofilter is my second line of defence. On the POP3 server I've
> got spammassasin running, it traps about 300 spams a day, with two or
> three false positives that I've discovered in the past few months.
>
> So my 400:20 spam:ham ratio gets down to 70:20 at the server, and
> further reduced to 3:20 at the Inbox. I can live with that.

Bicycle with one pedal, indeed.

500 to 1000 emails a day to my accounts.
150 to 300 of them are spams.
Typical false markups are 0 to 3 a week.

(I run a fairly agressive set of SARE rule sets in addition to the DNS tests
and default SpamAssassin rules.)

{^_^}

[jdow]

From: "James Wilkinson"
> I wrote:
> > Also, I'd strongly recommend training SA's Bayesian analysis, using
> > the sa-learn program. SpamAssassin won't use Bayesian analysis until
> > it has learnt 200 good ("ham") e-mails and 200 spams.
>
> Tim wrote:
> > Isn't that supposed to be the point of the junk/not-junk buttons on mail
> > clients?
>
> Usually, that trains the mail clients *own* spam filter.
>
> Unlike many other things, it's not really ideal to have multiple
> different spam filters. This is because if one thinks an e-mail is a bit
> iffy, but probably OK, it will let it through. But if multiple separate
> spam tests all think that an e-mail is dodgy, one can reject it with a
> lot more confidence.
>
> SpamAssassin is designed to incorporate different styles of checks --
> bayesian, DNSBL checks on the sending mail server, and *lots* of fixed
> rules -- and come up with one overall spam score. The more checks that
> SpamAssassin can do reliably (which in the case of Bayesian analysis,
> means training SA's own Bayesian engine), the more accurately it can
> spot spam and let through good e-mail.

Scores. The magic is in scores. No single rule (usually) should be
allowed to define spam. (BAYES_99 is good enough here I score it
high enough to guarantee markup as spam. Then I rely on the small
number of negative scoring rules to save random ham messages that
might get all the way to 0.99 BAYES spam probability.)

Besides, WTF good is Bayes with image spam?
{^_^}

[Carroll Grigsby]

On Tuesday 26 December 2006 17:09, jdow wrote:
> > > whack
> Scores. The magic is in scores. No single rule (usually) should be
> allowed to define spam. (BAYES_99 is good enough here I score it
> high enough to guarantee markup as spam. Then I rely on the small
> number of negative scoring rules to save random ham messages that
> might get all the way to 0.99 BAYES spam probability.)
>
> Besides, WTF good is Bayes with image spam?
> {^_^}

Answering a question with a question: When was the last time that anyone 
received an email with a GIF image that was _not_ spam?
-- cmg

[Gene Heskett]

On Tuesday 26 December 2006 16:59, jdow wrote:
> From: "James Wilkinson"
> > jdow wrote:
> > > How are you installing it? (I jettison the Fedora RPM and use cpan.
> > > That gives you a "canonical" install. Before removing the RPM save
> > > the file /etc/init.d/spamassassin. It is useful when it comes time to
> > > make spamd run.)
> > >
> > > I have a dummy spamassassin RPM that installs nothing other than the
> > > knowledge that a dummy spamassassin is present to satisfy the YUM
> > > monster.
> >
> > That's "canonical" as in "direct from
> > http://spamassassin.apache.org/%22?
>
> Effectively.
> > The download page says
> >  RPM: SpamAssassin RPMs can be built directly from the tar file.
> >
> > Simply run:
> >
> > rpmbuild -tb Mail-SpamAssassin-3.1.7.tar.gz
> >
> > If necessary use the following option if you grab the bzip2 tar
> > file instead of the gzip version:
> >
> > --define "srcext .bz2"
> >
> > You end up with two RPMs you need to install, instead of Fedora's one,
> > but you end up with canonical SpamAssassin managed by RPM.
>
> The missing files that are in that second RPM are the main reason I went
> to CPAN installs. It also means I can add the perl features that can
> make SpamAssassin somewhat stronger.
>
> RPM management seems to leave you too much at the mercy of the Fedora
> Core folks who seem to be allergic to including features from time to
> time.

"Allergic" Joanne?  In some cases its more like prophylactic shock, 
resulting in a near death experience.  But I understand the need to be a 
lady. :-)
> {^_^}

[Tim]

On Tue, 2006-12-26 at 17:59 -0500, Carroll Grigsby wrote:
> When was the last time that anyone received an email with a GIF image
> that was _not_ spam?

Probably when a stupid friend discovers Incredimail...

Though, I used to simply declare all mail spam that was HTML, or had
MIME parts that included bat, pif, gif, midi, exe, screen savers (either
by file name or MIME description)...

[Todd Zullinger]

Charles Curley wrote:
> On Wed, Dec 27, 2006 at 12:46:46AM +1030, Tim wrote:
> > Tim:
> > > Anyway, the fix for this one is simple, install perl-Archive-Tar.
> >
> > Hmm, well that fixes the installation on FC5.  FC6 has a different
> > error:
> >
> > Can't locate LWP/UserAgent.pm in @INC (@INC
> > contains: /usr/lib/perl5/vendor_perl /5.8.8/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/s ite_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.7/i386-linux -thread-multi /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi /usr/lib/pe rl5/site_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/ lib/perl5/site_perl/5.8.7 /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_per l/5.8.5 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thr ead-multi /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl 5/vendor_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.7 /us r/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/ve ndor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi /usr/lib/perl5/5.8.8) at /usr/bin/sa-update line 92.
> > BEGIN failed--compilation aborted at /usr/bin/sa-update line 92.
> >
> > How do you work out what package it wants to fulfill its needs?
>
> It says, "Can't locate LWP/UserAgent.pm in @INC..." So it's looking
> for a perl module called "LWP::UserAgent" somewhere in the list of
> directories that constitute the array @INC (short for "include"). A
> quick look at line 92 of sa-update shows that it says:
>
> eval { use LWP::UserAgent; };
>
> And the comment just above indicates that the reason the author used
> the eval trick was to bypass RPM dependency checking.

Yes, lovely of them to do so.  I am almost curious enough to trawl
through the SA buglist to see if there is a reason given for doing
this.  It seems that they don't want to pull in the deps required for
sa-update into the RPMS that they ship (they as in the SpamAssassin
developers).  But really they should split sa-update into a separate
rpm if they want its dependencies kept separate.
> Next question: do you really need it? I don't have it installed, and
> sa-update runs and gets updates without complaining. But I don't see
> how sa-update can get updates without it.

Uninstalling it causes the same error as Tim reported here.
> Another question is, how do you get it? A bit of checking with yum
> indicates that it isn't available from core, extras or livna. So you
> need cpan2rpm so you can pull it in from cpan.org and RPM-ize it for
> installation.

It's in core.  You can install it like so:

sudo yum install 'perl(LWP::UserAgent)'

The 'perl(LWP::UserAgent)' thing is handy once you know how rpm does
perl deps.  Unfortunately it requires a little familiarity with perl
before it's at all obvious I think.

I'll try to file this bug so that the Fedora packages can get fixed.
Ideally it'd get corrected upstream, but I'm not going to file a bug
there without investing more time trying to figure out why they're
intentionally working against rpm's dep checking (and I don't have the
time to do that now).

[James Wilkinson]

jdow wrote:
> Besides, WTF good is Bayes with image spam?

Actually, I find that SA's Bayesian engine is pretty good at spotting
the random text they put in most image spam. With a few extra points for
technical stuff, most of it goes directly to the spam folder, and the
rest to the "unsure" folder with fairly high scores.

James.

[Les Mikesell]

Carroll Grigsby wrote:
> On Tuesday 26 December 2006 17:09, jdow wrote:
> > > > whack
> > Scores. The magic is in scores. No single rule (usually) should be
> > allowed to define spam. (BAYES_99 is good enough here I score it
> > high enough to guarantee markup as spam. Then I rely on the small
> > number of negative scoring rules to save random ham messages that
> > might get all the way to 0.99 BAYES spam probability.)
> >
> > Besides, WTF good is Bayes with image spam?
> > {^_^}
>
> Answering a question with a question: When was the last time that anyone 
> received an email with a GIF image that was _not_ spam?

Around Christmas time?  Probably almost everyone who has friends with
a digital camera and children...  Actually those will usually be
jpegs but you are also likely to get letters with some embedded
festive gifs from people other than advertisers. Of course if you
reject all your email without reading it you might not have any
friends...

[Todd Zullinger]

> I'll try to file this bug so that the Fedora packages can get fixed.
> Ideally it'd get corrected upstream, but I'm not going to file a bug
> there without investing more time trying to figure out why they're
> intentionally working against rpm's dep checking (and I don't have
> the time to do that now).

Added to BZ #193100[1].  Justin Mason, one of the SA authors replied
to that bug to say they're aware of the issue and are indeed thinking
of splitting sa-update into a separate rpm in their packages.  So one
way or another this should get resolved for the Fedora packages as
well.

[1] https://bugzilla.redhat.com/193100

[Aaron Konstam]

On Wed, 2006-12-27 at 11:56 +0000, James Wilkinson wrote:
> jdow wrote:
> > Besides, WTF good is Bayes with image spam?
>
> Actually, I find that SA's Bayesian engine is pretty good at spotting
> the random text they put in most image spam. With a few extra points for
> technical stuff, most of it goes directly to the spam folder, and the
> rest to the "unsure" folder with fairly high scores.
>
> James.

I guess I have to point this out with a little bit of trepidation. But
if you have an unsure folder you are probably using SpamBayes not
Spamassassin.
--
=======================================================================
Help! I'm trapped in a Chinese computer factory!
=======================================================================
Aaron Konstam telephone: [phone] e-mail: [e-mail]

[jdow]

From: "Aaron Konstam"
> On Wed, 2006-12-27 at 11:56 +0000, James Wilkinson wrote:
> > jdow wrote:
> > > Besides, WTF good is Bayes with image spam?
> >
> > Actually, I find that SA's Bayesian engine is pretty good at spotting
> > the random text they put in most image spam. With a few extra points for
> > technical stuff, most of it goes directly to the spam folder, and the
> > rest to the "unsure" folder with fairly high scores.
> >
> > James.
>
> I guess I have to point this out with a little bit of trepidation. But
> if you have an unsure folder you are probably using SpamBayes not
> Spamassassin.

Thanks for saying it for me. {^_-} Bayes alone is a bicycle with one pedal.
Add some rules and DNS tests and you go from "unsure" to "pretty darned
sure" - at the level of one in a thousand or so. Add FuzzyOCR and you
step up from coaster brakes to caliper brakes. A fully loaded SpamAssassin
is to a mere SpamBayes as a top of the line multi-speed bicycle is to
a broken down beach cruiser with one pedal broken off. SpamBayes has its
uses if one lives on an Internet Beach, I suppose. It make it look to the
locals like you get some exercise even if it can't go anywhere.

{^_-}

[Florin Andrei]

Tim wrote:
> How do you work out what package it wants to fulfill its needs?

Um... learn Perl?

[Florin Andrei]

jdow wrote:
> (I jettison the Fedora RPM and use cpan. That
> gives you a "canonical" install.

yum install cpan2rpm

Then you can run cpan2rpm on any tar.gz from CPAN and build RPM 
packages. That's twice "canonical" - you're using the newest CPAN stuff 
_and_ it's all in RPM.
> I have a dummy spamassassin RPM that installs nothing other than the
> knowledge that a dummy spamassassin is present to satisfy the YUM monster.

No need to do that. Just pull all the prerequisites from CPAN, run 
cpan2rpm on each one and install the packages, then build and install 
the spamassassin RPM (rpmbuild -ta spamassassin*.tar.gz).

[Florin Andrei]

jdow wrote:
> RPM management seems to leave you too much at the mercy of the Fedora
> Core folks who seem to be allergic to including features from time to time.

lol

Nobody stops you from building your own packages.

[Florin Andrei]

Jeffrey Ross wrote:
> Be forewarned,  spamassassin can be a memory pig, I start it with the 
> following command:
>
> /usr/bin/spamd -c -m 20 -d -r /var/run/spamd.pid

Another idea is to run SpamAssassin together with Amavisd-new. This will 
allow a very simple way to also run ClamAV or any other anti-virus 
supported by Amavisd-new.

http://www.ijs.si/software/amavisd/

http://dag.wieers.com/packages/amavisd-new/

Essentially, the daemon running in this case is not spamd but amavisd, 
plus clamd. SpamAssassin then becomes just one of the libraries used by 
amavisd. Messages are also passed through clamd by amavisd.

I use this setup in conjunction with Postfix and the results are very 
good. SpamAssassin catches the spam proper, ClamAV catches the infected 
messages (and some phishing stuff too), I configured Amavis to just flag 
the spam / infected messages and Cyrus-IMAPd / Sieve is filtering those 
bad messages automatically to a spam folder and a virus folder.
Every once in a while I do a cursory search on those folders and, if 
there's no false positive, I just nuke all messages. Of course, Amavis 
could be configured to bounce the bad messages (bad idea!), or simply 
discard them. The choice is yours.

Works very well, haven't had issues with email in a very long time.

[jdow]

From: "Florin Andrei"
> Tim wrote:
> > How do you work out what package it wants to fulfill its needs?
>
> Um... learn Perl?

Which it? If you mean spamassassin and added rule sets the SARE rules
are documented for their characteristics.

http://www.rulesemporium.com/rules.htm

Read the descriptions. You'll note that "bigevil.cf" exists for legacy
or special needs. It is a delayed version of the filtering you get with
DNS tests to surbl.org. 70_sare_evilnum*.cf looks for known commonly used
addresses and phone numbers. 7x_sare_redirect_* are a host of rules of
varying "strength" from triggers on no ham in the test corpus (updated
more or less continuously) to lets almost no redirect spam trough but
has a bad habit of triggering on some ham messages.

Read. I know this is a foreign concept to young folks these days. But
the data is present. Use it.

{^_-}

[jdow]

From: "Florin Andrei"
> jdow wrote:
> > (I jettison the Fedora RPM and use cpan. That
> > gives you a "canonical" install.
>
> yum install cpan2rpm
>
> Then you can run cpan2rpm on any tar.gz from CPAN and build RPM packages. 
> That's twice "canonical" - you're using the newest CPAN stuff _and_ it's 
> all in RPM.
> > I have a dummy spamassassin RPM that installs nothing other than the
> > knowledge that a dummy spamassassin is present to satisfy the YUM 
> > monster.
>
> No need to do that. Just pull all the prerequisites from CPAN, run 
> cpan2rpm on each one and install the packages, then build and install the 
> spamassassin RPM (rpmbuild -ta spamassassin*.tar.gz).

cpan gives the opportunity to install dependancies on the fly.
{^,-}

Cpan does NOT allow uninstall in any convenient manner I have found.
{O,o} to me.

You win some and you lose some.

{^_^}

[jdow]

From: "Florin Andrei"
> Jeffrey Ross wrote:
> > Be forewarned,  spamassassin can be a memory pig, I start it with the 
> > following command:
> >
> > /usr/bin/spamd -c -m 20 -d -r /var/run/spamd.pid
>
> Another idea is to run SpamAssassin together with Amavisd-new. This will 
> allow a very simple way to also run ClamAV or any other anti-virus 
> supported by Amavisd-new.
>
> http://www.ijs.si/software/amavisd/
>
> http://dag.wieers.com/packages/amavisd-new/
>
> Essentially, the daemon running in this case is not spamd but amavisd, 
> plus clamd. SpamAssassin then becomes just one of the libraries used by 
> amavisd. Messages are also passed through clamd by amavisd.
>
> I use this setup in conjunction with Postfix and the results are very 
> good. SpamAssassin catches the spam proper, ClamAV catches the infected 
> messages (and some phishing stuff too), I configured Amavis to just flag 
> the spam / infected messages and Cyrus-IMAPd / Sieve is filtering those 
> bad messages automatically to a spam folder and a virus folder.
> Every once in a while I do a cursory search on those folders and, if 
> there's no false positive, I just nuke all messages. Of course, Amavis 
> could be configured to bounce the bad messages (bad idea!), or simply 
> discard them. The choice is yours.
>
> Works very well, haven't had issues with email in a very long time.

Wull, my take on amavisd is that is its own punishment. Configuring it
to something akin to the way I have procmail working is a royal pain
in the <sitapon part.> I see way more problems on the SA list with
amavisd, mailscanner, or things like that than I do with procmail. But
then everybody seems to be scared off from procmail. Maybe that's why.

(And introducing sendmail into the feed path can drive people insane.)

{^_^}

[Les Mikesell]

jdow wrote:
> Wull, my take on amavisd is that is its own punishment. Configuring it
> to something akin to the way I have procmail working is a royal pain
> in the <sitapon part.> I see way more problems on the SA list with
> amavisd, mailscanner, or things like that than I do with procmail. But
> then everybody seems to be scared off from procmail. Maybe that's why.
>
> (And introducing sendmail into the feed path can drive people insane.)

The sane way to glue all the mail scanning steps you want together
so they run in real time during the smtp conversation is with
Mimedefang: http://www.mimedefang.org/.
It runs as a sendmail milter and lets you control your choice of
operations with clam, spamassassin, etc. with a small snippet of
perl.  And it has a fairly active mail list for help.

[Les Mikesell]

jdow wrote:
> > No need to do that. Just pull all the prerequisites from CPAN, run 
> > cpan2rpm on each one and install the packages, then build and install 
> > the spamassassin RPM (rpmbuild -ta spamassassin*.tar.gz).
>
> cpan gives the opportunity to install dependancies on the fly.
> {^,-}

Does cpan2rpm understand cpan's knowledge of dependencies?

[Florin Andrei]

jdow wrote:
> From: "Florin Andrei"
> > Tim wrote:
> > > How do you work out what package it wants to fulfill its needs?
> >
> > Um... learn Perl?
>
> Which it? If you mean spamassassin and added rule sets the SARE rules
> are documented for their characteristics.

The guy was asking about the Perl modules required by SpamAssassin. See 
his message.
> Read. I know this is a foreign concept to young folks these days. But
> the data is present. Use it.

Time to follow your own advice. ;-)

[Florin Andrei]

jdow wrote:
> Wull, my take on amavisd is that is its own punishment. Configuring it
> to something akin to the way I have procmail working is a royal pain
> in the <sitapon part.>

I agree it's not for everyone. But for the average sysadmin should be 
easy enough.

It's not even close to the difficulty of sendmail.cf or a typical 
procmail config, mostly because it's self-explanatory, full of comments, 
and the implicit values for most variables are fine in most cases.

Finally, let's not mislead the uninitiated bystanders that amavisd is a 
procmail replacement. The two are rather different in purpose.
> I see way more problems on the SA list with
> amavisd, mailscanner, or things like that than I do with procmail. But
> then everybody seems to be scared off from procmail. Maybe that's why.

The problem with procmail (and with spamc by the way) is that you're 
running a new process (sometimes several) for each new message. This is 
fine for a low-profile server, but it gets overwhelming pretty quick 
when the traffic goes up.
amavisd is a service that runs continuously and it scales pretty well 
with traffic. Just increase the number of child daemons if need be.

[jdow]

From: "Florin Andrei"
> jdow wrote:
> > From: "Florin Andrei"
> > > Tim wrote:
> > > > How do you work out what package it wants to fulfill its needs?
> > >
> > > Um... learn Perl?
> >
> > Which it? If you mean spamassassin and added rule sets the SARE rules
> > are documented for their characteristics.
>
> The guy was asking about the Perl modules required by SpamAssassin. See 
> his message.

That is also in the wiki with examples. I didn't see the original. All I
saw was the above quotes. I started following this thread late. Normally
I pounce on spam messages. But I've been working my self to sleep for
my customer. (520k of code in the last three months or so.)
> > Read. I know this is a foreign concept to young folks these days. But
> > the data is present. Use it.
>
> Time to follow your own advice. ;-)

If it were there ....

{^_-}

[Craig White]

On Wed, 2006-12-27 at 20:17 -0800, jdow wrote:
> From: "Florin Andrei"
> > Jeffrey Ross wrote:
> > > Be forewarned,  spamassassin can be a memory pig, I start it with the 
> > > following command:
> > >
> > > /usr/bin/spamd -c -m 20 -d -r /var/run/spamd.pid
> >
> > Another idea is to run SpamAssassin together with Amavisd-new. This will 
> > allow a very simple way to also run ClamAV or any other anti-virus 
> > supported by Amavisd-new.
> >
> > http://www.ijs.si/software/amavisd/
> >
> > http://dag.wieers.com/packages/amavisd-new/
> >
> > Essentially, the daemon running in this case is not spamd but amavisd, 
> > plus clamd. SpamAssassin then becomes just one of the libraries used by 
> > amavisd. Messages are also passed through clamd by amavisd.
> >
> > I use this setup in conjunction with Postfix and the results are very 
> > good. SpamAssassin catches the spam proper, ClamAV catches the infected 
> > messages (and some phishing stuff too), I configured Amavis to just flag 
> > the spam / infected messages and Cyrus-IMAPd / Sieve is filtering those 
> > bad messages automatically to a spam folder and a virus folder.
> > Every once in a while I do a cursory search on those folders and, if 
> > there's no false positive, I just nuke all messages. Of course, Amavis 
> > could be configured to bounce the bad messages (bad idea!), or simply 
> > discard them. The choice is yours.
> >
> > Works very well, haven't had issues with email in a very long time.
>
> Wull, my take on amavisd is that is its own punishment. Configuring it
> to something akin to the way I have procmail working is a royal pain
> in the <sitapon part.> I see way more problems on the SA list with
> amavisd, mailscanner, or things like that than I do with procmail. But
> then everybody seems to be scared off from procmail. Maybe that's why.

----
you guys are comparing apples and oranges. Joanne is describing a single
user setup and Florin is describing a multi-user setup and generally, if
sendmail is my MTA, I'm still dealing with procmail to move e-mails that
are tagged by clamav or spamassassin anyway. I personally have found
procmail impossible to use if the users don't have a shell.

The fact that you see 'way more problems on the SA list with amavisd,
mailscanner, or things like that than I do with procmail' is
obvious...mailscanner, amavisd-new, etc. hook into spamassassin whereas
procmail has to deal with the results and essentially, any posting on
the SA list about procmail is off-topic. MailScanner, Amavisd-new simply
assume a basic, functional spamassassin and don't offer much in terms of
detail of extending spamassassin since that isn't their bailiwick.
----
> (And introducing sendmail into the feed path can drive people insane.)

----
sendmail like procmail and spamassassin has it's own learning curve but
it certainly is a hill most can climb if they are inclined to do so. I
find postfix a bit easier to get what I want but there are some
advantages to sendmail too.

I think that the concept that Florin was trying to get across (and Les
too) is that there are some really good wrapper systems (MimeDefang,
Amavisd-new, MailScanner) that in their own way, insert themselves into
the MTA process and call spamassassin, clamav, whatever the programs
deem appropriate and my own experimentation with them has led me to
believe that using one of these wrappers is more effective and more
efficient and I am gathering that is a common methodology for those who
run mail servers for multiple users.

As Joanne points out, spamassassin is fairly extensible and many of
those extensions aren't packaged in the common repos for fedora.

Anyway, not that this matters, I am with Florin on this as I too use
Postfix, Cyrus, Sieve but I use MailScanner and not Amavisd-new because
I have found MailScanner to be easy, effective and a bit simpler to
install/maintain/implement though my last effort at amavisd-new
(required for CiviCRM/CiviMail) demonstrated that installation and
configuration has become easier.

Craig

[Dan Carl]

I am a system admin and this subject had consumed most of my spare time.
Here's whats worked well for me.
Do some checks at the MTA level first. This will greatly reduce load on your
server.
example (sendmail.mc)
FEATURE(`dnsbl', `relays.ordb.org', `"550 Email rejected due to sending
server misconfiguration - see http://www.ordb.org/faq/%5C#why_rejected%22%27)dnl
FEATURE(`dnsbl',`sbl-xbl.spamhaus.org',`Rejected - see
http://spamhaus.org/%27)dnl
This will stop tons on spam.
I've used mailscanner for years makes admin much easier.
Mailscanner has a great RPM that contains itself, spamassassin, and clamav.
It installs flawlessly on my Fedora box.
Be sure to go through the conf file and tell it to use spamassassin.
I use razor, pyzor and bayes in spamassassin.
Also I suggest installing "rulesdujour"
To also catch those emails that contain images I use FuzzyOCR.
Be warned  FuzzyOCR was a pain to get installed.
I also run DCC as a deamon (theres a howto on the mailscanner site)
This setup stops about 99% of our SPAM, with very few false postitves.
Hope this helps others.
PS I you get stuck the folks on the Mailscanner list are very helpful

----- Original Message ----- 
From: "Craig White" 
To: "For users of Fedora" 
Sent: Thursday, December 28, 2006 10:23 AM
Subject: Re: Blocking Spam
> On Wed, 2006-12-27 at 20:17 -0800, jdow wrote:
> > From: "Florin Andrei"
> > > Jeffrey Ross wrote:
> > > > Be forewarned,  spamassassin can be a memory pig, I start it with the
> > > > following command:
> > > >
> > > > /usr/bin/spamd -c -m 20 -d -r /var/run/spamd.pid
> > >
> > > Another idea is to run SpamAssassin together with Amavisd-new. This

will
> > > allow a very simple way to also run ClamAV or any other anti-virus
> > > supported by Amavisd-new.
> > >
> > > http://www.ijs.si/software/amavisd/
> > >
> > > http://dag.wieers.com/packages/amavisd-new/
> > >
> > > Essentially, the daemon running in this case is not spamd but amavisd,
> > > plus clamd. SpamAssassin then becomes just one of the libraries used

by
> > > amavisd. Messages are also passed through clamd by amavisd.
> > >
> > > I use this setup in conjunction with Postfix and the results are very
> > > good. SpamAssassin catches the spam proper, ClamAV catches the

infected
> > > messages (and some phishing stuff too), I configured Amavis to just

flag
> > > the spam / infected messages and Cyrus-IMAPd / Sieve is filtering

those
> > > bad messages automatically to a spam folder and a virus folder.
> > > Every once in a while I do a cursory search on those folders and, if
> > > there's no false positive, I just nuke all messages. Of course, Amavis
> > > could be configured to bounce the bad messages (bad idea!), or simply
> > > discard them. The choice is yours.
> > >
> > > Works very well, haven't had issues with email in a very long time.
> >
> > Wull, my take on amavisd is that is its own punishment. Configuring it
> > to something akin to the way I have procmail working is a royal pain
> > in the <sitapon part.> I see way more problems on the SA list with
> > amavisd, mailscanner, or things like that than I do with procmail. But
> > then everybody seems to be scared off from procmail. Maybe that's why.
>
> you guys are comparing apples and oranges. Joanne is describing a single
> user setup and Florin is describing a multi-user setup and generally, if
> sendmail is my MTA, I'm still dealing with procmail to move e-mails that
> are tagged by clamav or spamassassin anyway. I personally have found
> procmail impossible to use if the users don't have a shell.
>
> The fact that you see 'way more problems on the SA list with amavisd,
> mailscanner, or things like that than I do with procmail' is
> obvious...mailscanner, amavisd-new, etc. hook into spamassassin whereas
> procmail has to deal with the results and essentially, any posting on
> the SA list about procmail is off-topic. MailScanner, Amavisd-new simply
> assume a basic, functional spamassassin and don't offer much in terms of
> detail of extending spamassassin since that isn't their bailiwick.
> > (And introducing sendmail into the feed path can drive people insane.)
>
> sendmail like procmail and spamassassin has it's own learning curve but
> it certainly is a hill most can climb if they are inclined to do so. I
> find postfix a bit easier to get what I want but there are some
> advantages to sendmail too.
>
> I think that the concept that Florin was trying to get across (and Les
> too) is that there are some really good wrapper systems (MimeDefang,
> Amavisd-new, MailScanner) that in their own way, insert themselves into
> the MTA process and call spamassassin, clamav, whatever the programs
> deem appropriate and my own experimentation with them has led me to
> believe that using one of these wrappers is more effective and more
> efficient and I am gathering that is a common methodology for those who
> run mail servers for multiple users.
>
> As Joanne points out, spamassassin is fairly extensible and many of
> those extensions aren't packaged in the common repos for fedora.
>
> Anyway, not that this matters, I am with Florin on this as I too use
> Postfix, Cyrus, Sieve but I use MailScanner and not Amavisd-new because
> I have found MailScanner to be easy, effective and a bit simpler to
> install/maintain/implement though my last effort at amavisd-new
> (required for CiviCRM/CiviMail) demonstrated that installation and
> configuration has become easier.
>
> Craig
>
> -- 
> fedora-list mailing list
> [e-mail]
> To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

[Tim]

Tim:
> > > How do you work out what package it wants to fulfill its needs?

Florin Andrei:
> > Um... learn Perl?

That's a bit over the top to do, just to find out how you determine what
Perl package relates to what Perl file it asks for.  With RPM based
things, I know I can use yum to determine what package I want for most
files.  That approach didn't work with Perl.

jdow:
> Which it? If you mean spamassassin and added rule sets the SARE rules
> are documented for their characteristics.
>
> http://www.rulesemporium.com/rules.htm

Knowing a good place to start is a help...

Thanks.

[Res]

On Fri, 29 Dec 2006, Tim wrote:
> > http://www.rulesemporium.com/rules.htm
>
> Knowing a good place to start is a help...

I trust you are not going to do this Tim, since I was called a racist by 
you for deciding what my users can or can not get in the way of 
blocking mail, including spam by using methods like SA.

However if you've actually woken up and smelt the morning coffee, I'd also 
suggest RuleDeJour script which gets all the stuff from rulesemporium 
and keeps it up to date in a daily run from crontab, 
http://www.fsl.com/support/Rules_Du_Jour.tar.gz

[James Wilkinson]

I wrote:
> Actually, I find that SA's Bayesian engine is pretty good at spotting
> the random text they put in most image spam. With a few extra points for
> technical stuff, most of it goes directly to the spam folder, and the
> rest to the "unsure" folder with fairly high scores.

"Aaron Konstam" replied:
> I guess I have to point this out with a little bit of trepidation. But
> if you have an unsure folder you are probably using SpamBayes not
> Spamassassin.

No, it's *definitely* SpamAssassin. I used to use SpamBayes, and I
brought the concept with me to SpamAssassin. I sort into "unsure",
"spam", and my normal mailboxes through procmail based on the
SpamAsssassin score.

And I *do* think an "unsure" folder is a Good Thing. The downside, of
course, is a folder I must manually check. The aim is to keep it empty.
But the *first* thing you should know when you want to filter spam is
that no system can be perfect. There will be spam that gets around all
the rules (yes, even DNS lists, if you happen to be first on a spam run)
and there will be non-spam that looks spammy. An "unsure" box is a
Pretty Good Way of getting that uncertainty into one place where you can
keep an eye on it.

This way, I can set the score needed for the "spam" box to be
comfortingly high, so practically no real mail goes there. I can set the
score needed to be marked as "real" mail to be very low (currently about
2), so practically no spam disturbs normail e-mail reading. And I still
get maybe eight "unsures" per day, most of which can be deleted on sight.

And yes, I've *still* known SpamAssassin sort good e-mail into the spam
box. Flipping Yahoo *added* enough spam words into their mandatory extra
signature (in a "Tired of _____ spam?" advert) that it triggered content
rules and Bayes. In my book, that's not a false positive, that's
electronic vandalism.

After that, all I can say is that Yahoo e-mail Should Not Be Used.

James.

[Aaron Konstam]

On Mon, 2007-01-01 at 07:56 +0000, James Wilkinson wrote:
> I wrote:
> > Actually, I find that SA's Bayesian engine is pretty good at
>
> spotting
> > the random text they put in most image spam. With a few extra points
>
> for
> > technical stuff, most of it goes directly to the spam folder, and
>
> the
> > rest to the "unsure" folder with fairly high scores.
>
> "Aaron Konstam" replied:
> > I guess I have to point this out with a little bit of trepidation.
>
> But
> > if you have an unsure folder you are probably using SpamBayes not
> > Spamassassin.
>
> No, it's *definitely* SpamAssassin. I used to use SpamBayes, and I
> brought the concept with me to SpamAssassin. I sort into "unsure",
> "spam", and my normal mailboxes through procmail based on the
> SpamAsssassin score.
>
> And I *do* think an "unsure" folder is a Good Thing. The downside, of
> course, is a folder I must manually check. The aim is to keep it
> empty.
> But the *first* thing you should know when you want to filter spam is
> that no system can be perfect. There will be spam that gets around all
> the rules (yes, even DNS lists, if you happen to be first on a spam
> run)
> and there will be non-spam that looks spammy. An "unsure" box is a
> Pretty Good Way of getting that uncertainty into one place where you
> can
> keep an eye on it.

Well I guess I can accuse you of being hypnotized by the SpamBayes
people who also think the unsure folder is a good thing. Well in my
opinion it is not. What is good about having an extra folder to check
for spam and ham? Nothing I would say. But we Linux people believe in
each persons individual freedom to make mistakes -:)
--
=======================================================================
A fool and your money are soon partners.
=======================================================================
Aaron Konstam telephone: [phone] e-mail: [e-mail]

[James Wilkinson]

Aaron Konstam wrote:
> Well I guess I can accuse you of being hypnotized by the SpamBayes
> people who also think the unsure folder is a good thing. Well in my
> opinion it is not. What is good about having an extra folder to check
> for spam and ham?

"Hypnotized", I think, is unwarranted. They think it's good engineering.
So do I. You don't.

There's only one of it, there's never much in it, and it means you
*don't* have to check the other boxes in the same way. You can sort that
one box in about ten seconds once a day. Because you *know* the e-mail
is uncertain, you're not too pre-disposed to call it spam, which means
you're more likely to *accurately* judge it. (It's well-known that
humans often won't spot one "good" e-mail in a pile of spam).

If you want an antispam program to Just Work, then you're going to be
disappointed -- it's Not Possible. If you're happy with one that works
nearly all the time, but occasionally needs help, you might just get
that.

James.

[Aaron Konstam]

On Mon, 2007-01-01 at 20:06 +0000, James Wilkinson wrote:
> Aaron Konstam wrote:
> > Well I guess I can accuse you of being hypnotized by the SpamBayes
> > people who also think the unsure folder is a good thing. Well in my
> > opinion it is not. What is good about having an extra folder to check
> > for spam and ham?
>
> "Hypnotized", I think, is unwarranted. They think it's good engineering.
> So do I. You don't.
>
> There's only one of it, there's never much in it, and it means you
> *don't* have to check the other boxes in the same way. You can sort that
> one box in about ten seconds once a day. Because you *know* the e-mail
> is uncertain, you're not too pre-disposed to call it spam, which means
> you're more likely to *accurately* judge it. (It's well-known that
> humans often won't spot one "good" e-mail in a pile of spam).

I don't want to start a religious war. I use spamassasin and it works
well. Are you saying that in addition to the unsure box you don't have
to check th spam box and of course you check the ham box (that's the
mail that was not identified as spam or unsure). If you check the unsure
box only once a day then there will probably be some mail you will not
see promptly. But if it works for you great.
>

--
=======================================================================
Pray: To ask that the laws of the universe be annulled in behalf of a
single petitioner confessedly unworthy. -- Ambrose Bierce
=======================================================================
Aaron Konstam telephone: [phone] e-mail: [e-mail]