[Techtalk] spam filters

[Kathryn Andersen]

On Tue, Oct 24, 2006 at 10:55:09AM +0200, Rudy Zijlstra wrote:
> On Tue, 24 Oct 2006, Maria McKinley wrote:
> 
> >I am currently using spamassasin to filter spam on our mail server. Lately 
> >it has not been doing a very good job. I keep trying to update it, but I 
> >always seem to be at the latest release. Does anyone have any suggestions? 

My own system works pretty well for me.  Note that I have my own domain,
and my own server, so some of the things I do may not fit everyone.  It
has a few layers of filtering.

Postfix:
I have the "smtpd_recipient_restrictions" set to a long list of things
which filters out mail coming from
(a) unknown/invalid hosts
(b) dynamic IPs
(c) blacklisted IPs

I admit, I basically cut-and-pasted the settings from our setup at work
(which was set up by a savvy guy who has since left, alas)

This is what I have in my main.cf:

smtpd_recipient_restrictions = permit_mynetworks,
    reject_non_fqdn_hostname,
    reject_non_fqdn_sender,
    reject_non_fqdn_recipient,
    reject_rbl_client sbl-xbl.spamhaus.org,
    reject_rbl_client relays.ordb.org,
    reject_rbl_client dnsbl.njabl.org,
    reject_rbl_client dnsbl.ahbl.org,
    reject_rbl_client dnsbl.sorbs.net,
    reject_rbl_client relays.visi.com,
    reject_rhsbl_client blackhole.securitysage.com,
    reject_rhsbl_sender blackhole.securitysage.com,
    reject_rhsbl_client rhsbl.ahbl.org,
    reject_rhsbl_sender rhsbl.ahbl.org,
    reject_rhsbl_client rhsbl.sorbs.net
    reject_rhsbl_sender rhsbl.sorbs.net,
    reject_rhsbl_client block.rhs.mailpolice.com,
    reject_rhsbl_sender block.rhs.mailpolice.com,
    reject_rhsbl_client dynamic.rhs.mailpolice.com,
    reject_rhsbl_sender dynamic.rhs.mailpolice.com,
    reject_rhsbl_client bogusmx.rfc-ignorant.org,
    reject_rhsbl_sender bogusmx.rfc-ignorant.org,
    reject_rhsbl_client dsn.rfc-ignorant.org,
    reject_rhsbl_sender dsn.rfc-ignorant.org
    reject_rbl_client list.dsbl.org,
    reject_unknown_sender_domain,
    reject_unauth_destination

This cuts out quite a bit of spam, but occassionally causes problems for
friends who have things like hotmail addresses (which tend to get
blacklisted).  Since they never actually read the bounce messages that
get sent back, they just ask me if I've changed my email address (which
of course I haven't).
I guess if too much isn't coming through, then one could reduce the
number of blacklists this uses.

Spam Assassin + Maildrop + Mutt:

I run Spam Assassin through my maildrop filter, rather than on a server-wide
basis, since I'm the only one who uses my server.  This means I can tune
it to my own preferences; in my user_prefs file, I beef up some scores
for things that my friends don't send me, such as HTML-only email.
I also set the spam score to 4.

Then in my maildrop filters, I do a few things:
1) anything which is flagged as spam (that is, with a score of 4 or
more) gets deleted.

        xfilter spamassassin
        if (/^X-Spam-Status: Yes/:h)
        {
                log "tagged spam! DIE!"
                exit
        }

2) anything which has a score of two or three (which I can check by
counting the '*' in the X-Spam-Level: header) gets refiled into the junk
mail folder.

        if (/^X-Spam-Level: \*\*\*/:h)
        {
                log "Dubious; refiling"
                to "$MAILDIR/.Incoming.junk/"
        }
        if (/^X-Spam-Level: \*\*/:h)
        {
                log "Somewhat dubious; refiling"
                to "$MAILDIR/.Incoming.junk/"
        }

3) anything which is sent to a particular address which got spammed to
death a few years ago, also gets sent to the junk folder

4) I do some filtering to particular folders depending on where it's
sent from (such as, all mail from "Cron Daemon" gets sent to the cron
folder, all email sent from livejournal.com gets sent to the lj folder
and so on).

5) anything which has an "In-Reply-To:" header which indicates that it's
a reply to an email of mine, gets sent to the main folder.  There's a
trick one can do with mutt to set your message id so that it has a
recognisable string which will then be used in the "In-Reply-To:"
header.

In my .muttrc:

# set my hostname to a fake hostname
# which should be used to set the Message-ID
set hostname=(someprefix).(my-real-hostname)
set hidden_host=yes

6) anything which has an X-Spam-Level of one, gets refiled to the "bulk"
folder (since it's only somewhat dubious, but probably isn't personal
email so it can be looked at later).  I also do another level of
refiling to other folders depending on senders and addresses the mail
was sent to.

I find that the "junk" folder is 99% junk mail, and not only that, but
90% of the junk mail is stuff that was sent to the "spam-listed"
address.  I go through it anyway, just in case it is wrongly marked as
"ham", which I then correct with sa-learn; likewise with the rare legit
mail which is marked as spam, which needs to be listed as ham.
Likewise, if, rarely, spam gets through to my main folder, I use
sa-learn to mark it as spam.

Protecting email addresses:
I learned my lesson from the email address that got spammed to death.
1) Never get a .com domain; they're huge targets for dictionary attacks
2) Never put an unprotected email address on a website.
   a) if you have to give an email address to someone who is going to
      carelessly put it on a website, then make it a special "email
      address for giving to naieve people"
   b) put protections in place for email addresses that need to be on a
      website.
3) I have a separate email address (and account, actually) which I only
use for mailing lists.  It has its own set of filter rules, for refiling
the various mailing list mails into different folders for each list.  It
also uses the In-Reply-To trick.  That generally takes care of most
things, since the only legitimate mail to that address is either mailing
list mail or replies to my own mail that I've sent to mailing lists.

Well, there's my 20c worth...

Kathryn Andersen
-- 
 _--_|\     | Kathryn Andersen	<http://www.katspace.com>
/      \    | 
\_.--.*/    | GenFicCrit mailing list <http://www.katspace.com/gen_fic_crit/>
      v     | 
------------| Melbourne -> Victoria -> Australia -> Southern Hemisphere
Maranatha!  |	-> Earth -> Sol -> Milky Way Galaxy -> Universe