SpamAssassin 3.04 - installed fine, but doesn't seem to do anything.

[Aeon]

After much messing around, I think I finally have SA 3.04 installed correctly. However, it still does not filter newly incoming spam. When I run my mailbox on the server through it (for example: spamassassin -t --mbox < /var/spool/mail/ds runs through the mailbox and prints out things such as

Spam detection software, running on the system "artwithin.com", has
identified this incoming email as possible spam.  The original message
has been attached to this so you can view it (if it isn't spam) or label
similar future email.  If you have any questions, see
abuse@westhost.com for details.

Content preview:  Removal Requests: 2269 Chestnut St. #631 San Francisco,
   CA 94123-2600 [...]

Content analysis details:   (6.2 points, 5.0 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 0.3 FROM_HAS_MIXED_NUMS    From: contains numbers mixed in with letters
 1.1 FROM_HAS_MIXED_NUMS3   From: contains numbers mixed in with letters
 0.2 HTML_90_100            BODY: Message is 90% to 100% HTML
 1.5 HTML_IMAGE_ONLY_12     BODY: HTML: images with 800-1200 bytes of words
 0.0 HTML_WEB_BUGS          BODY: Image tag intended to identify you
 0.3 MIME_HTML_MOSTLY       BODY: Multipart message mostly text/html MIME
 0.0 HTML_MESSAGE           BODY: HTML included in message
 1.5 MPART_ALT_DIFF         BODY: HTML and text parts are different
 1.4 DCC_CHECK              Listed in DCC (http://rhyolite.com/anti-spam/dcc/)

), it seems to classify spam correctly, but it does not do it automatically, and doesn't add the requested headers.

spamassassin -V reports the following from any directory:

$ spamassassin -V
SpamAssassin version 3.0.4
  running on Perl version 5.8.6

changed /etc/mail/spamassassin/local.cf to contain the following:

# This is the right place to customize your installation of SpamAssassin.
#
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
###########################################################################
#
rewrite_header Subject *****SPAM*****
# report_safe 1
# trusted_networks 212.17.35.
# lock_method flock

add_header spam Flag _YESNOCAPS_
add_header all Status _YESNO_, score=_SCORE_ required=_REQD_ tests=_TESTS_
autolearn=_AUTOLEARN_ version=_VERSION_
add_header all Level _STARS(*)_
add_header all Checker-Version SpamAssassin _VERSION_ (_SUBVERSION_) on
_HOSTNAME_

ok_languages en ru et fr

changed the /etc/procmailrc file to look like this:

# Added By HostDir
VERBOSE = "no"
LOGABSTRACT = "no"
SHELL = "/bin/sh"

RESIDUE = `/bin/procmail_checker  $DEFAULT $LOGNAME 2>/dev/null`

# Feed redirected spam to sa-learn, and also store a copy in a folder called spam.
# This folder of false negatives could be useful if we needed to rebuild our Bayes
# database in the future.
:0
* ^To:.*spam@artwithin.com
   {
   * < 256000
   :0c: spamassassin.spamlock
   | sa-learn --spam

   :0: spamassassin.filelock
   spam
   }

# Send all other mail through SpamAssassin

:0fw: spamassassin.lock
 * < 256000
 | spamassassin

# Mail that is very likely spam (>15) can be saved on the server
# (not forwarded), or by moving the # down one line, even dropped
# on the floor.  Note that dropping mail on the floor is a *bad*
# idea unless you really, really believe no false positives will
# have a score greater than 15.  If you want all mail forwarded,
# just add #'s in front of each of these lines:

:0: spamassassin.filelock2
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
##/dev/null
 almost-certainly-spam

:0
* ? test $RESIDUE != "unlimited"
{
 :0
 * > $RESIDUE
 {
    LOGFILE=/proc/self/fd/2
    LOG="554 - Mailbox quota exceeded by $LOGNAME
  "
    EXITCODE=69
    :0
    /dev/null
 }
}

# Jul-04 fix - Added by WH
:0 fhw
| formail -R X-UIDL: X-UIDL-OLD:

[Aeon]

spamassassin --lint -D reports:

$ spamassassin --lint -D
debug: SpamAssassin version 3.0.4
debug: Score set 0 chosen.
debug: running in taint mode? yes
debug: Running in taint mode, removing unsafe env vars, and resetting PATH
debug: PATH included '/home/artwithin/bin', keeping.
debug: PATH included '/home/artwithin/perl586/bin', keeping.
debug: PATH included '/bin', keeping.
debug: PATH included '/usr/local/bin', keeping.
debug: PATH included '/usr/bin', keeping.
debug: PATH included '.', which is not absolute, dropping.
debug: PATH included '/usr/local/apache/bin', keeping.
debug: PATH included '/sa/bin', keeping.
debug: Final PATH set to: /home/artwithin/bin:/home/artwithin/perl586/bin:/bin:/usr/local/bin:/usr/bin:/usr/local/apache/bin:/sa/bin
debug: diag: module installed: DBI, version 1.48
debug: diag: module installed: DB_File, version 1.811
debug: diag: module installed: Digest::SHA1, version 2.10
debug: diag: module installed: IO::Socket::UNIX, version 1.21
debug: diag: module installed: MIME::Base64, version 3.05
debug: diag: module installed: Net::DNS, version 0.51
debug: diag: module installed: Net::LDAP, version 0.33
debug: diag: module not installed: Razor2::Client::Agent ('require' failed)
debug: diag: module installed: Storable, version 2.13
debug: diag: module installed: URI, version 1.35
debug: ignore: using a test message to lint rules
debug: using "/etc/mail/spamassassin/init.pre" for site rules init.pre
debug: config: read file /etc/mail/spamassassin/init.pre
debug: using "/sa/share/spamassassin" for default rules dir
debug: config: read file /sa/share/spamassassin/10_misc.cf
debug: config: read file /sa/share/spamassassin/20_anti_ratware.cf
debug: config: read file /sa/share/spamassassin/20_body_tests.cf
debug: config: read file /sa/share/spamassassin/20_compensate.cf
debug: config: read file /sa/share/spamassassin/20_dnsbl_tests.cf
debug: config: read file /sa/share/spamassassin/20_drugs.cf
debug: config: read file /sa/share/spamassassin/20_fake_helo_tests.cf
debug: config: read file /sa/share/spamassassin/20_head_tests.cf
debug: config: read file /sa/share/spamassassin/20_html_tests.cf
debug: config: read file /sa/share/spamassassin/20_meta_tests.cf
debug: config: read file /sa/share/spamassassin/20_phrases.cf
debug: config: read file /sa/share/spamassassin/20_porn.cf
debug: config: read file /sa/share/spamassassin/20_ratware.cf
debug: config: read file /sa/share/spamassassin/20_uri_tests.cf
debug: config: read file /sa/share/spamassassin/23_bayes.cf
debug: config: read file /sa/share/spamassassin/25_body_tests_es.cf
debug: config: read file /sa/share/spamassassin/25_hashcash.cf
debug: config: read file /sa/share/spamassassin/25_spf.cf
debug: config: read file /sa/share/spamassassin/25_uribl.cf
debug: config: read file /sa/share/spamassassin/30_text_de.cf
debug: config: read file /sa/share/spamassassin/30_text_fr.cf
debug: config: read file /sa/share/spamassassin/30_text_nl.cf
debug: config: read file /sa/share/spamassassin/30_text_pl.cf
debug: config: read file /sa/share/spamassassin/50_scores.cf
debug: config: read file /sa/share/spamassassin/60_whitelist.cf
debug: using "/etc/mail/spamassassin" for site rules dir
debug: config: read file /etc/mail/spamassassin/local.cf
debug: using "/home/artwithin/.spamassassin" for user state dir
debug: using "/home/artwithin/.spamassassin/user_prefs" for user prefs file
debug: config: read file /home/artwithin/.spamassassin/user_prefs
debug: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
debug: plugin: registered Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84e1960)
debug: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
debug: plugin: registered Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8d353f8)
debug: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
debug: plugin: registered Mail::SpamAssassin::Plugin::SPF=HASH(0x8d0c118)
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84e1960) implements 'parse_config'
debug: plugin: Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8d353f8) implements 'parse_config'
config: SpamAssassin failed to parse line, skipping: autolearn=_AUTOLEARN_ version=_VERSION_
config: SpamAssassin failed to parse line, skipping: _HOSTNAME_
debug: using "/home/artwithin/.spamassassin" for user state dir
debug: bayes: 20324 tie-ing to DB file R/O /home/artwithin/.spamassassin/bayes_toks
debug: bayes: 20324 tie-ing to DB file R/O /home/artwithin/.spamassassin/bayes_seen
debug: bayes: found bayes db version 3
debug: using "/home/artwithin/.spamassassin" for user state dir
debug: bayes: Not available for scanning, only 3 spam(s) in Bayes DB < 200
debug: bayes: 20324 untie-ing
debug: bayes: 20324 untie-ing db_toks
debug: bayes: 20324 untie-ing db_seen
debug: Score set 1 chosen.
debug: ---- MIME PARSER START ----
debug: main message type: text/plain
debug: parsing normal part
debug: added part, type: text/plain
debug: ---- MIME PARSER END ----
debug: bayes: 20324 tie-ing to DB file R/O /home/artwithin/.spamassassin/bayes_toks
debug: bayes: 20324 tie-ing to DB file R/O /home/artwithin/.spamassassin/bayes_seen
debug: bayes: found bayes db version 3
debug: bayes: Not available for scanning, only 3 spam(s) in Bayes DB < 200
debug: bayes: 20324 untie-ing
debug: bayes: 20324 untie-ing db_toks
debug: bayes: 20324 untie-ing db_seen
debug: metadata: X-Spam-Relays-Trusted:
debug: metadata: X-Spam-Relays-Untrusted:
debug: decoding: no encoding detected
debug: Loading languages file...
debug: Language possibly: en,sco
debug: metadata: X-Languages: en sco
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84e1960) implements 'parsed_metadata'
debug: is Net::DNS::Resolver available? yes
debug: Net::DNS version: 0.51
debug: trying (3) akamai.com...
debug: looking up NS for 'akamai.com'
debug: NS lookup of akamai.com failed horribly => Perhaps your resolv.conf isn't pointing at a valid server?
debug: All NS queries failed => DNS unavailable (set dns_available to override)
debug: is DNS available? 0
debug: Running tests for priority: 0
debug: running header regexp tests; score so far=0
debug: registering glue method for check_hashcash_double_spend (Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8d353f8))
debug: registering glue method for check_for_spf_helo_pass (Mail::SpamAssassin::Plugin::SPF=HASH(0x8d0c118))
debug: all '*From' addrs: ignore@compiling.spamassassin.taint.org
debug: registering glue method for check_hashcash_value (Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8d353f8))
debug: all '*To' addrs:
debug: registering glue method for check_for_spf_softfail (Mail::SpamAssassin::Plugin::SPF=HASH(0x8d0c118))
debug: registering glue method for check_for_spf_pass (Mail::SpamAssassin::Plugin::SPF=HASH(0x8d0c118))
debug: registering glue method for check_for_spf_helo_softfail (Mail::SpamAssassin::Plugin::SPF=HASH(0x8d0c118))
debug: registering glue method for check_for_spf_fail (Mail::SpamAssassin::Plugin::SPF=HASH(0x8d0c118))
debug: registering glue method for check_for_spf_helo_fail (Mail::SpamAssassin::Plugin::SPF=HASH(0x8d0c118))
debug: running body-text per-line regexp tests; score so far=-2.623
debug: running uri tests; score so far=-2.623
debug: registering glue method for check_uridnsbl (Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84e1960))
debug: Razor2 is not available
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84e1960) implements 'check_tick'
debug: running raw-body-text per-line regexp tests; score so far=-2.623
debug: running full-text regexp tests; score so far=-2.623
debug: Razor2 is not available
debug: Current PATH is: /home/artwithin/bin:/home/artwithin/perl586/bin:/bin:/usr/local/bin:/usr/bin:/usr/local/apache/bin:/sa/bin
debug: Pyzor is not available: pyzor not found
debug: DCCifd is not available: no r/w dccifd socket found.
debug: executable for dccproc was found at /home/artwithin/bin/dccproc
debug: DCC is available: /home/artwithin/bin/dccproc
debug: entering helper-app run mode
debug: setuid: helper proc 20357: ruid=2369 euid=2369
debug: DCC: got response: X-DCC-sonic.net-Metrics: artwithin.com 1117; Body=29123 Fuz1=7122793 Fuz2=7119267
debug: leaving helper-app run mode
debug: DCC: Listed! BODY: 29123 of 999999 FUZ1: 7122793 of 999999 FUZ2: 7119267 of 999999
debug: Running tests for priority: 500
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84e1960) implements 'check_post_dnsbl'
debug: running meta tests; score so far=-1.25
debug: running header regexp tests; score so far=0.32
debug: running body-text per-line regexp tests; score so far=0.32
debug: running uri tests; score so far=0.32
debug: running raw-body-text per-line regexp tests; score so far=0.32
debug: running full-text regexp tests; score so far=0.32
debug: Running tests for priority: 1000
debug: running meta tests; score so far=0.32
debug: running header regexp tests; score so far=0.32

[Aeon]

debug: using "/home/artwithin/.spamassassin" for user state dir
debug: lock: 20324 created /home/artwithin/.spamassassin/auto-whitelist.lock.artwithin.com.20324
debug: lock: 20324 trying to get lock on /home/artwithin/.spamassassin/auto-whitelist with 0 retries
debug: lock: 20324 link to /home/artwithin/.spamassassin/auto-whitelist.lock: link ok
debug: Tie-ing to DB file R/W in /home/artwithin/.spamassassin/auto-whitelist
debug: auto-whitelist (db-based): ignore@compiling.spamassassin.taint.org|ip=none scores 0/0
debug: AWL active, pre-score: 0.32, autolearn score: 0.32, mean: undef, IP: undef
debug: DB addr list: untie-ing and unlocking.
debug: DB addr list: file locked, breaking lock.
debug: unlock: 20324 unlink /home/artwithin/.spamassassin/auto-whitelist.lock
debug: Post AWL score: 0.32
debug: running body-text per-line regexp tests; score so far=0.32
debug: running uri tests; score so far=0.32
debug: running raw-body-text per-line regexp tests; score so far=0.32
debug: running full-text regexp tests; score so far=0.32
debug: is spam? score=0.32 required=5
debug: tests=ALL_TRUSTED,DCC_CHECK,MISSING_DATE,MISSING_SUBJECT,NO_REAL_NAME
debug: subtests=__HAS_MSGID,__MSGID_OK_DIGITS,__MSGID_OK_HOST,__SANE_MSGID,__UNUSABLE_MSGID
lint: 2 issues detected.  please rerun with debug enabled for more information.

[Aeon]

I'm fairly stumped. I don't know how to get it to work. Any help would be very much appreciated - we're getting over a hundred spams per user account per day, which makes using email extremely uncomfortable (even with bayesian filtering on client side).

[jalal]

Oooh, a troll. Don't see many of those around here....



(notice how he cleverly inserts a link to the "Terrorist Organization"...).

[jalal]

I'm fairly stumped. I don't know how to get it to work. Any help would be very much appreciated - we're getting over a hundred spams per user account per day, which makes using email extremely uncomfortable (even with bayesian filtering on client side).

Make sure you use the full path to the new spamassasin, like so:

/sa/bin/spamassassin

(remember, daemons may not have a path so may not know where to find spamassassin).

If that doesn't help, then you can turn on full logging in procmail and find out where things are going from the logs.

HTH

[Aeon]

this is a bit odd. I notice now this in the logs - but these are from jul 1st, I'm guessing, from before I upgraded spamassassin - but looks like even before that, it was not working.
$ tail /var/log/maillog

Jul  1 17:48:15 artwithin.com spamc[1106]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#3 of 3): Connection refused
Jul  1 17:48:16 artwithin.com spamc[1106]: connection attempt to spamd aborted after 3 retries
Jul  1 17:48:19 artwithin.com spamc[1144]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#1 of 3): Connection refused
Jul  1 17:48:20 artwithin.com spamc[1144]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#2 of 3): Connection refused
Jul  1 17:48:21 artwithin.com spamc[1144]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#3 of 3): Connection refused
Jul  1 17:48:22 artwithin.com spamc[1144]: connection attempt to spamd aborted after 3 retries
Jul  1 17:49:40 artwithin.com spamc[1765]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#1 of 3): Connection refused
Jul  1 17:49:41 artwithin.com spamc[1765]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#2 of 3): Connection refused
Jul  1 17:49:42 artwithin.com spamc[1765]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#3 of 3): Connection refused
Jul  1 17:49:43 artwithin.com spamc[1765]: connection attempt to spamd aborted after 3 retries

if I try to start the new spamd manually, I get the following error:
[/sa/bin]$ spamd

unix dgram connect: No such file or directory at ./spamd line 1746
Could not create INET socket on 127.0.0.1:783: Permission denied (IO::Socket::INET: Permission denied)

Any idea what I need to do about this?

And as to procmailrc, if I understand correctly, I should change all instances of spamassassin to /sa/bin/spamassassin, and do the same for sa-learn? What about things like spamassassin.lock, spamassassin.spamlock and spamassassin.filelock?

[jalal]

As far as I know, spamd/spamc can't be used on WH VPSs (because it can't bind to a socket).

The only thing you should need to do is point procmail to the correct spamassassin binary, in /sa/bin/, sa-learn and the lock files are not needed (or are handled automatically).
HTH

[Aeon]

Ok. changed. But is there any faster way to test the setup other than just waiting for some new spam to roll in?

[jalal]

Ok. changed. But is there any faster way to test the setup other than just waiting for some new spam to roll in?

... which takes about 30 secs on my system...

LOL

:-)