PDA

View Full Version : Spam Assassin not rewriting message subject...



maida
08-31-2003, 12:17 AM
I just was converted to 2.0 on one of my accounts. I installed and configured the Spam Assassin on the account, and it is identifying the SPAM, however it is not rewriting message subject although I do have that box checked and just changed the Rewrite Message Subjects Text to '***SPAM***'

Any ideas on why that's not working?

Also, I'd like to know the settings that were used with SPAM Assassin before the switch to 2.0. It was working great for me until the switchover, so I'd like to use the same settings. Specifically these fields (all but the rewriting message subject fields):

Score Threshold:
Encapsulate Spam in Attachments:
Use Terse Reports:
AutoLearn-Use Bayes System:
AutoLearn-Use Auto Learning:

I have several other accounts that will be switched over in the coming weeks, so if I could figure this out for this account, that will make the transitions in the other accounts much easier!

Thanks for your time,

Maida

jim
08-31-2003, 08:50 AM
Gosh. You covered it so well. I just posted a similar but less comprehensive message in the Westhost2 General discussion area.

FZ
08-31-2003, 10:37 AM
If it is not rewriting subjects for you, then you should try editing your .spamassassin/user_prefs file manually to see if that works.

I think the following is what you should try:


required_hits 6
rewrite_subject 1
subject_tag ***SPAM***

I believe 6 was the default threshold value on WestHost 1.0. I have mine set to 4.5... Alternatively, you may want to replace ***SPAM*** (like I have) with {_HITS_} which will show you the score of that particular e-mail in the subject. For example: "{10.00} Earn $$$ from home" where 10.00 is the score for the e-mail.

Other (1.0) defaults as I remember them:

Encapsulate Spam in Attachments:

report_safe 1

Use terse reports:

use_terse_report 0

If SpamAssassin frequently marks your non-spam as spam, you should "reverse" the two settings above (that's what I did), so that all mail is untouched (but spam is marked as spam in the subject and e-mail headers).

Use bayes is supposed to be on by default, if it isn't:

use_bayes 1

I am not sure if the Bayes/Auto-learning system is installed on WestHost 2.0 though (it was on 1.0) - I remember something on the Beta test forum saying that they would look into installing it on 2.0 in the future. I think you can check by looking in the .spamassassin directory to see if you have bayes_* files.

Lastly, auto-learning is supposed to be on by default as well, if not:

auto_learn 1

I think the following settings are very useful as well:

auto_learn_threshold_spam 8.5;
auto_learn_threshold_nonspam -10;

I say this because the spam threshold default of 15 is way too high, and the default for non-spam of -2 also too high. The case may be different for you.

If you have the time, you should look over the full configuration options documentation: http://www.spamassassin.org/doc/Mail_SpamAssassin_Conf.html

It is very useful. I've been changing some of the settings over the past month or two and they work like a charm for me (blocking almost 100 spam e-mails a day). You might also want to take a look at this post: http://forums.westhost.com/phpBB2/viewtopic.php?t=151#538 if you want to move mail marked as spam to a seperate folder on the server to save you from having to download it.

Hope this helps :)

maida
09-01-2003, 11:43 AM
Yes, that totally helped! I will work on tweaking my settings because a little too much SPAM is getting through.

I wonder why the web interface did not configure it properly. When I looked at the file, all the lines were commented out. Which is probably why the interface could not modify it.

It did take me a little while to find the user_prefs file using my FTP program. I just went up a couple of directories and found the .spamassassin directory at the top.

Thanks again, design64. :D

Maida

jim
09-01-2003, 02:43 PM
If it is not rewriting subjects for you, then you should try editing your .spamassassin/user_prefs file manually to see if that works.
My user_prefs file doesn't have most of th stuff you suggest changing.

Here's it is:


# SpamAssassin user preferences file. See 'perldoc Mail::SpamAssassin::Conf'
# for details of what can be tweaked.
################################################## #########################

# How many hits before a mail is considered spam.
# required_hits 5

# Whitelist and blacklist addresses are now file-glob-style patterns, so
# "friend@somewhere.com", "*@isp.com", or "*.domain.net" will all work.
# whitelist_from someone@somewhere.com

# Add your own customised scores for some tests below. The default scores are
# read from the installed spamassassin rules files, but you can override them
# here. To see the list of tests and their default scores, go to
# http://spamassassin.org/tests.html .
#
# score SYMBOLIC_TEST_NAME n.nn

# Speakers of Asian languages, like Chinese, Japanese and Korean, will almost
# definitely want to uncomment the following lines. They will switch off some
# rules that detect 8-bit characters, which commonly trigger on mails using CJK
# character sets, or that assume a western-style charset is in use.
#
# score HEADER_8BITS 0
# score HTML_COMMENT_8BITS 0
# score SUBJ_FULL_OF_8BITS 0
# score UPPERCASE_25_50 0
# score UPPERCASE_50_75 0
# score UPPERCASE_75_100 0
jim

maida
09-01-2003, 03:05 PM
Neither did mine, but I added the ones I wanted to use.

Everything in your current file is just a comment since it has a # sign in front of it.

If you look at the documentation at spamassassin.org, you can find out all about the different configurations that are possible.

But, if all you want to do right now is get the subject line modified, this is all you need:

required_hits 5
rewrite_subject 1
subject_tag ***SPAM***

Put those 3 lines in your user_pref file and you should get most of your SPAM with a rewritten subject line.

Hope that helps,

Maida

jim
09-01-2003, 03:11 PM
thanks

ccwebb
09-02-2003, 06:15 PM
I want to put some header, subject and body rules into my user_prefs file. Like this:

# Trap mail with cindi in subject
subject L_s_OURNAME2 /cindi/i
describe L_s_OURNAME2 Look for cindi in subject
score L_s_OURNAME2 10.000

Is there anything else I have to do to get spamassassin to look at this rule in my user_prefs file? Is where it appears in the user_prefs file important?

Thanks!

Charlie

FZ
09-02-2003, 06:38 PM
I've never created or played with custom tests, so I'm not an expert... But, the first line looks incorrect to me (I referred to http://eu.spamassassin.org/doc/Mail_SpamAssassin_Conf.html). According to the documentation, it should be:


header L_S_OURNAME2 subject =~ /cindi/i

I do not think order matters, but for your own clarity you should group related lines together. Also, if you have a look at the manual, it says that these are "privilaged settings" so if you are on a WestHost 2.0 server you should put it in your /etc/mail/spamassassin file and not user_prefs. If you aren't on a new server then you can try it in your user_prefs but it may not work.

jim
09-02-2003, 08:49 PM
if you have a look at the manual, it says that these are "privilaged settings" so if you are on a WestHost 2.0 server you should put it in your /etc/mail/spamassassin file and not user_prefs.

Hmmm . . .

Well, my /etc/mail/spamassassin/local.cf file appears correct yet SpamAssassin wasn't working. I made the necessary changes and additions to user_prefs and everythng began working.

jim

ccwebb
09-02-2003, 08:49 PM
Fayez:

Thanks for your help.

One question - I am on 2.0. How do I find the /etc/mail/spamassassin file?

Charlie

FZ
09-02-2003, 09:08 PM
You're welcome, Charlie.

I'm sorry, I meant /etc/mail/spamassassin/local.cf as Jim pointed out. If you know how to use SSH, just type "pico /etc/mail/spamassassin/local.cf" (without the quotes) at the prompt and press enter - it will let you edit the file. If not, then you should be able to use an FTP program to connect to your account and browse to that directory to locate this file, download it to your hard drive, make the necessary changes and then re-upload it (renaming the original on the server just in case/as a backup). Lastly, you might be able to use the File Manager in your Site Manager to browse to this directory, though I am not sure if any non-HTML directories are available to it, in which case you'd have to use SSH or FTP.

Let me know if you need any more help.

jim
09-02-2003, 09:09 PM
How do I find the /etc/mail/spamassassin file?
What utility are you using to access the server? Try moving up directories until you can't go any further, then move down into etc, then mail . . .

It isn't really a file named spamassassin. It's a file named local.cf in the spamassassin directory.

jim

ccwebb
09-02-2003, 09:20 PM
Jim and Fayez:

I use CuteFTP for ftp.

I have a folder named .spamassassin. The folder is in /home/mydomainname.

In .spamassassin are four files:
user_prefs and three bayes_.... files.

No file named local.cf

jim
09-02-2003, 09:54 PM
I have a folder named .spamassassin. The folder is in /home/mydomainname.
That looks like you haven't been switched to the new system (Westhost2) yet. That's the directory structure for the old system.

If I were you I'd wait until switched or you'll just have to do it over again in the next day or so.

ccwebb
09-02-2003, 10:01 PM
Jim:

I am on 2.0 (I think) For instance to pull my email I had to change from username to username@mydomain.com.

I am setup under mydomain.com/manager.

How can I tell for sure? I was one of those who got no emails telling me the conversion was coming or an email telling me it was done.

I found out by being unable to pull emails until I changed username.

Charlie

jim
09-02-2003, 10:26 PM
Charlie

I think this is over my head. My directory structure is the one you see in the paths previously given. Your directory structure is the one I had before the conversion.

jim

ccwebb
09-02-2003, 10:28 PM
Jim:
Thanks a lot.

I'll call Westhost.

Charlie

FZ
09-03-2003, 11:35 AM
If you are on 2.0, you'd just use your FTP program to go up the the root directory (/), like Jim said - by going "up one directory" each time, and then you'd enter each of these directories: etc, then mail, then spamassassin and then locate the local.cf file. If you can't access it or can't see etc in the root (/) directory, that probably means you are still on an old server, in which case you could wait for 2.0 or you could try your new settings in user_prefs in the .spamassassin/ to see if that works or not.

trinity.westhost.com
09-03-2003, 11:48 AM
Okay, I finally managed to get SpamAssasin to mark subjects. Everything design64 states is true and correct, except for a few details.

I don't believe that /etc/mail/spamassassin/local.cf does anything at all except provide you with a base file to copy to your user's user_pref file.

I had to edit the user_pref file for my primary user (located in /home/username/.spamassassin/user_pref) and each secondary pop user (located in /ftp/pub/user/.spamassassin/user_pref). When I added the prescribed lines to each user's preferences file, suddenly Spam got marked.

You may need to be sure that each user has a "home" directory using the site manager to give ftp access and home directory for each user.

jeff
09-03-2003, 11:56 AM
I believe the problem with the default spamassassin config file - /etc/mail/spamassassin/local.cf - that makes it not do anything, is that it sets the values of the options to "true" or "false" instead of 1 or 0.

Spamassassin doesn't appear to like this very much and ignores those lines. You can fix the default local.cf file by manually editing it. I submitted a help ticket for this, because it looks to be a general problem.

ccwebb
09-03-2003, 12:01 PM
I figured out my problem with the help of Jim and Fayez.

I was FTP'ing using username instead of username@mydomain.com. I was pulling up the old file structure instead of the 2.0.

Now if westhost would have only told me this (I received no emails telling me they were converting me over and no emails telling me I had been converted ) I could have avoided the problems.

Actually they probably emailed me but somehow they lost 1 1/2 days worth of emails - yet to be found.

Thanks again!

Charlie

FZ
09-03-2003, 12:48 PM
Trinity,

The local.cf is (supposed to be) the "master" or "global" file - everything in that file is applied to all POP3s on your account. As Jeff stated, it seems to be configured incorrectly and that is the reason it is not working. I would confirm this, but I am not on a 2.0 account as yet so I cannot. Try making the changes Jeff suggested. Also, if you get local.cf working as it should, you won't have to go and edit each user's user_prefs manually. Oh and just a tip: you do NOT have to use /ftp/pub/username for an E-mail/FTP account - I found this out when I was part of the Beta test group. I believe adding each user to /home/username/ instead of /ftp/pub/ is better since it maintains "consistency" with your account/home directory. You could also put them in /home/users/username/ if you like so that they are all under one "dedicated" location. Finally, if a user is not given FTP access, they don't have a home directory and therefore don't have access to personalize/customize SpamAssassin, etc. - at least that is how it was during the Beta test. Maybe WestHost has changed/fixed this...

FZ
09-03-2003, 12:49 PM
Glad you got it working, Charlie! 8)

jim
09-03-2003, 01:04 PM
The local.cf is (supposed to be) the "master" or "global" file - everything in that file is applied to all POP3s on your account. As Jeff stated, it seems to be configured incorrectly and that is the reason it is not working. I would confirm this, but I am not on a 2.0 account as yet so I cannot.

Here's my local.cf:

# SpamAssassin config file for version 2.5x

# How many hits before a message is considered spam.
required_hits 6

# Whether to change the subject of suspected spam
rewrite_subject true

# Text to prepend to subject if rewrite_subject is used
subject_tag ***SPAM***

# Encapsulate spam in an attachment
report_safe false

# Use terse version of the spam report
use_terse_report false

# Enable the Bayes system
use_bayes false

# Enable Bayes auto-learning
auto_learn false

jim

ccwebb
09-03-2003, 02:14 PM
Jim:

I have encapsulate turned on - all spam mail comes to me as an attachment. Someone told me that the spammers have the capability to tell that you clicked on their mail which means they know you "looked at " the mail.

If it is encapsulated you never open it.

Comments?????

Charlie

FZ
09-03-2003, 02:33 PM
If I may take that one, Charlie ;)

What you said/heard is partly true. They don't know when you have actually clicked on it - they do know you when you access it because they can include HTML in their message with, say, a reference to an image or external file located on their server with a query string (e.g. www.getspamforfree.com/mainlogo.jpg?userid=318719785) and then when you open/view that e-mail your e-mail program tries to load the image or file. All that is left to do is for the spammer to take a look at his/her access logs and do a simple search to get a hold of the IDs that were "confirmed". SpamAssassin actually picks up on that and increases the mail's score to increase the probability of it being marked as spam. If you like, you could set a higher score for mail that includes any kind of unique identification methods. Pretty scary thought, don't you think? :evil:

So, if you do receive a lot of spam - in particular spam with HTML and images in it, then you should leave that option on. However, if it frequently marks legitimate mail as spam, it gets annoying having to open the mail if its an attachment. So, in the latter case you might want to disable it at the risk of confirming your address is a "live" one. What I have done is use Procmail to filter mail marked as spam (moving it off the POP3 and into a seperate mail folder), and have turned off the encapsulate option. Once a day I login through SSH to take a look at the mail using Pine (which does not support HTML and so is at no risk) to make sure it is spam and then I delete it forever. Of course that requires a lot of learning in terms of Procmail, etc. so you might consider an easier alternative such as ePrompter http://www.eprompter.com a free Windows program which sits in your system tray and checks your mail account(s) after a specified interval. It does not support HTML, and allows you to preview the mail (in text mode) you have received without any risk - so, you just need to preview mail to see if it is Spam or not and if so, mark it for deletion at the next interval. Keep in mind though that it does not allow you to open attachments, so you won't be able to get at mail that is "encapsulated".

If you would like me to point you to some helpful sites (and posts on this forum) on Procmail, let me know.

And yes - if it is an attachment you are not affected (unless you open the attachment and view the mail, of course).

Hope that helps you.

FZ
09-03-2003, 02:42 PM
Thanks for posting your file, Jim. What I actually meant was that I don't have access to the whole system as such to actually test it - not just the file :oops:

However, you appear to be correct in saying that using true and false is incorrect - the manual and examples all use 1's and 0's. Can you confirm you actually have it working by changing them all accordingly? If you set one option to using the "word" method, test it, and then use the "binary" method and test again, does it only work in the latter case?

Thanks.

jim
09-03-2003, 03:06 PM
I have encapsulate turned on [snip]

If it is encapsulated you never open it.

Comments?????

The file I posted isn't controlling spamassassin on my account. That's the local.cf which doesn't work, so I'm not using it. I posted it so Fayez could see what one looks like. I use the user_prefs which does the trick for me.

But I don't have encapsulate turned on because all incoming mail marked ***SPAM*** goes straight into the trash -- I never see it and thus never open it.

Like Fayez explained, the problem only happens if you open a message with an HTML link (something like a .gif) that phones home. If you don't open the message that doesn't happen.

If you're tempted to open the messages marked spam then it might be useful to encapsulate. But, then if you open the spam why wouldn't you open the encapsulated spam and boom, ET phones home.

jim

jim
09-03-2003, 04:44 PM
Can you confirm you actually have it working by changing them all accordingly? If you set one option to using the "word" method, test it, and then use the "binary" method and test again, does it only work in the latter case?

OK, it works if you replace the True/False in the local.cf file with numbers. To verify it worked I deleted the user_prefs and changed subject word to TEST in local.cf.

One thing I canít seem to get working however is making it *not* report all this junk:

---- Start SpamAssassin results
20.30 points, 6 required;
* 2.0 -- Sent with 'X-Priority' set to high
* 0.5 -- BODY: Something is emphatically guaranteed
* 2.2 -- BODY: Information on getting a larger penis or breasts
* 2.8 -- BODY: Information on getting a larger penis or breasts (2)
* 0.5 -- BODY: One hundred percent guaranteed
* 0.5 -- BODY: Asks you to click below (in capital letters)
* 2.2 -- BODY: Talks about exercise with an exclamation!
* 0.5 -- BODY: Message is 80% to 90% HTML
* 0.1 -- BODY: HTML has "tbody" tag
* 0.2 -- BODY: HTML font face is not a commonly used face
* 0.1 -- BODY: HTML font color not within safe 6x6x6 palette
* 0.1 -- BODY: HTML font color is red
* 0.1 -- BODY: HTML included in message
* 0.1 -- BODY: FONT Size +2 and up or 3 and up
* 0.0 -- BODY: HTML has very strong "shouting" markup
* 0.7 -- RAW: Message text in HTML without specified charset
* 0.3 -- RAW: Quoted-printable line longer than 76 characters
* 0.5 -- URI: Completely unnecessary %-escapes inside a URL
* 0.3 -- Date: is 3 to 6 hours before Received: date
* 3.3 -- Forged mail pretending to be from MS Outlook
* 0.1 -- Message only has text/html MIME parts
* 0.5 -- Message has X-MSMail-Priority, but no X-MimeOLE
* 2.7 -- HTML comments which obfuscate text

---- End of SpamAssassin results

I include
use_terse_report 1
thinking this should suppress the junk but it doesnít. Do you know how to avoid all that stuff?

jim

FZ
09-03-2003, 06:05 PM
Thanks for testing, Jim!

In response to your question, I don't think you can totally remove the report(s) - you can have it in the header only if you like, or there AND in the body too. But, you might want to try the following two options and see if it helps:


always_add_headers 0
always_add_report 0

Let me know how it goes.

Useful: http://eu.spamassassin.org/doc/Mail_SpamAssassin_Conf.html