PDA

View Full Version : Procmail - Triggering Perl scripts with emails



robinhemmings
08-07-2003, 11:54 AM
I wonder if anybody has had any luck using Procmail (or any other method, but I think this is the way to go..?) to get Perl scripts to run in response to incoming emails?

Although Westhost don't provide tech support for Procmail, they do have v 3.22 installed which is A Good Thing! After reading other topics, including this one http://forums.westhost.com/phpBB2/viewtopic.php?t=151 I have had success in forwarding mails by doing the following:

1. Don't have a .forward
2. /home/username/.procmailrc reads:
LOGFILE=$HOME/procmail.log
VERBOSE=yes
:0c
! myotheraccount@hotmail.com
3. chmod 0644 .procmailrc

This forwards a copy of every email on to my Hotmail account, and presumably means that other forwarding recipes will work too (I haven't tried, this isn't what I want to do).

:idea: I was hoping to change the last line in (2) above to read
|/usr/bin/perl /home/username/www/cgi-bin/incoming/triggerscript.pl

:cry: It doesn't work. All that happens is I receive the email through my catch-all account. The script remains very, very untriggered.

:? Funny thing is, I saved a blank email as "testmail.eml" in my home directory, then issued this through SSH:
cat testmail.eml | /home/username/www/cgi-bin/incoming/triggerscript.pl
:!: Whaddaya know? It works! But the problem remains - how to automate it with Procmail.

Has anybody out there had a similar problem? Any thoughts, ideas or solutions greatly appreciated! Thanks!

FZ
08-08-2003, 03:24 PM
robinhemmings,

I'm not sure if the following will help (I haven't tried redirecting mail to scripts), but you might as well try...

1) Have you set the PATH "config option"? Try setting it at the top of .procmailrc:

PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/bin:/home/USERNAME/bin
Replace USERNAME...

2) Have a look at this post: http://forums.westhost.com/phpBB2/viewtopic.php?t=18 He is also redirecting to a Perl script, but he does not have /usr/bin/perl... Maybe it's redundant? Also, maybe you need a space after the |

3) Have a look at your procmail log... Seeing as you set VERBOSE=yes, it might give you a clue as to what is wrong.

4) Maybe you need to change the permissions on your Perl script?

jhalbrook
08-09-2003, 07:58 AM
Might follow this thread:
http://clue.denver.co.us/pipermail/clue-tech/2002-October/005978.html

Also, here's a link to a Linux article on how to do this:
http://www.linux-mag.com/2001-07/guru_04.html

BOL,

Joe Halbrook
CleanMyMailbox.com

robinhemmings
08-12-2003, 03:00 PM
First I'd like to thank you both with your efforts toward trying to solve this problem. Unfortunately I'm still playing detective, despite trying many variations on the recipe as suggested in various forum and web articles.

Design64, your four points are all good....
1. I'm pretty hazy on the PATH you give and its purpose, but a few articles mention it so I've incorporated it... Doesn't seem to help on its own though.
2. Yes I agree: I think the Westhost server doesn't need that rather stately "/usr/bin/perl" preamble, so it's best left out.
3. Yes the log does make interesting reading. It shows that only mails sent from within the system (ie by SSH or by Cron) are making it run. If I just email the domain, nothing appears in the procmail.log! Now that can't be right... It feels similar to the fact that I can "cat" a blank email saved on the system via SSH and pipe it into the file, and get the right result, but the script won't trigger by itself... It is as if mails from outside aren't making it through. Big, big clue - but I can't see how to use it to advantage!
4. Possibly, but I have all three 'x' and 'r' bits set and only owner 'w' bit... Same with the folder.

Jhalbrook, your links are most useful... The Linux-Mag one is a nice primer and makes me wonder if piping back through mail to the script internally is the answer, but also raises the issue of locking, quote:"We don't use procmail locking here; if the script does any writing to files, it will need to do its own locking (procmail locking is not recommended for this purpose)." My log file shows locking happens when cron sends me emails internally. I'm also given cause to wonder about procmail flags (-fbw anyone?) by the thread you reference...

My "best guess" is now:

PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/bin:/home/username/bin
MAILDIR=/var/spool/mail/username
LOGFILE=${MAILDIR}/procmail.log
VERBOSE=yes
LOG="--- Logging a response... "

:0c
| /home/username/www/cgi-bin/scriptfolder/capturescript.pl

Well, any further ideas still needed, although I've got plenty of leads to follow up I'm not sure which way to go!

Cheers:

FZ
08-12-2003, 04:18 PM
Very, very strange... I decided to kill some time and test this out myself - and I got it working in 5 minutes!

My .procmailrc (only the relevant bit):

PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/bin:/home/design64/bin
MAILDIR=$HOME/mail
LOGFILE=$HOME/procmail.log
VERBOSE=yes

:0:
* ^To.*script@design64.net
| /home/design64/www/cgi-bin/test.pl >>/home/design64/www/cgi-bin/test.txt

... (my other rules) ...

:0:
${DEFAULT}
The >> bit isn't actually supposed to be on a new line; it's just been wrapped here. design64 is my username (and design64.net my domain, obviously).


My test.pl:

#!/usr/bin/perl

while (<>) {
print $_;
}
.procmailrc is CHMOD'd 644, and my test.pl 755 (775 also works).


Here's what I think:
* Check your permissions again ;)

* Maybe you can't keep a copy of a mail being sent to a script (i.e. the c flag might need to go) and/or you need to have a lock file (one named by you or a default one. In the case of a default one, just add a second colon as above)?

* Maybe your Perl script is buggy? Did you upload it in ASCII mode? Try a simple script first (like mine above) and see if you can't get that working before you try something complex.

* And finally, what seems most likely: maybe your (old) condition had incorrect syntax. Try it exactly as I have above - matching on the To: header (you need to have "catch all e-mail's" turned on).

I tried e-mailing from two different programs using two different SMTP servers and it worked fine in both cases.

Hope this helps! If not, you might want to send me your Perl script and I'll see whether it works for me or not.

FZ
08-12-2003, 04:23 PM
I just noticed you are using /var/spool/mail/ as your mail directory... This is not necessary. If you are not on a WestHost 2.0 server, this is actually incorrect in that we don't have access to this directory. You can name your maildir anything you want - I recommend using $HOME/mail. This way you have full access and your log file will be accessible too. I see, though, that you do seem to have access/rights to this folder - are you a new WestHost customer? :)

robinhemmings
08-13-2003, 02:24 PM
:o Curiouser and curiouser...

Hi D64, thanks for taking the time to experiment and reply!

I like your ideas, so I've put your Perl script up in ASCII format ( :) and checked that the other file is too - It was put up as part of a batch with other files which I know work). Chmod to 755 too, of course. The following works:
cat testmail.eml | /home/hypnosisdownloads/www/cgi-bin/test.pl >>/home/hypnosisdownloads/www/cgi-bin/test.txt
It creates a copy of the testmail.eml file in test.txt of cgi-bin as it should... but with the following _exact_ .procmailrc it's still no go:
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/bin:/home/hypnosisdownloads/bin
MAILDIR=$HOME/mail
LOGFILE=$HOME/procmail.log
VERBOSE=yes

:0:
* ^To.*script@hypnosisdownloads.com
| /home/hypnosisdownloads/www/cgi-bin/test.pl >>/home/hypnosisdownloads/www/cgi-bin/test.txt

:0:
${DEFAULT}

I mailed to script@hypnosisdownloads to trigger the rule, and to my default address -- still not so much as an entry in the procmail.log

I too was wondering about the c flag and the mail address. Now it seems we've ruled those out - although the latter was clearly wrong :oops: BTW I do have access to my mail file (the username after /spool/ ) but not the folder above it.

Looks like it might be locking then, because of writing to files (which your test.pl also does) although it is strange that it seems to be working for you without paying attention to that :(

I think next step is, I will of course repeat the whole uploading process forcing ASCII mode, but I think file locking might be the topic to zero in on.. any ideas on that?

I'll be here smashing my mind out with a slice of melon if anyone needs me :shock:

Cheers

FZ
08-13-2003, 03:34 PM
I'm stumped :?

You say there are no entries in your Procmail log file - is it (procmail.log) at least created in the process (is it blank?!)? i.e. is Procmail itself working for you at all? If it is, it should not be created and left blank - it will at least have a few lines of what it did that caused its execution. Remember to name it properly (.procmailrc) no file name, the whole thing is just a file "extension"; uploaded in ASCII mode to your /home/username/ directory; and CHMOD'd 644, not 755/775. Try setting up a simple Procmail rule to ensure it's working and then proceed to debug the other stuff. I can't tell from your post whether your ">>/home/..." line is wrapped or if you actually put it on a new line - if it's the latter, that's incorrect... Move it back up to after the "| /home/..." line.

I doubt the problem is with lockfiles - if this was the case, the Procmail log would be filled with a "useful description" of what happened (e.g. the actions it took trying to assign a lockfile, if it failed, etc.). I did not bother with lockfiles or anything else other than what I posted above, and it worked 100%...

I am sure there is just a silly mistake somewhere in your code (or something)... That's almost always the problem I have :x

Good luck and let me know if you get it working or not!

robinhemmings
08-14-2003, 01:53 PM
There are entries in my procmail.log - but they only ever refer to mail that was sent from within the system. Ie, they refer to my Cron tasks, both of which send an email to my Westhost domain. .procmailrc-yes, that's the filename I use - hence 'hidden' from FTP progs but appears with a ls -a). Procmail.log says "no match on (condition)", "locking /var/spool/mail/hypnosisdownloads", "acquiring kernel lock", "unlocking /var/spool...", and "notified comsat: "hypnosisdownloads@325340:/var/spool/mail/hypnosisdownloads"". Then there's a bit of email header. This happens for every cron email sent, I think. (certainly for an hourly email I have going out.)

I did manage to redirect incoming mail using .procmailrc (witha !), but that's about it. I wonder if it's worth re-sending the email from within there ... but that's probably only going to confuse things. Oh yes, the wrapping - that shell command's all on one line, it was just wrapped in the forum. Sorry.

I will fiddle with it over the next couple of days afresh. I see what you mean about locking - if it works for you it should work for me. Therefore, there *must* be an obvious mistake somewhere... I may even try it on another server to see if I get results any different!

I'll let you know - Meantime, once again my thanks!!

Bobcat

robinhemmings
08-17-2003, 02:56 PM
Hmmm... I think I may be onto something here.

I've just found (testing using the test.pl suggestion given above) that I seem to be able to trigger the script using internal emails (ie login via SSH and use mail command... mail hypnosisdownloads@westhost.net)

This seems to tally with the fact that other "internal" mails are working (ie the Cron tasks which send mails, as shown above - they come from "root").

Using my outside-westhost email address I have also just found that I can mail hypnosisdownloads@westhost38.westhost.net and this triggers the script via .procmailrc! Eureka! 8) :lol:

Wonder why this is the case though, and how best to use this new knowledge to fully solve the problem. Forward all mail to westhost38 address perhaps... Hmmm... :?

FZ
08-17-2003, 03:15 PM
Once again, that is really strange :?

At least you kind of got it working. The only thing that comes to mind that could (logically) affect this is permissions. Maybe it's time you contacted WestHost tech support about it, because it obviously is not working correctly :( Though they say they don't offer support, you could refer them to this topic and show them you have tried everything and still can't get it working correctly/fully... Maybe you should try to find someone else on the same WestHost server [westhost38] as you and see if they can get it working - if not, then it's obviously a problem with the server and not your script, etc.

robinhemmings
09-10-2003, 08:16 AM
Hi,

Just wanted to say thanks again for all your efforts, and to let you know how it all worked out.

Really things haven't moved on much since previous posts. The definitive solution, it appears, is to not have a catch-all ("default@...") account at all. I can still have POP accounts to my heart's content. But with no catch-all, anything which doesn't match a POP goes into Procmail ... And from then on it's all plain sailing.

Practically all of the Procmail recipes we discussed earlier were correct in one form or another. There is no problem with carbon copying messages into pipes either. So you can emulate the catch-all by just ccing any emails into pipes that you want, and then at the end of the recipe have a redirect (using "!") to - say, a Westhost POP account!

The only problem there is that incoming mails trigger the script as required, but also appear in your new catch-all ! destination. Which may be overkill. But the only alternative to that is to not have a catch-all in any form whatsoever, and run the risk of losing misaddressed emails.

It seems this is as good as it gets, without specific Procmail help from Westhost support for Procmail and an alternative mode of operation for the catch-all, neither of which are on offer. :roll:

Still, this is an almost perfect solution in an imperfect world... Cheers! :)

FZ
09-10-2003, 06:14 PM
Hey Bobcat,

Thank you for the update. I am glad you have it working. However, I'm still a little worried that you cannot get it working the way it is supposed to and are having to do all sorts of other things that should not be necessary. Are you on 2.0 yet?

What worries me specifically is that you are having to sacrifice the catch-all feature just to get it working. This is not how it should be and is not the case with me. I have catch-all enabled for both of my accounts - a 1.0 and a 2.0 account. Procmail works perfectly on the former, and on the latter it works, but seems a little dodgy (buggy).

And about specific help from WestHost, I am sure they would be happy to help you in this case where it is obvious that Procmail is not functioning correctly... Just not in the next few week though, while they are squashing high priority bugs and support tickets with people that have had 2.0 migration problems.

robinhemmings
11-27-2003, 09:27 AM
Hello Fayez,

Ha - after two months I have finally seen your latest message!

Thanks for your comments and I think you are right. Also as you mentioned, perhaps 2.0 is going better with Procmail now since, given we are 2 months down the line, teething troubles may now be sorted :)

I'll check with my client whether he's on 1.0 or 2.0 ... as the time has already come when he may need to use a catch-all and he probably won't want to have to delete all the subscribe / unsubscribe mails manually!

Best Regards

FZ
11-27-2003, 10:49 AM
Cool. Hope you get it working as needed...

robinhemmings
12-07-2003, 02:06 PM
Hi FZ.

I checked with my client and he's on 1.0 ... WestHost Support have told me about 2.0 and it sounds like the answer to all these problems, and I can do away with Procmail too. Apparently you have a control panel where you can just pipe it in anyway without resorting to Procmail.

I had, in fact got it pretty much working except that any change to the default address requires a change to my own "catch all surrogate" at the end of the Procmail recipe...Oh, and the unwanted copy of each subscribe mail still goes to that address.

Still, all is good enough for now, and as I say 2.0 will probably lay this problem to rest!

All the best

Robin

FZ
12-07-2003, 02:46 PM
Hey Robin,

Good to hear. Yes, there is a place in the 2.0 Site Manager where you can pipe specific addresses to scripts (http://manual.westhost.com/part4.html#aliases), so hopefully that will solve your problem(s). Let's just hope they get you on 2.0 soon, seems like they've delayed further conversions a lot longer than people would have liked...

SJP
02-05-2004, 09:03 AM
I'm not on 2.0 yet. Procmail isn't installed or at least I don't think so. There's no .procmail. Wondering if I could simply add a line to my .redirect file that would instruct the MTA to pipe a message to a particular user to a program doing something like:

pictures |$home/program

Is the .redirect syntax synonomous with .forward, because I know the above (less the first part - pictures) is permissable.

thanks, SJP

FZ
02-05-2004, 10:18 AM
SJP, Procmail is definitely installed on 1.0 servers too - I know because I had an old account on 1.0. I forget where the .procmailrc file goes, but I do remember that I did not have to fiddle with .redirect or anything. Hmm, actually, you can read the 1.0 Procmail "guide" I posted a while back here: http://forums.westhost.com/phpBB2/viewtopic.php?p=538#538

SJP
02-05-2004, 09:44 PM
Thank you for the update FZ. I'll use procmail as a last resort, but for my needs .forward would be adequate - I'm just afraid of screwing up the catch-all. My question shouldn't be such a stretch. Is there no one that does anything differently, because the general lack of usable information in the WH manuals are leading me to believe so.

SJP

SJP
02-06-2004, 12:35 AM
Thanks again FZ. Fortunately the man pages are still around on 1.0. Is it true they're gone under 2.0? I did a man on procmail and it told me where everything was and your write-up helped to 8) So what I ended up with was:

:0
* ^To: <?pictures@
|$HOME/pictures.pl

Of course I have no idea if it's going to work. I haven't written the pictures.pl program yet and much to my dismay there was no entry in the error-log that it wasn't found :( , but it didn't bounce or end up in the catch-all account which I think is a good sign 8)

SJP

FZ
02-06-2004, 10:42 AM
You're welcome, SJP. Procmail should not screw up your catch-all - if anything, I think .forward would do much worse ;) Unfortunately, it is true that the man pages are gone in 2.0 - but it's a minor annoyance, I mean you can get those exact manuals on the net (just search Google) in a much easier to read/use/search through format.

As for your code, it looks okay, but to be really effective, I would recommend a minor change - use ^TO_pictures instead of ^To: ... because the former will match CC:, BCC: and so on. If the mail seems to go missing with no visible errors, what you might want to do first is replace | $HOME... with $HOME/test to see if mail is being processed and saved to a mailbox file. If so, you can move on to the next step of piping it to your program. Also, remember to turn Procmail logging on just so you can see what is happening - try it with the VERBOSE=YES option too.

SJP
02-07-2004, 12:37 AM
FZ, you're saving me some major headaches and premature balding. Thanks.

SJP

FZ
02-07-2004, 06:30 AM
No problem.

SJP
02-21-2004, 03:05 PM
I'm so grateful for the help. Here's a sample application. E-mail a picture to sample@sanjuanpersonals.com and then look at the http://www.sanjuanpersonals.com/sample.html file. The perl program extracts a base64 encoded jpeg image and hands it over to a public domain base64 decoder program written in C. You can see the source by putting http://www.sanjuanpersonals.com/sample.pl into your browser. The .procmailrc in the root of your FS would read
:0
* ^TO_<?sample@
|$HOME/sample.pl

SJP

FZ
02-21-2004, 08:03 PM
It works! Good job :)

tajuho01
02-24-2005, 06:28 AM
Excellent thread. Exactly what I was looking for. I still have one Qs, is it possible to pass the subject of the email as an argument to the perl script ?. Any example .procmail rule ?

Thanks

SJP
02-26-2005, 11:04 PM
No takers eh? What happened to FZ? He'd be on top of this in a hot second!

I don't know how to get procmail to pass the subject, but since it pipes the entire message to STDIN you could extract it yourself. e.g.

while (<>) {
last unless (/./);
if (/Subject:\s+/g) { (my $sub)= /\G(.+)/; }
}


SJP