procmail replacement in Python
Gerhard Häring
and run away screaming. I'd very much prefer a Python replacement to
such badly written C code.
Does anything exist that I can start from? I've even contemplated not
inventing yet another filter language, but just using plain Python
functions.
If I do this, I'd like to replace fetchmail with getmail as well.
What would you suggest as approach?
- writing it as a getmail plug-in
- writing it as a standalone program to deliver to mailboxes a la
procmail
- let getmail deliver to a temporary folder, then using my procmail
replacement to filter into the right folders, delete spam, etc.
The third one looks to be the least likely to lose mail, but the
temporary folder approach looks like a kludge.
Gerhard
PS: I've seen something _very_ limited written in Python on Sourceforge
that only does Maildir, but the name is escaping me.
--
This sig powered by Python!
Außentemperatur in München: 23.3 °C Wind: 0.9 m/s
Sean 'Shaleh' Perry
On 18-Jun-2002 Gerhard Häring wrote:
> Does such a thing exist? I've recently looked into the procmail sources
> and run away screaming. I'd very much prefer a Python replacement to
> such badly written C code.
>
> Does anything exist that I can start from? I've even contemplated not
> inventing yet another filter language, but just using plain Python
> functions.
>
there is also a nice filter language from CMU called Sieve (I think). You
could implement a python based Sieve interpreter.
Gerhard Häring
> On 18-Jun-2002 Gerhard Häring wrote:
> > Does such a thing exist?
>
> could implement a python based Sieve interpreter.
I'd just switch to Cyrus IMAP instead. I'm currently using Courier IMAP
and am quite satisfied, so I have little reason to switch. I even found
Cyrus difficult to install, even though it's also available as a FreeBSD
package (just like Courier).
All that I'm currently using from procmail are regexes on header fields,
combined with AND, and maybe OR conditions. And having two custom
filters before matching the header fields:
- spamassassin
- this one to fight stupidity in the setup of the German freemailer GMX:
they have some braindead software that does mbox-like From_ quoting on
their POP3 setup :-(
#!/usr/bin/env python
import re, sys
BROKEN_FROM = re.compile(r"^(>+)(from )(.*)", re.I | re.DOTALL)
for line in sys.stdin.xreadlines():
match = BROKEN_FROM.match(line)
if not match:
sys.stdout.write(line)
else:
# Remove one quote char added by braindead GMX software
groups = match.groups()
escape_chars = groups[0][:-1]
sys.stdout.write(escape_chars + groups[1] + groups[2])
Gerhard
--
This sig powered by Python!
Außentemperatur in München: 23.1 °C Wind: 0.9 m/s
François Pinard
> I've recently looked into the procmail sources and run away screaming.
ROTFL! If I remember well, the original author knew he had a peculiar
writing style, and was writing in some prominent file (either an FAQ or
the README, or something like) that his style was not to be discussed! :-)
I wondered if the new maintainer revised the sources or not, I did not check.
> I'd very much prefer a Python replacement to such badly written C code.
Is it bad style? I never went as far as asserting that it was bad style,
because tastes may differ. Yet in that case, I knew that tastes may differ
very widely. :-).
Surely, `procmail' successfully addressed difficult problems, and did it
rather solidly, with almost no bugs. So, despite the writing style is rather
alien to mine, it surely had mysterious virtues. I could not condemn it!
> Does anything exist that I can start from?
This has been discussed on 2000-02-07 on this list. In particular:
Donn Cave <do...@u.washington.edu> says:
Anyway, it might be interesting in this context to look
at Sift-Mail, a Tcl application from Lawrence Lundblade:
http://www.island-resort.com/sm.htm and the Sieve language proposed as
an Internet standard for mail filtering, http://www.cyrusoft.com/sieve/
Bernhard Reiter <bre...@usf.Uni-Osnabrueck.DE> says:
Just for completion, there is another newcomer in the mailfilter
competion. Still a greenhorn, but maildrop. Of course, who ever could
get a C++ save. :) http://www.flounder.net/~mrsam/maildrop/README.html
Skip Montanaro <sk...@mojam.com> says:
Worth noting that the procmail tips page at
http://www.procmail.org/jari/pm-tips.html mentions Lua as a possible
replacement for the procmail language (see the end of question 3.5).
> I've even contemplated not inventing yet another filter language, but
> just using plain Python functions.
`procmail' syntax is a bit opaque, but it seems to be very efficient, and
there is a great deal of experience which has accumulated over this syntax.
It might be difficult to raise a challenging competitor. I guess that
one should overview all tools of this kind before tackling such a project.
There is also a lot of experience and solved difficulties in such tools,
portable locking and solid error recovery not being the least.
> If I do this, I'd like to replace fetchmail with getmail as well.
What is `getmail'?
> - writing it as a getmail plug-in
> - writing it as a standalone program to deliver to mailboxes a la
> procmail
> - let getmail deliver to a temporary folder, then using my procmail
> replacement to filter into the right folders, delete spam, etc.
My experience with `procmail' as a SPAM filter told me that it may get
frustratingly slow, when you use hundreds of filtering rules. A few years
ago, I really pondered many alternative approaches, this would be a long
story. In these days, for SPAM and virus filtering, I gave in and wrote
a Python tool which is good for me, but which would not be comprehensive
enough for publication. However, there are also many SPAM filtering systems
around which have been published. You might want to give a comprehensive
look into all these things, before adding yet another one. Maybe! :-)
Gerhard Häring
> [Gerhard Häring]
>
> > I've recently looked into the procmail sources and run away screaming.
>
> ROTFL! If I remember well, the original author knew he had a peculiar
> writing style, and was writing in some prominent file (either an FAQ or
> the README, or something like) that his style was not to be discussed! :-)
> I wondered if the new maintainer revised the sources or not, I did not check.
>
> > I'd very much prefer a Python replacement to such badly written C code.
>
> Is it bad style? I never went as far as asserting that it was bad style,
> because tastes may differ. Yet in that case, I knew that tastes may differ
> very widely. :-).
I prefer Python code which I can hack very easily. Hacking well-written
C code is just much more difficult. Hacking C code that is written in a
style I immediately don't like, is something I'd like to avoid. Btw. I
have no serious reasons to dislike it, it's just gut feeling.
> Surely, `procmail' successfully addressed difficult problems, and did
> it rather solidly, with almost no bugs.
I've heard different opinions, wrt. to losing emails and efficiency. And
I've seen maildrop recommended as a better alternative multiple times.
> > Does anything exist that I can start from?
>
> This has been discussed on 2000-02-07 on this list. In particular:
Many thanks for summarizing all this. Very nice :-)
> > If I do this, I'd like to replace fetchmail with getmail as well.
>
> What is `getmail'?
A fetchmail replacement, written in Python. fetchmail is also rumoured
to be able to lose mail. Looks like this was one of the reasons for
writing getmail.
> My experience with `procmail' as a SPAM filter told me that it may get
> frustratingly slow,
procmail isn't very efficient in my experience. That's also what the
maildrop docs say.
> when you use hundreds of filtering rules.
I have only a few rules, most of them are for putting mailing list
messages in the right folders.
> In these days, for SPAM and virus filtering, I gave in and wrote a
> Python tool which is good for me, but which would not be comprehensive
> enough for publication.
I'm currently just using spamassassin (the best spam detector around,
very configurable, but written in Perl ;-). And a few hardcoded rules
that immediately put messages from known bulkmailing software to
/dev/null. I'm also investingating Pyzor, as I found the Razor author to
be too clueless (also the razor servers aren't open-source).
> However, there are also many SPAM filtering systems
> around which have been published. You might want to give a comprehensive
> look into all these things, before adding yet another one. Maybe! :-)
Yup. I've solved the spam filtering problem with spamassassin. I'm only
looking into a procmail replacement.
Thanks for your answer.
Gerhard
--
mail: gerhard <at> bigfoot <dot> de registered Linux user #64239
web: http://www.cs.fhm.edu/~ifw00065/ OpenPGP public key id AD24C930
public key fingerprint: 3FCC 8700 3012 0A9E B0C9 3667 814B 9CAA AD24 C930
reduce(lambda x,y:x+y,map(lambda x:chr(ord(x)^42),tuple('zS^BED\nX_FOY\x0b')))
Christopher Browne
> [Gerhard Häring]
>
>> I've recently looked into the procmail sources and run away screaming.
>
> ROTFL! If I remember well, the original author knew he had a peculiar
> writing style, and was writing in some prominent file (either an FAQ or
> the README, or something like) that his style was not to be discussed! :-)
> I wondered if the new maintainer revised the sources or not, I did not check.
>
>> I'd very much prefer a Python replacement to such badly written C code.
>
> Is it bad style? I never went as far as asserting that it was bad style,
> because tastes may differ. Yet in that case, I knew that tastes may differ
> very widely. :-).
>
> Surely, `procmail' successfully addressed difficult problems, and did it
> rather solidly, with almost no bugs. So, despite the writing style is rather
> alien to mine, it surely had mysterious virtues. I could not condemn it!
The thing that would be more interesting would be a translator that
would transform procmail _scripts_ into (say) Sieve scripts.
Who cares what the procmail code base looks like if you can take the
sets of rules and transform _them_ into a form that proved more
useful?
--
(concatenate 'string "cbbrowne" "@cbbrowne.com")
http://www.ntlug.org/~cbbrowne/ifilter.html
All generalizations are false, including this one.
Radovan Garabik
: Does such a thing exist? I've recently looked into the procmail sources
<shameless advertising plug>
http://melkor.dnp.fmph.uniba.sk/~garabik/pycmail.html
</shameless advertising plug>
--
-----------------------------------------------------------
| Radovan Garabik http://melkor.dnp.fmph.uniba.sk/~garabik |
| __..--^^^--..__ garabik @ fmph . uniba . sk |
-----------------------------------------------------------
Antivirus alert: file .signature infected by signature virus.
Hi! I'm a signature virus! Copy me into your signature file to help me spread!
Gerhard Häring
> Gerhard Häring <gh_pyt...@gmx.de> wrote:
> : Does such a thing exist? I've recently looked into the procmail sources
>
> <shameless advertising plug>
> http://melkor.dnp.fmph.uniba.sk/~garabik/pycmail.html
> </shameless advertising plug>
Whoa! _Exactly_ what I was searching. Would you mind registering it at
Freshmeat, so other people can find it more easily?
Looks like a good project for my next weekend to convert my
fetchmail/procmail setup to getmail/pycmail.
Gerhard
--
This sig powered by Python!
Außentemperatur in München: 26.6 °C Wind: 1.3 m/s
Radovan Garabik
: * Radovan Garabik <sp...@melkor.dnp.fmph.uniba.sk> [2002-06-19 08:38 +0200]:
:> Gerhard Häring <gh_pyt...@gmx.de> wrote:
:> : Does such a thing exist? I've recently looked into the procmail sources
:>
:> <shameless advertising plug>
:> http://melkor.dnp.fmph.uniba.sk/~garabik/pycmail.html
:> </shameless advertising plug>
: Whoa! _Exactly_ what I was searching. Would you mind registering it at
: Freshmeat, so other people can find it more easily?
http://freshmeat.net/projects/pycmail/
has been there for a long time:-)
: Looks like a good project for my next weekend to convert my
: fetchmail/procmail setup to getmail/pycmail.
I would welcome especially suggestion about possible syntax
changes. Since I cannot help myself, but the syntax (chosen functions
and procedures dealing with mail) does feel a bit clumsy to me
(even though I am the author :-))
Oleg Broytmann
> [Gerhard HДring]
>
> > I've recently looked into the procmail sources and run away screaming.
>
> ROTFL! If I remember well, the original author knew he had a peculiar
> writing style, and was writing in some prominent file (either an FAQ or
> the README, or something like) that his style was not to be discussed! :-)
When I first saw the procmail code, I thought it was generated by some
code obfuscator :)
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ p...@phd.pp.ru
Programmers don't die, they just GOSUB without RETURN.
François Pinard
> > What is `getmail'?
> A fetchmail replacement, written in Python.
Would you kindly give me an URL for `getmail'?
Mark McEahern
I think the top hit here is what you're looking for:
http://www.google.com/search?q=python+getmail
Namely:
http://www.qcc.sk.ca/~charlesc/software/getmail-2.0/
It's one of those rare cases where you can use the google I'm feeling lucky
button:
http://www.google.com/search?q=python+getmail&btnI=I'm%20feeling%20lucky
Cheers,
// m
-
Oleg Broytmann
> http://www.qcc.sk.ca/~charlesc/software/getmail-2.0/
"A POP3 mail retriever..." Hence, it is not fetchmail replacement, as
fetchamil also does IMAP, and does it very well. Fetchmail also has good
GUI configuration program, written in Python and Tkinter.
Gerhard Häring
> On Wed, Jun 19, 2002 at 09:08:53AM -0500, Mark McEahern wrote:
> > http://www.qcc.sk.ca/~charlesc/software/getmail-2.0/
>
> "A POP3 mail retriever..." Hence, it is not fetchmail replacement, as
> fetchamil also does IMAP, and does it very well.
I don't need to retrieve emails from an IMAP server, and the whole point
of the exercise for me is to put them into my IMAP server at home. It
also looks an easy task to enhance getmail to include imap support. Hey,
it's written in Python, not C, where you'd need to reinvent to protocol
handling for the n-th time.
> Fetchmail also has good GUI configuration program, written in Python
> and Tkinter.
Pointless. fetchmail (and getmail even more) configuration is very easy.
If there were a GUI tool to write procmail rules, now that would make
sense. But even then only if it included a regex-wizard ;-)
Michael Hudson
> > Fetchmail also has good GUI configuration program, written in Python
> > and Tkinter.
>
> Pointless. fetchmail (and getmail even more) configuration is very easy.
> If there were a GUI tool to write procmail rules, now that would make
> sense. But even then only if it included a regex-wizard ;-)
What *I* want is a tool that will subscribe you to a mailman mailing
list, write an appropriate procmail rule to shove the messages into a
new mail folder, send the confirmation and tell gnus about the folder.
That would be neat.
Cheers,
M.
--
The Internet is full. Go away.
-- http://www.disobey.com/devilshat/ds011101.htm
François Pinard
> I'm currently just using spamassassin (the best spam detector around,
> very configurable, but written in Perl ;-).
I wanted to peek at Spamassassin, which seems to have a good reputation.
And now, you are trying to discourage me? :-) It's really a strange
phenomenon: more I work with Python, more Perl gets unreadable :-)
> I'm also investingating Pyzor, as I found the Razor author to be too
> clueless (also the razor servers aren't open-source).
Thanks both for pointers and hints.
I'm glad that this discussion popped up `pycmail', yet the warning of its
documentation are a bit chilling...
Skip Montanaro
François> [Gerhard Häring]
>> I'm currently just using spamassassin (the best spam detector around,
>> very configurable, but written in Perl ;-).
François> I wanted to peek at Spamassassin, which seems to have a good
François> reputation. And now, you are trying to discourage me? :-)
François> It's really a strange phenomenon: more I work with Python,
François> more Perl gets unreadable :-)
>> I'm also investingating Pyzor, as I found the Razor author to be too
>> clueless (also the razor servers aren't open-source).
I strongly recommend SA. I just trust Craig Hughes to get the Perl
right. ;-) As for Razor, it seems fine (and I report spam that slips
through SA to Razor), but I don't use it because the network overhead is to
high. (SA can automatically ask Razor for a verdict on a message.)
--
Skip Montanaro
sk...@pobox.com
consulting: http://manatee.mojam.com/~skip/resume.html
Gerhard Häring
> I strongly recommend SA.
ACK. I've also never needed to hack the code (I wouldn't know Perl,
anyway), as it can be configured very nicely via its config file.
> > > [Gerhard Häring]
> >> I'm also investingating Pyzor, as I found the Razor author to be too
> >> clueless (also the razor servers aren't open-source).
That needs some explanation:
http://sf.net/tracker/index.php?func=detail&aid=532357&group_id=3978&atid=103978
He doesn't get mbox parsing, and continues to use his own b0rken
implementation. Having described the bug in a very detailed way, and
even uploaded a test case, and still having it marked as "invalid"
made be /a little/ bitter.
David LeBlanc
forwarding to another mta. Has both allow and deny lists. Uses regex.
A least one of the bundled 3rd party modules (timeoutsocket) is way out of
date, but the software had a recent release-candidate distro.
David LeBlanc
Seattle, WA USA
> -----Original Message-----
> From: python-l...@python.org
> [mailto:python-l...@python.org]On Behalf Of Gerhard Häring
> Sent: Tuesday, June 18, 2002 16:59
> To: pytho...@python.org
> Subject: procmail replacement in Python
>
>
> Does such a thing exist? I've recently looked into the procmail sources
> and run away screaming. I'd very much prefer a Python replacement to
> such badly written C code.
>
> Does anything exist that I can start from? I've even contemplated not
> inventing yet another filter language, but just using plain Python
> functions.
>
> If I do this, I'd like to replace fetchmail with getmail as well.
>
> What would you suggest as approach?
>
> - writing it as a getmail plug-in
> - writing it as a standalone program to deliver to mailboxes a la
> procmail
> - let getmail deliver to a temporary folder, then using my procmail
> replacement to filter into the right folders, delete spam, etc.
>
> The third one looks to be the least likely to lose mail, but the
> temporary folder approach looks like a kludge.
>
> Gerhard
>
> PS: I've seen something _very_ limited written in Python on Sourceforge
> that only does Maildir, but the name is escaping me.
> --
> This sig powered by Python!
> Außentemperatur in München: 23.3 °C Wind: 0.9 m/s
>
>
Oleg Broytmann
> I wanted to peek at Spamassassin, which seems to have a good reputation.
Recently they released version 2.30 which is very buggy. Do not use it,
download previous version, 2.20.
Oleg Broytmann
> > > On Wed, Jun 19, 2002 at 09:08:53AM -0500, Mark McEahern wrote:
> > > > http://www.qcc.sk.ca/~charlesc/software/getmail-2.0/
> > >
> > > "A POP3 mail retriever..." Hence, it is not fetchmail replacement, as
> > > fetchamil also does IMAP, and does it very well.
> >
>
> CMIIamW, I thought Oleg was talking about the description of getmail, not about your particular needs.
Exactly.
Dmitri I GOULIAEV
On Wed, Jun 19, 2002 at 05:04:12PM +0200, Gerhard Haring wrote:
> * Oleg Broytmann <phdOphd.pp.ru> [2002-06-19 18:19 +0400]:
> > On Wed, Jun 19, 2002 at 09:08:53AM -0500, Mark McEahern wrote:
> > > http://www.qcc.sk.ca/~charlesc/software/getmail-2.0/
> >
> > "A POP3 mail retriever..." Hence, it is not fetchmail replacement, as
> > fetchamil also does IMAP, and does it very well.
>
> I don't need to retrieve emails from an IMAP server [...]
CMIIamW, I thought Oleg was talking about the description of getmail, not about your particular needs.
[...]
> Gerhard
Best regards,
--
DIG (Dmitri I GOULIAEV)
Sheila King
Did anyone grab a copy that they can share? I would like to look
at this package. We are often get potential clients at asking about
our web hosting services, who would like to use procmail. However,
procmail doesn't run well on our system due to the Qmail mail permissions
and such. It would be interesting to see if this python procmail
replacement might do the job?
--
Sheila King
http://www.thinkspot.net/sheila/
http://www.k12groups.org/
On Wed, 19 Jun 2002 11:52:28 +0200, Radovan Garabik
<sp...@melkor.dnp.fmph.uniba.sk> wrote in comp.lang.python in article
<s8kpea...@127.0.0.1>:
> Gerhard H?ring <ger...@bigfoot.de> wrote:
> : * Radovan Garabik <sp...@melkor.dnp.fmph.uniba.sk> [2002-06-19 08:38 +0200]:
François Pinard
> Here is a copy of my copy:
Oops, sorry!
I should not have Cc:ed python-list with the full archive. On the other
hand, as it is rather small, I think I will nevertheless sleep tonight :-).
In a word, I apologise, and will try paying more attention in the future.
--
François Pinard http://www.iro.umontreal.ca/~pinard
Sheila King
wrote in comp.lang.python in article
> [François Pinard]
>
> > Here is a copy of my copy:
>
> Oops, sorry!
>
> I should not have Cc:ed python-list with the full archive. On the other
> hand, as it is rather small, I think I will nevertheless sleep tonight :-).
> In a word, I apologise, and will try paying more attention in the future.
Francois,
Thanks or the copy I received in email. However, that tarball that you sent
seems to be incomplete and doesn't have any .py source files. Seems mostly
to have .pycmailrc and other types of config files, .doc type files and
install files.
The README files make it look very intriguing. I'd sure like to get my
hands on the Python source.
François Pinard
> Thanks or the copy I received in email. However, that tarball that
> you sent seems to be incomplete and doesn't have any .py source files.
> Seems mostly to have .pycmailrc and other types of config files, .doc type
> files and install files. The README files make it look very intriguing.
> I'd sure like to get my hands on the Python source.
I glanced through `pycmail' and decided to postpone for now, so I do not
have much experience to share with you.
The Python source is file `pycmail' at the top-level after extraction.
It does not have a `.py' extension.
For my things, I usually compile a `.py' at installation time and `import'
it through a very small bootstrap, to avoid re-compilation at each use.
Yet, I sometimes wonder if the extra disk accesses are really worth the
saving in compilation times. Maybe `pycmail' does it the efficient way?
David LeBlanc
there. You can add the .py extension for use under windows or just "python
name" will work. I think only the pycmail file is python: i.e. it's all in
that one file.
You should note that this has a dependency on Unix features.
David LeBlanc
Seattle, WA USA
Sheila King
wrote in comp.lang.python in article
> I glanced through `pycmail' and decided to postpone for now, so I do not
> have much experience to share with you.
>
> The Python source is file `pycmail' at the top-level after extraction.
> It does not have a `.py' extension.
Thank you.
I feel totally silly now. I must've looked at almost every file in the
package except that one. :/
Anyhow, thank you, I've looked at it now. It looks very interesting and is
something I may work with further when I get more time. Hope that the
author gets his DNS situation sorted out.
Radovan Garabik
: The tarball URL for the pycmail packages is not working for me.
there were just some power outages, it should work now
: Did anyone grab a copy that they can share? I would like to look
: at this package. We are often get potential clients at asking about
: our web hosting services, who would like to use procmail. However,
: procmail doesn't run well on our system due to the Qmail mail permissions
: and such. It would be interesting to see if this python procmail
: replacement might do the job?
it depends
pycmail just expects the incoming mail at stdin, and copies it to
various destinations, so under standard scheme it is invoked with
UID of the user expecting mail, and if it has proper rights to write it
into destination mailboxes, all should work.
(what kind of problems are you having with procmail?)