filtering 'userid' to 'userid+folder'
David Lee
Executive summary: We'd like to be able to adjust a delivery of simple
"toaddr" to "toaddr+spamfolder" if a certain header is present.
Detail:
We are happily running MailScanner (MS): this converts the normal simple
sendmail
into a sequence:
sendmail | MailScanner | sendmail
When MS encounters dubious email it inserts a (configurable) extra header,
example:
X-MailScanner-SpamCheck: ORDB-URL
(Incidentally, we are thinking of augmenting this with SpamAssassin.)
We are hoping to configure sendmail to detect this extra header, and
change any simple local delivery, say, 'userid' into something like
'userid+spamfolder'. (The final delivery is handled by Washington's
lightweight 'tmail' which can handle this syntax, and integrates well with
the IMAP and POP daemons. We'd rather not break this integration by
moving to the more heavyweight alternatives such as "procmail" for
delivery.)
But I'm stuck in my testing.
I have constructed some ".cf" things (which would ultimately go into
a "mc", of course):
HX-MailScanner-SpamCheck: $>CheckSpam
SCheckSpam
R <...set a marker that this is spam...>
and adjusted "Slocaladdr=5" (actually "SLocal_localaddr") to detect this.
Individually all these rulesets work when tested (-bt).
But the sequence in which sendmail calls these means that the processing
of the envelope delivery (culminateing in ruleset 5) is entirely finished
before the "CheckSpam" ruleset is called. I was doing a test delivery:
/usr/lib/sendmail -v -d21.2 -ffromaddr toaddr < sample-msg
and the references to "toaddr" finished before the first appearance of
"CheckSpam".
Any thoughts on getting a header to adjust an envelope delivery address?
--
: David Lee I.T. Service :
: Systems Programmer Computer Centre :
: University of Durham :
: http://www.dur.ac.uk/t.d.lee/ South Road :
: Durham :
: Phone: +44 191 374 2882 U.K. :
Andrzej Filip
Have you considered:
* storing "header detected" info in persistent macro
* making local mailer "expensive" (no instant delivery attempt) with separate
queue group for local mailer and persistent queue runners (requires sendmail
8.12)
OR use "sendmail -bg -q5m -qdq -O MinQueueAge=30m" for pre 8.12
--
Andrzej [pl>en: Andrew] Adam Filip http://www.polbox.com/a/anfi/
I may disagree with the following *random epigram* :
Worst Month of 1981 for Downhill Skiing:
August. The lift lines are the shortest, though.
-- Steve Rubenstein
David Lee
> David Lee wrote:
>
> > Currently using sendmail-8.11.1, but am also testing with 8.12.6 .
> >
> > Executive summary: We'd like to be able to adjust a delivery of simple
> > "toaddr" to "toaddr+spamfolder" if a certain header is present.
> >
> > [...]
> > But the sequence in which sendmail calls these means that the processing
> > of the envelope delivery (culminating in ruleset 5) is entirely finished
> > before the "CheckSpam" ruleset is called. I was doing a test delivery:
> > /usr/lib/sendmail -v -d21.2 -ffromaddr toaddr < sample-msg
> >
> > and the references to "toaddr" finished before the first appearance of
> > "CheckSpam".
> >
> > Any thoughts on getting a header to adjust an envelope delivery address?
>
> Have you considered:
> * storing "header detected" info in persistent macro
> * making local mailer "expensive" (no instant delivery attempt) with separate
> queue group for local mailer and persistent queue runners (requires sendmail
> 8.12)
>
> OR use "sendmail -bg -q5m -qdq -O MinQueueAge=30m" for pre 8.12
Thanks for the reply, Andrzej. (By the way, before posting my request, I
had visited your WWW site to see if it gave any clues to this issue.)
I don't see how any of those options actually help with the message
actually being processed.
1. We are already running: "-bd -q2m -odq" to smooth out the delivery
process.
2. I suspect your "-bg -q5m -qdq -O MinQueueAge=30m" contains at least
one typo ("-bg"->"-bd"?) and perhaps another ("-qdq->"-odq"?).
A group of message sitting in /var/spool/mqueue. A sendmail instance
processes them sequentially. Although I can use a macro to detect the
header, the problem is that all the delivery-envelope processing is done
and finished by the time the headers are processed: it is too late for
that message. That is what I was attempting with my tests:
/usr/lib/sendmail -v -d21.2 -ffromaddr toaddr < sample-msg
(which, in real life, would become "-bd -q2m -odq" or similar).
So not only will the persistent "ThisIsSpam" macro fail to be applied to
this message, it will (worse) be applied to the next (probably innocent)
message in the queue, won't it? I'm happy to be demonstrated wrong on any
or all of this.
(We use Washington "tmail" for final, Mlocal, delivery, so all mailbox
access is through the common Washington imap/pop/tmail suite. I'm trying
to avoid, if reasonably possible, breaking this consistent suite.)
Andrzej Filip
My "typing" mistake you have detected so easily.
> A group of message sitting in /var/spool/mqueue. A sendmail instance
> processes them sequentially. Although I can use a macro to detect the
> header, the problem is that all the delivery-envelope processing is done
> and finished by the time the headers are processed: it is too late for
> that message. That is what I was attempting with my tests:
> /usr/lib/sendmail -v -d21.2 -ffromaddr toaddr < sample-msg
>
> (which, in real life, would become "-bd -q2m -odq" or similar).
>
> So not only will the persistent "ThisIsSpam" macro fail to be applied to
> this message, it will (worse) be applied to the next (probably innocent)
> message in the queue, won't it? I'm happy to be demonstrated wrong on any
> or all of this.
>
> (We use Washington "tmail" for final, Mlocal, delivery, so all mailbox
> access is through the common Washington imap/pop/tmail suite. I'm trying
> to avoid, if reasonably possible, breaking this consistent suite.)
Have you moved info about detected "bad" header into persistent macro ?
Can you post *.mc lines you have created to implement the trick ?
--
Andrzej [pl>en: Andrew] Adam Filip http://www.polbox.com/a/anfi/
I may disagree with the following *random epigram* :
When a shepherd goes to kill a wolf, and takes his dog along to see
the sport, he should take care to avoid mistakes. The dog has certain
relationships to the wolf the shepherd may have forgotten.
-- Robert Pirsig, "Zen and the Art of Motorcycle Maintenance"
David Lee
> David Lee wrote:
> [...]
> > What I was attempting with my tests:
> > /usr/lib/sendmail -v -d21.2 -ffromaddr toaddr < sample-msg
> [...]
> Can you post *.mc lines you have created to implement the trick ?
Many thanks for the offer. Here, between the "=== snip ===", is the ".mc"
file. The attempted added functionality is entirely in the LOCAL_RULESETS
at the end.
Main environment is 8.11.1, but seems no different under 8.12.6.
Test command:
/usr/lib/sendmail -v -d21.2 -ffromaddr toaddr < sample-msg
where file "sample-msg" contains a test message include a header
X-MailScanner-SpamCheck: anything
All references to "toaddr" are completed before "check_eoh" is invoked,
so the existence of the header cannot be detected in ruleset 5
(localaddr and its Local_localaddr subroutine).
=================== snip =====================
divert(-1)
#
# Configuration for departmental server machines.
#
divert(0)dnl
VERSIONID(`@(#)mailspool.mc 1.7 10/02/02')
OSTYPE(solaris2)
define(`TBLDIR',`/etc/mail')dnl
dnl
ifdef(`CF_LEVEL',dnl
dnl At 8.10.0, `nouucp' began to require an argument.
dnl Fortuntely, that release also introduce `CF_LEVEL'.
`FEATURE(nouucp,`reject')',dnl
`FEATURE(nouucp)'dnl
) dnl
dnl
dnl Any re-signature should also apply to envelope.
dnl
FEATURE(masquerade_envelope)
dnl
dnl Our domains (for incoming e-mail)
define(`confCW_FILE', `TBLDIR/sendmail.localhosts')
FEATURE(use_cw_file)
dnl
dnl NIS group lookup is expensive and unnecessary. Turn off.
define(`confDONT_INIT_GROUPS', `True')
dnl
dnl Outgoing: What and how we re-sign
dnl
MASQUERADE_DOMAIN_FILE(TBLDIR/sendmail.localhosts)
MASQUERADE_AS(`$j')
dnl
dnl Outgoing: How we deliver
dnl
define(`SMART_HOST', `hermes.dur.ac.uk')
dnl
dnl Set blank substitution to blank.
dnl Seems to default to . which wrecks filters.
dnl
define(`confBLANK_SUB', ` ')
dnl
dnl Restrict snoopers
define(`confPRIVACY_FLAGS',``authwarnings,noexpn,novrfy'')
dnl
dnl For `local' use c-client (tmail from University of Washington IMAP/POP)
include(`c-client.m4')dnl
dnl
MAILER(local)
MAILER(smtp)
dnl
dnl Only accept from campus mailhubs
dnl
dnl include(`check_relay_hubs.m4')dnl
dnl include(`check_start.m4')dnl
dnl
dnl Anti-spam delivery to "userid+spamfilter"
LOCAL_RULESETS
############################################################################
# TDL testing spam-filter October 2002
############################################################################
Kstorage macro
############################################################################
# Spam redirection to folder
#
# Principle:
# If email is spam, change delivery 'userid' into 'userid+spamfilter'.
#
# Implementation:
# Another agency (e.g. MailScanner, SpamAssassin) inserts an extra header.
# This acts as our trigger.
#
# Ruleset 'CheckSpam' records this trigger in {SpamHeader}.
#
# At end of headers, 'check_eoh' {SpamHeader} is copied to {IsSpam}.
# Also {SpamHeader} is then cleared (in case this sendmail run is
# processing multiple messages).
#
# If the destination address is a single-token 'userid', and if {IsSpam}
# is set, then adjust the address to 'userid+spamfilter'.
#
############################################################################
D{SpamFolder}spamfilter
HX-MailScanner-SpamCheck: $>CheckSpam
SCheckSpam
# Record the presence of the header for this message.
R$* $: $(storage {SpamHeader} $@ $1 $) $1
Scheck_eoh
# Called once per message, when headers have been read.
# Set {IsSpam} for this message; clear {SpamHeader} for possible future message.
R$* $: $(storage {IsSpam} $) $1 clear residue from previous msgs
R$* $: < $&{SpamHeader} > $1 spam in this msg?
R< $+ > $* $: < $(storage {IsSpam} $@ $1 $) > $2 if set, copy
R< $* > $* $: $(storage {SpamHeader} $) $2 clear per-msg
SLocal_localaddr
# If address is simple (single token, $-), and message is spam, then redirect.
R$- $: $1 < $&{IsSpam} >
R$+ < $+ > $#local $@ $2 $: $1 + ${SpamFolder}
R$+ < > $: $1
############################################################################
=================== snip =====================