Milter rulesets and envrcpt ordering
Joe Maimon
Pardon for posting from google, I dont have a news account.
I have been testing some rough patches to add the ability to specify
rulesets for milter commands to be evaluated before sending the
command to the milter.
With the patch one can add to the end of the milter like this
INPUT_MAIL_FILTER(`verbose-milter',
`S=local:/home/joe/tmp/verbose-milter.sock, F=,
T=C:30s;S:1m;R:1m;E:1m,R=C:ConnectRuleSet;M:MailRuleSet;R:RcptRuleSet;R:spamFriend')
I am attempting this rather than patching milters in the manner done
here
https://savannah.nongnu.org/patch/?func=detailitem&item_id=3372
This patch seems to work well.
I have been also working on a patch in the vein of this post from
David Skoll
Its pretty rough but it also seems to work.
I intend to further work on a patch to allow a milter to call into
sendmail for evaluation of an arbitrary ruleset.
I will be happy to provide patches to anybody who wishes to review
them.
Any and all feedback here or to my email is greatly appreciated.
Joe Maimon
Claus Aßmann
> I have been testing some rough patches to add the ability to specify
> rulesets for milter commands to be evaluated before sending the
> command to the milter.
Can you please explain what the purpose is? Which problem(s) are
you trying to solve?
--
A: Maybe because some people are too annoyed by top-posting.
Q: Why do I not get an answer to my question(s)?
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
Joe Maimon
> Joe Maimon wrote:
>
> > I have been testing some rough patches to add the ability to specify
> > rulesets for milter commands to be evaluated before sending the
> > command to the milter.
>
> Can you please explain what the purpose is? Which problem(s) are
> you trying to solve?
Thanks for your response!
My immediate problem is that I would like to cut out spam/avirus
cpu/ram cycle usage for people who have been configured in sendmail to
not get RBL processing. My opinion is that they simply dont deserve
it. My observation is that RBL's will cut down most of the junk in the
first place.
I also run a private RBL (spamikaze) that I feed spam and avirus
notifications and this should be my first line of defense.
With theses patches I can now let sendmail worry about whether the
milters should be invoked for certain recpients -- I use a ruleset
like this one
(convert appropriate spaces to tabs)
SspamFriend
dnl R$* $: $1 $| $&{rcpt_mailer}
dnl R$* $| error $#discard
dnl R$* $| $* $: $1
R$* $: <> $>canonify $1
R<> $* < @ $+ . > $* <> $1 < @ $2 > $3
R<> $+ < @ $* > $* $: <L> $1 < @ $2 > $3 $| <F: $1@$2 >
<D: $2 > <U: $1@>
R<> $* $: <L> $1 $| <U: $1@>
R<> $* $@ $1
R<L> $* $| $* $: <L> $1 $| $>SearchList <! Spam> $|
$2 <>
R<L> $+ < @ $+ > $| <?> $@ $1@$2
R<L> $* $| <FRIEND> $#discard
R<L> $+ < @ $+ > $| $* $@ $1@$2
(the first three lines let me filter out recipients who are
undeliverable, see envrcpt ordering patch)
(I personally think it may be nice to let milters indicate whether
they wish to recieve undeliverable rcpts' or not)
This ruleset and patch will not send to milters any rcpts who are
marked in access db as Spam:FRIEND
Should the milter not get ANY recipients, milter_abort is called.
This is convenient because now I can control ALL my relevant milters
without patching/configuring all of them in one place -- my
sendmail.mc file. This includes spamass-milter (which I patched),
clamav-milter, amavis-milter. Furthermore, any of those folks who may
have binary only milters --- this is the only way to go.
Most of these milters have a fair bit of code put in to the milter to
decide when to cut and run. (man clamav-milter, see the -o and -l
flags). This feels like a duplication of effort.
I have not run any tests but aside from and more important to me than
an expected performance gain, my goal here is a centralization of
policy.
The patch I wrote about is a bit more general than the above case. I
tried to have it be able to handle any conceivable milter command one
would wish to filter -- the connect, the ehlo/helo, mail from, rcpt
to:, headers.....
The patch also supports multiple rulesets for each milter command.
Currently rejected from one is rejected from all.
Additionaly, my personal opinion is that the more things in sendmail
that are ruleset controllable -- the better.
Yours,
Joe M
Claus Aßmann
> My immediate problem is that I would like to cut out spam/avirus
> cpu/ram cycle usage for people who have been configured in sendmail to
> not get RBL processing. My opinion is that they simply dont deserve
> it. My observation is that RBL's will cut down most of the junk in the
> first place.
Do you have performance test data that shows how much faster
the MTA runs with these patches?
> With theses patches I can now let sendmail worry about whether the
> milters should be invoked for certain recpients -- I use a ruleset
What happens if the client sends mail to two recipients, one of
which wants filtering and the other doesn't? Your patch will not
"tell" milter about the second one, right? Now the milter "knows"
only about one recipient and then may request a data rejection
(after scanning the body), which means that neither recipient
receives the mail, even though one of them might have wanted it.
It might make sense to avoid sending data to a milter if you can
make this decision at connection time, i.e., before a milter is
invoked. Later on it just can cause "confusion" and maybe errors.
BTW: I'm just trying to define this behavior for sendmail X but I
haven't found a good solution yet. Here's a small part of the long
discussion:
Anti-Spam Checks: Functionality for first Release
For each step an anti-spam function returns up to three values:
1. the lookup results (SUCCESS (FOUND), NOTFOUND, TEMPFAIL, PERMFAIL). If
the result is TEMPFAIL or PERMFAIL then the system should have a
default reply code (which could be different for each test). For now
TEMPFAIL and PERMFAIL are ignored, i.e., the default reply code is
CONTINUE.
2. if the result was SUCCESS, then it returns a reply code
RELAY 100 Allow relaying
OK 250 Accept command, no further tests in this stage
CONTINUE 300 Run further tests
SSD 421 Service shutting down
TEMP 450 Temporary error
REJECT 550 Permanent error
This code is modified by adding a constant to it if the RHS string
contained QUICK[*].
3. if the result was SUCCESS an optional string can be returned which
contains the SMTP reply code, a DSN code, and an error text.
Here is a proposal for the basic anti-spam algorithm:
* connect: client host name, IP address:
+ RELAY: allow relaying, store this information in the session
state; override other checks for connect, e.g., potential DNS BL
lookups.
+ QUICK:RELAY: allow relaying, store this information in the session
state; override all other checks.
+ OK: override other checks for connect, e.g., potential DNS BL
lookups.
+ QUICK:OK: override all other checks except for relay tests.
+ QUICK:REJECT: reject session.
+ REJECT: reject session unless delay-checks is set.
+ TEMP: tempfail session unless delay-checks is set.
+ CONTINUE: continue with other checks.
Any other result is stored in the session state, i.e., the triple
(result, RHS, text).
* MAIL: lookup mail sender address with tag From:
+ OK: override other checks for MAIL.
+ QUICK:REJECT: reject transaction.
+ REJECT: reject transaction unless delay-checks is set.
+ TEMP: tempfail transaction unless delay-checks is set.
+ CONTINUE: continue with other checks.
Any other result is stored in the transaction state.
* RCPT: recipient address: first perform lookups with the tag To:
+ RELAY: accept recipient, run other tests if feature delay-checks
is turned on.
+ QUICK:RELAY: accept recipient, don't run any other tests, even if
feature delay-checks is turned on.
+ OK: override other checks for RCPT.
+ QUICK:OK: override other checks except for relay tests.
+ REJECT: reject recipient.
+ TEMP: tempfail recipient.
+ CONTINUE: continue with other checks.
If the lookup returns neither REJECT nor RELAY then the usual
anti-relay tests are performed.
Interaction between Anti-Spam Checks and Milter
Question: should OK from access map disable policy milter checks?
That would be ``confusing'' for a milter that wants that data, even
though it would match the REJECT behavior because milter is not
invoked in that case. However, this is not really a symmetric
situation: REJECT means that the command is not accepted and hence
in almost all cases it does not matter what a milter would do about
it. for OK the milter might want to do something. However, that
would be a contradiction (a misconfiguration), i.e., one part of
the configuration explicitly accepts the command another rejects
it. The right return value in this case is CONTINUE instead of OK.
In order for the milter to keep track of the SMTP dialog (and hence
the current state) it might be possible to send the information to
the milter including the fact that the command has been accepted
and hence cannot be rejected by the milter (``for your information
only'').
Note: the interaction between anti-spam checks and milter is even more
complicated if feature delay-checks is turned on. In sendmail 8 this was a
simple hack that only influenced the rulesets (by renaming the connect and
mail check and calling them from the rcpt ruleset). In sendmail X this is
solved in a different way: the checks are run at the appropriate stage in
the SMTP dialogue, but the results may be delayed. Hence a connection
might be rejected unless a recipient test overrides it (``spam friend'').
Question: How should this interact with milter tests? Should they just run
``normally'' as the other tests and their results are delayed?
David F. Skoll
> What happens if the client sends mail to two recipients, one of
> which wants filtering and the other doesn't?
There's no nice way to solve this problem without changing SMTP.
What my milter does, if it notices groups of recipients with
different policies, is re-mail separate messages to each group
(grouped by policy) and then discard the original. The re-mailed
messages can then be filtered according to policy for each group.
This solution has some negative aspects, of course, but it seems to
work fairly well.
The correct way to do it would be to modify SMTP so that
after the end of DATA, a "COPY" command could tell the system
to send mail to additional recipients. For example:
S: 220 ...
C: HELO foo
S: 250 ...
C: MAIL FROM:<f...@origin.net>
S: 250 ...
C: RCPT TO:<b...@destination.org>
S: 252 ...
C: DATA
S: 354 ...
C: mail body
C: .
S: 250 ...
C: COPY TO:<anoth...@destination.org>
S: 554 Prohibited Content
C: COPY TO:<thir...@destination.org>
S: 250 ...
[anti-spam algorithm removed]
In my experience, it's extremely difficult to come up with an algorithm
that's flexible enough to meet everyone's needs without making things
complex for people who don't need weird tests. I would keep the Sendmail
algorithm as simple as possible, and tell people who want more complexity
to push that complexity into a milter.
> Question: should OK from access map disable policy milter checks?
Nope.
> That would be ``confusing'' for a milter that wants that data, even
> though it would match the REJECT behavior because milter is not
> invoked in that case. However, this is not really a symmetric
> situation: REJECT means that the command is not accepted and hence
> in almost all cases it does not matter what a milter would do about
> it.
That's OK. Generally speaking, REJECTions are "ORed" -- if anything
REJECTs mail, then it's rejected. OKs are "ANDed" -- everyone has to
say OK before mail is accepted.
> Question: How should this interact with milter tests? Should they just run
> ``normally'' as the other tests and their results are delayed?
I don't think putting hairy logic into Sendmail is necessary, because
it's usually far easier to put that logic in the milter itself. I think
if a milter rejects a message, it should be rejected regardless of Sendmail
rule sets.
Regards,
David.
Andrzej Adam Filip
> Claus Aßmann wrote:
>>What happens if the client sends mail to two recipients, one of
>>which wants filtering and the other doesn't?
>
> There's no nice way to solve this problem without changing SMTP.
Another option would be to set up "IP based virtual MTAs" but it would require
many changes in sendmail.cf.
--
Andrzej [en:Andrew] Adam Filip an...@priv.onet.pl an...@xl.wp.pl
Home Page http://anfi.homeunix.net/ [ PageRank 6 ]
*Random Epigram* :
Great trees keep little ones down.
-- American Proverb
Joe Maimon
> Joe Maimon wrote:
>
> > My immediate problem is that I would like to cut out spam/avirus
> > cpu/ram cycle usage for people who have been configured in sendmail to
> > not get RBL processing. My opinion is that they simply dont deserve
> > it. My observation is that RBL's will cut down most of the junk in the
> > first place.
>
> Do you have performance test data that shows how much faster
> the MTA runs with these patches?
Not at this time.
Any possible perf savings are highly dependant on the milters and
system in question.
My performance benefit is in not needing a patched spamass-milter that
forks a shell script that calls editmap at least half a dozen times
for each recipient. I will attempt to measure that after I figure out
how. Pointers would help.
Its hard to imagine any practical internal sendmail ruleset processing
that would require more processing than the above approach I took with
the spamass-milter.
But what I really wanted to avoid was also needing to patch
clamav-milter and amavis-milter and so on and so forth.
There is no doubt in my mind that the full clamav and spamassassin and
amavis+uvscan+clamscan processing is many orders more demanding than
internal ruleset processing. I can see that using top. Whats debatable
performance wise is how much savings one gets from this patch versus
patching all the relevant milters on the system.
The other benefits include centralized policy by sharing rulesets
amongst multiple milters and having an easy way to disable a milter
for many but not all cases.
There is also no doubt that milters are having to embed varying
amounts of intelligence (with varying degrees of trouble free
operation) into themselves to deal with common concerns such as
-- avoiding miltering locally generated email
-- avoiding miltering/rejecting for recipients who dont wish it
-- determining the above
-- avoiding miltering/rejecting for email sent from trusted network
sources
-- determining the above
-- avoiding miltering/rejecting for email sent from "trusted" email
addresses
Perhaps everybody would be better off if most common cases could be
handled by sendmail with a couple of rulesets?
>
> > With theses patches I can now let sendmail worry about whether the
> > milters should be invoked for certain recpients -- I use a ruleset
>
> What happens if the client sends mail to two recipients, one of
> which wants filtering and the other doesn't? Your patch will not
> "tell" milter about the second one, right? Now the milter "knows"
> only about one recipient and then may request a data rejection
> (after scanning the body), which means that neither recipient
> receives the mail, even though one of them might have wanted it.
>
Very fair question.
Short Answer:
Most milters I have seen do not handle this at all (well) anyways.
This case is potentialy undesirable but manageable by returning a
$#abort from the relevant ruleset. Otherwise Admin Beware. However,
this is essentialy no different than the current status quo with most
milters I have seen.
Very Long Answer:
(in the following long winded discussion when discussing the patch in
question *theoretically* one can substitute header for recipient)
(relevant URLS
http://lists.gnu.org/archive/html/spamass-milt-list/2004-09/msg00018.html
http://lists.gnu.org/archive/html/spamass-milt-list/2004-09/msg00027.html
)
With the patch I currently use on spamass-milter, the milter will not
return a reject if *ANY* rcpt filtering happened. I believe this is by
no means constantly desirable behavior, as undesirables may use this
behavior for undesirable results. Obscure references intentional.
Resulting in a net loss for the ones who wish they were being miltered
apropriately. Tyranny of the minority?
It should be a valid local policy decision to reject email to a
recipient due to the accompanying recipient whom we have decided does
not want the message at any cost. The presence of the picky recipient
may be considered an error with the entire message.
This is a decision of whether a recipient's supposed desire to not
receive the message outweighs a recipient's desire to receive the
message. When and if 90+% incoming is junk odds are that there will be
plenty more who will be saying "reject for all because of one(most)"
than those that say "accept for all because of one".
With this patch, one can have the ruleset return a $#abort which will
guarantee no rejection will occur, since the milter will be aborted
for the current message or closed for the connection.
Without this, the local policy is, in effect, that any email
objectionable to any recipient to cause rejection will be rejected for
all. Which unless a milter takes explicit steps to avoid is my
generaly observed behavior anyways.
If a milter has the brains to determine what to do based on recipients
who should prevent any potential rejection, you should not be
filtering that out from the milter. If the milter does not treat
recipients differently than you wont be making the situation any worse
and likely much better.
I have seen ways of doing this in both clamav-milter (internal
whitelist) and spamass-milter (user spamassassin preferences) that
seemed far from perfect. In any event the impact of the "internal
filtering" on any potential rejection were not obvious to me after
reading source. This is apparently reinventing the wheel in each
milter.
This is an opportunity to add inteligence to milter handling. If
someone is not carefull and does not know enough about their setup,
they may detract from the available intelligence the milters have.
Admin Beware. Dont filter milters who do a better job on their own.
Perhaps a flag that will instruct sendmail to ignore any rejection
action from the milter? This looks like swampland...opinions welcome.
> It might make sense to avoid sending data to a milter if you can
> make this decision at connection time, i.e., before a milter is
> invoked. Later on it just can cause "confusion" and maybe errors.
>
The patch supports configuring connection ruleset that will do that.
Real easy way to avoid miltering locally generated email.
I had thought to add some kind of "order of evaluation" whereby I
could delay the evaluation of certain rulesets until certain commands
were received.
For instance, evaluate the connection, mail from, rcpt rulesets after
EOM was received. I imagine that would entail storing the relevant
data with the associated rulesets for the lifetime of the
message/connection.
Should anyone feel that this is worthwhile, any moral support to
continue looking at this direction is more than welcome.
> BTW: I'm just trying to define this behavior for sendmail X but I
> haven't found a good solution yet. Here's a small part of the long
> discussion:
>
>
This discussion is huge. I will see if I can reply to that after I
digested it and my head stops hurting.
I appreciate very much yours and anyones feedback.
Yours,
Joe
P.S Any suggestions as to the feasibilty/practicality of actualy
replacing the data sent to the milter with the rewritten results?(some
milters would be better off not having to concern themselves with say,
brackets or none around the address...)
David F. Skoll
> David F. Skoll wrote:
>> Claus Aßmann wrote:
>>>What happens if the client sends mail to two recipients, one of
>>>which wants filtering and the other doesn't?
>> There's no nice way to solve this problem without changing SMTP.
>> [...]
> Another option would be to set up "IP based virtual MTAs" but it would
> require many changes in sendmail.cf.
That works for different domains, but not for "us...@domain.net" vs
"us...@domain.net".
--
David.
David F. Skoll
> But what I really wanted to avoid was also needing to patch
> clamav-milter and amavis-milter and so on and so forth.
Did you try MIMEDefang (http://www.mimedefang.org) ?
It integrates with Clam and Spamassassin (and uvscan, for that
matter), and lets you write your policy in Perl. That way, you only
need one milter, and you have a flexible language for expressing
policy.
> There is also no doubt that milters are having to embed varying
> amounts of intelligence (with varying degrees of trouble free
> operation) into themselves to deal with common concerns such as...
I concluded that too, which led to the design of MIMEDefang: It's
completely programmable in Perl.
--
David.
Joe Maimon
> Joe Maimon wrote:
>
> > But what I really wanted to avoid was also needing to patch
> > clamav-milter and amavis-milter and so on and so forth.
>
> Did you try MIMEDefang (http://www.mimedefang.org) ?
I have not ever used MIMEDefang. As you describe it, it would give me
a way to control the tasks that I currently use other milters for.
I had approached the setup originally with the attitude of the right
milter for each task.
Why is there such a duplication of milter work? There are at least
five milters that integrate with spamassassin - spamass-milt, the
amavisd-new project's amavisd-milter, http://www.runestig.com/osp.html
, milter-spamc and MIMEDefang.
Anti-Virus has clamav-milter, amavisd-milter, amavisd-new, MIMEDefang
just to name a few.
I am sure I missed some.
There are more milters for these things than there are popular open
source MTA's.
Part of that is probably the ease in writing a simple milter. Take the
example milter, add a few dozen lines of C and you have a milter that
can do *something*. Seems like each milter starts small and then
starts accumulating intelligence as its lack is felt.
>
> It integrates with Clam and Spamassassin (and uvscan, for that
> matter), and lets you write your policy in Perl.
Thanks for the suggestion and I should give that a try as soon as I
emerge from this sendmail haze I am currently in. MIMEDefang has an
excellent reputation from what I hear.
The ability to write policy in Perl should be a good reason to not
filter MIMEDefang's input, however not all milters are as blessed.
Personally I currently feel more comfortable writing sendmail policy
in sendmail rulesets than perl in the milter. One of the reasons I
feel this way is because Sendmail rulesets get to access quite easily
a lot of sendmail facilities and information (all the maps, all the
macros). I expect many others dont feel that way.
How would you like to be able to call ruleset rewriting from a milter?
That would give you the best of both.
Interesting that there is such a need for a milter to control all the
tasks that "less capable" milters already exist to do.
If anybody thinks the current direction I am in has any future, I
would love to hear it. As it is I plan on taking it as far as I can
handle.
Joe
Jeff Rife
> Part of that is probably the ease in writing a simple milter. Take the
> example milter, add a few dozen lines of C and you have a milter that
> can do *something*. Seems like each milter starts small and then
> starts accumulating intelligence as its lack is felt.
Having experience with MIMEDefang, this is where it is very different.
The milter itself does very little other than to manage the pool of
multiple Perl jobs and pass them the data from sendmail. All the policy
is done with Perl, thus "accumlating intelligence" is easy for *anybody*
to do. I'm sure David will correct me if I am wrong, but this seems to
be the whole point of MIMEDefang.
I'm a Perl beginner in many ways, but I managed to add forged HELO
blocking to my MIMEDefang Perl filter. Adding another milter to do
something like this now seems to be a dumb way to do things.
> How would you like to be able to call ruleset rewriting from a milter?
> That would give you the best of both.
Why bother? Although the syntax is obviously different, you can do the same
sort of re-writing that sendmail does from within the Perl process that
MIMEDefang runs.
--
Jeff Rife | "Having your book turned into a movie is like
SPAM bait: | seeing your ox turned into bouillon cubes."
Ask...@usdoj.gov |
sp...@ftc.gov | -- John Le Carré
Joe Maimon
<snipped>
> BTW: I'm just trying to define this behavior for sendmail X but I
> haven't found a good solution yet. Here's a small part of the long
> discussion:
>
> Anti-Spam Checks: Functionality for first Release
>
> For each step an anti-spam function returns up to three values:
>
> 1. the lookup results (SUCCESS (FOUND), NOTFOUND, TEMPFAIL, PERMFAIL). If
> the result is TEMPFAIL or PERMFAIL then the system should have a
> default reply code (which could be different for each test). For now
> TEMPFAIL and PERMFAIL are ignored, i.e., the default reply code is
> CONTINUE.
>
> 2. if the result was SUCCESS, then it returns a reply code
>
> RELAY 100 Allow relaying
> OK 250 Accept command, no further tests in this stage
> CONTINUE 300 Run further tests
> SSD 421 Service shutting down
> TEMP 450 Temporary error
> REJECT 550 Permanent error
>
> This code is modified by adding a constant to it if the RHS string
> contained QUICK[*].
>
> 3. if the result was SUCCESS an optional string can be returned which
> contains the SMTP reply code, a DSN code, and an error text.
>
Presumably this is so that a later "summing" can review each steps
decisions and come up with an answer that fits local policy.
> Here is a proposal for the basic anti-spam algorithm:
>
> * connect: client host name, IP address:
> + RELAY: allow relaying, store this information in the session
> state; override other checks for connect, e.g., potential DNS BL
> lookups.
Unless I am mistaken this is current behavior
> + QUICK:RELAY: allow relaying, store this information in the session
> state; override all other checks.
New
> + OK: override other checks for connect, e.g., potential DNS BL
> lookups.
Current
> + QUICK:OK: override all other checks except for relay tests.
New
> + QUICK:REJECT: reject session.
Current without feature delay-checks
> + REJECT: reject session unless delay-checks is set.
> + TEMP: tempfail session unless delay-checks is set.
Provided by the current delay-checks feature.
> + CONTINUE: continue with other checks.
Current
> Any other result is stored in the session state, i.e., the triple
> (result, RHS, text).
New. I presume this is instead of merely allowing later stages to
rerun the relevant checks. Therefore this is merely caching.
> * MAIL: lookup mail sender address with tag From:
> + OK: override other checks for MAIL.
> + QUICK:REJECT: reject transaction.
Current without delay-checks
> + REJECT: reject transaction unless delay-checks is set.
> + TEMP: tempfail transaction unless delay-checks is set.
Current with delay-checks
> + CONTINUE: continue with other checks.
> Any other result is stored in the transaction state.
> * RCPT: recipient address: first perform lookups with the tag To:
> + RELAY: accept recipient, run other tests if feature delay-checks
> is turned on.
Current
> + QUICK:RELAY: accept recipient, don't run any other tests, even if
> feature delay-checks is turned on.
Appears like we want more control over when to obey delay checks.
Perhaps this could be solved with returning something other than $#ok
in sendmail 8?
> + OK: override other checks for RCPT.
Current
> + QUICK:OK: override other checks except for relay tests.
New
> + REJECT: reject recipient.
> + TEMP: tempfail recipient.
> + CONTINUE: continue with other checks.
> If the lookup returns neither REJECT nor RELAY then the usual
> anti-relay tests are performed.
Current
>
>
> Interaction between Anti-Spam Checks and Milter
>
> Question: should OK from access map disable policy milter checks?
> That would be ``confusing'' for a milter that wants that data, even
> though it would match the REJECT behavior because milter is not
> invoked in that case. However, this is not really a symmetric
> situation: REJECT means that the command is not accepted and hence
> in almost all cases it does not matter what a milter would do about
> it. for OK the milter might want to do something. However, that
> would be a contradiction (a misconfiguration), i.e., one part of
> the configuration explicitly accepts the command another rejects
> it. The right return value in this case is CONTINUE instead of OK.
> In order for the milter to keep track of the SMTP dialog (and hence
> the current state) it might be possible to send the information to
> the milter including the fact that the command has been accepted
> and hence cannot be rejected by the milter (``for your information
> only'').
>
IMHO allowing for rulesets to decide this on a per-milter case by case
basis is superior.
Since there is a whole package of stuff a milter might want to do with
OK'd messages I think its unwise to control milter based on the result
of accept or reject from policy rulesets.
And as has been previously pointed out with the current unpatched
envrcpt milter callback that even receives bad recipients, a milter
might want that.
The current API docs state that after sending a
SMFIS_REJECT/SMFIS_TEMPFAIL from a callback other than envrcpt a
milter should not expect to do anything more than an abort/close. I
think that should be left as it is.
Maybe a SMFIS_CONDREJECT?
I do not see a similar guarantee for reject/tempfail/tempreject codes
set by smfi_setreply()
> Note: the interaction between anti-spam checks and milter is even more
> complicated if feature delay-checks is turned on. In sendmail 8 this was a
> simple hack that only influenced the rulesets (by renaming the connect and
> mail check and calling them from the rcpt ruleset).
If you add more return codes from the rulesets other than $#ok this
would be well more than a hack.
In fact delay checks could actualy do everything (non-milter related)
thats been discussed here so far. Checks would be run at apropriate
stages, return values examined and potentialy acted on or stored in a
macro.
The later stages could retreive the macros, decide whether to take
action or rerun the tests or run new tests.
Then they could line up all results like this and decide again.
> In sendmail X this is
> solved in a different way: the checks are run at the appropriate stage in
> the SMTP dialogue, but the results may be delayed.
Cached in other words.
> Hence a connection
> might be rejected unless a recipient test overrides it (``spam friend'').
> Question: How should this interact with milter tests? Should they just run
> ``normally'' as the other tests and their results are delayed?
If the sendmail policy is not accept any rejection for whatever stage
its in than the fact the milter wanted a rejection should be noted for
later consideration and the milter should be closed/aborted for the
duration of the connection/message
This would fit well with the suggestion I made regarding rulesets
where a flag could be set on a milter to indicate that if any
filtering of certain elements (headers/recipients) occurs then any
potential rejection should be forestalled.
Joe
Mr. Uh Clem
>
> Note: the interaction between anti-spam checks and milter is even more
> complicated if feature delay-checks is turned on. In sendmail 8 this was a
> simple hack that only influenced the rulesets (by renaming the connect and
> mail check and calling them from the rcpt ruleset). In sendmail X this is
> solved in a different way: the checks are run at the appropriate stage in
> the SMTP dialogue, but the results may be delayed. Hence a connection
> might be rejected unless a recipient test overrides it (``spam friend'').
> Question: How should this interact with milter tests? Should they just run
> ``normally'' as the other tests and their results are delayed?
I'm still a sendmail amateur, but what the hey...
I've not yet messed with milters. Partly because most of them seem to
be written in Perl (and I've not had a chance to learn it) and partly
because the next things I want to do are some simple header checks
in rulesets so that users can have access_db entries like 'textonly:'
and 'nohtml:'. (Going much beyond that would seem to take a milter
or similar.) I've not seen a way for DATA phase processing (milter or
ruleset) to tell if there are any FRIENDS in the recipients. If that
happens, it should be possible to give the message a pass. If no
FRIENDS, it can be rejected based on DATA phase tests. If there
is not already something in the session state to indicate the number
of FRIENDS/HATERS, perhaps there should be.
I guess also, might as well note that I'd like to see an additional
level in delay_checks. Anyone for FRIEND/DISLIKER/HATER? I'd like
this mostly for controlling DNSBL checks. I've gotten pretty far
just on DNSBLs and a ruleset which rejects connections from hosts
with no rDNS (thanks for the help on that!)
--
Clem
"If you push something hard enough, it will fall over."
- Fudd's first law of opposition
Joe Maimon
http://www.jmaimon.com/sendmail/patches/milter_rwsets_rcpt_rewrite_eagain.tar.gz
This patch has the Milter Ruleset feature, the RCPT error feature and
a new feature, smfi_rewrite() which will ask sendmail to rewrite
strings and return results back to the milter.
Standard disclaimers of any and all usuability or anything related
including a statement of caution that the only thing I know about this
patch is: It compiles (for me) and What I have tested works (for me).
I expect to be heartily embarrassed by glaring problems in the patch.
(I expect this because it was easier writing it than I had thought it
would be)
Any and all feedback is appreciated.
(Claus: It occured to me that a milter that can rewrite just like
sendmail can be written to implement the spam policy proposed as per
your detailed message in this thread for SM X in SM 8)
Joe