milter and 'user unknown' - order of checks

[Mike Scott]

I'm trying to use mimedefang to detect email addresses with particular
patterns indicating the content is spam, then use add_recipient() and
delete_recipient() to direct to a single spam-receiving mailbox for
further analysis.

I'm finding that if the email address isn't already known (I have a few
aliases to achieve this same aim), sendmail seems to reject the address
/before/ invoking mimedefang, so my code is never called. My code is
only called provided sendmail already knows the original address. Since
these addresses keep changing, I can't keep just adding them to the
aliases file!

Is there any way to alter this behaviour?


Oh, I have 'delay_checks' in effect, if that makes any difference. It's
needed for various reasons.


Thanks.


--
Mike Scott (unet2 <at> [deletethis] scottsonline.org.uk)
Harlow Essex England

[Claus Aßmann]

Mike Scott wrote:

> Is there any way to alter this behaviour?

See the fine documentation...

libmilter/docs/xxfi_negotiate.html

* SMFIP_RCPT_REJ: By setting this bit, a milter can request that the MTA should
also send RCPT commands that have been rejected because the user is unknown
(or similar reasons), but not those which have been rejected because of
syntax errors etc. If a milter requests this protocol step, then it should
check the macro {rcpt_mailer}: if that is set to error, then the recipient
will be rejected by the MTA. Usually the macros {rcpt_host} and {rcpt_addr}
will contain an enhanced status code and an error text in that case,
respectively.



--
Note: please read the netiquette before posting. I will almost never
reply to top-postings which include a full copy of the previous
article(s) at the end because it's annoying, shows that the poster
is too lazy to trim his article, and it's wasting the time of all readers.

[Mike Scott]

On 30/04/12 22:49, Claus Aßmann wrote:
> Mike Scott wrote:
>
>> Is there any way to alter this behaviour?
>
> See the fine documentation...
>
> libmilter/docs/xxfi_negotiate.html
>
> * SMFIP_RCPT_REJ: By setting this bit, a milter can request that the MTA should
> also send RCPT commands that have been rejected because the user is unknown

...
Thanks. I didn't know the magic words to search for.

Anyway, the upshot seems to be that mimedefang doesn't support this
feature. Back to the drawing board :-{

[Andrzej Adam Filip]

Mike Scott <usen...@scottsonline.org.uk.invalid> wrote:
> On 30/04/12 22:49, Claus Aßmann wrote:
>> Mike Scott wrote:
>>
>>> Is there any way to alter this behaviour?
>>
>> See the fine documentation...
>>
>> libmilter/docs/xxfi_negotiate.html
>>
>> * SMFIP_RCPT_REJ: By setting this bit, a milter can request that the MTA should
>> also send RCPT commands that have been rejected because the user is unknown
> ...
> Thanks. I didn't know the magic words to search for.
>
> Anyway, the upshot seems to be that mimedefang doesn't support this
> feature. Back to the drawing board :-{

Which specific version of MIMEDefang milter do you "defame"? :-)

After _very brief_ look at current stable Mimedefang-2.73 (mimedefang.c)
I am under impression that it _tries_ to negotiate ,SMFIP_RCPT_REJ by
default.
[I have not checked if it requires recompilation with non default options]

#v+
124 /* Show me rejected recipients?
125 (Sendmail 8.14.0 and higher only */
126 int SeeRejectedRecipients = 1;
....
1112 if (f1 & SMFIP_RCPT_REJ) {
1113 if (SeeRejectedRecipients) {
1114 *pf1 |= SMFIP_RCPT_REJ;
1115 }
1116 }
....
2344 /* Process command line options */
2345 while ((c = getopt(argc, argv, "GNCDHL:MP:R:S:TU:Xa:b:cdhkm:p:qrstvx:z:")) != -1) {
2346 switch (c) {
..
2352 case 'N':
2353 #ifdef MILTER_BUILDLIB_HAS_NEGOTIATE
2354 SeeRejectedRecipients = 0;
2355 #else
#v-

[Mike Scott]

On 01/05/12 13:05, Andrzej Adam Filip wrote:
> Mike Scott<usen...@scottsonline.org.uk.invalid> wrote:
>> On 30/04/12 22:49, Claus Aßmann wrote:
>>> Mike Scott wrote:
>>>
>>>> Is there any way to alter this behaviour?
>>>
>>> See the fine documentation...
>>>
>>> libmilter/docs/xxfi_negotiate.html
>>>
>>> * SMFIP_RCPT_REJ: By setting this bit, a milter can request that the MTA should
>>> also send RCPT commands that have been rejected because the user is unknown
>> ...
>> Thanks. I didn't know the magic words to search for.
>>
>> Anyway, the upshot seems to be that mimedefang doesn't support this
>> feature. Back to the drawing board :-{
>
> Which specific version of MIMEDefang milter do you "defame"? :-)
>
> After _very brief_ look at current stable Mimedefang-2.73 (mimedefang.c)
> I am under impression that it _tries_ to negotiate ,SMFIP_RCPT_REJ by
> default.
> [I have not checked if it requires recompilation with non default options]
>
> #v+
> 124 /* Show me rejected recipients?
> 125 (Sendmail 8.14.0 and higher only */
> 126 int SeeRejectedRecipients = 1;
> ....

> 1112 if (f1& SMFIP_RCPT_REJ) {

> 1113 if (SeeRejectedRecipients) {
> 1114 *pf1 |= SMFIP_RCPT_REJ;
> 1115 }
> 1116 }
> ....
> 2344 /* Process command line options */
> 2345 while ((c = getopt(argc, argv, "GNCDHL:MP:R:S:TU:Xa:b:cdhkm:p:qrstvx:z:")) != -1) {
> 2346 switch (c) {
> ..
> 2352 case 'N':
> 2353 #ifdef MILTER_BUILDLIB_HAS_NEGOTIATE
> 2354 SeeRejectedRecipients = 0;
> 2355 #else
> #v-

Thanks for the comment.


mimedefang 2.73/fbsd 8.x/sendmail 8.14.4


I've just checked - in spite of what you say, it doesn't seem to work here.

The relevant bits from my mimedefang-filter, with attendant debugging
stuff, are

sub filter_end {
my($entity) = @_;
....
### ON TEST Apr 2012
# send XXXXX.ln1@scottsonline to MERGED-ln1@scottsonline instead - it's
all spam
# (This may break spam count stuff though)(
my @ln1 = ();
open( my $fff, ">>", "/tmp/TEST" ) or die "TEST $!\n";
print $fff "START: ", join(' + ', @Recipients), "\n";
foreach my $recip (@Recipients) {
print $fff "check ", $recip, "\n";
if( $recip =~ /-.+\.ln1\@scottsonline[DELETEDSTUFF]/ ) {
print $fff " match ", $recip, "\n";
md_graphdefang_log('removing ln1 recipient ', $rcpt);
push @ln1, $recip;
delete_recipient($recip);
}
}
if( @ln1) {
my $rcpt = join(',', @ln1);
md_graphdefang_log('ln1 recipients removed', $rcpt);
action_add_header("X-ln1-Recipients", $rcpt);
add_recipient('MERGED-ln1@scottsonline[DELETEDSTUFF]');
}

close $fff;

######

# No sense doing any extra work
return if message_rejected();
...(etc)...
([DELETEDSTUFF] is the end of my domain name, in case that's not obvious.)


if I email a matching address that's in the aliases file, the email is
properly redirected to the spam collection mailbox instead of where
'aliases' says. A valid non-matching email address is correctly logged
by the above code and not redirected.

If however I use any address that's not in the aliases (or passwd) file,
the above code is never called; instead, sendmail logs a 'user unknown'
error.

I wonder if there's something odd about the freebsd build; more likely
there's something obvious I've missed.

[Kees Theunissen]

Talking about something obvious ....
How is the mimedefang daemon started? Mimedefang wants to see unknown
recipients by default but this can be switched off.

From the mimedefang(8) manpage:
-N Normally, mimedefang sees all envelope recipients, even ones that
Sendmail knows to be invalid. If you don't want Sendmail to
perform a milter callback for recipients it knows to be invalid,
invoke mimedefang with the -N flag. Please note that this flag
only works with Sendmail and Milter 8.14.0 and newer. It has no
effect if you're running an older version of Sendmail or Milter.


Regards,

Kees.

--
Kees Theunissen.

[Mike Scott]

On 01/05/12 16:55, Kees Theunissen wrote:
> Mike Scott wrote:
>> On 01/05/12 13:05, Andrzej Adam Filip wrote:
>>> Mike Scott<usen...@scottsonline.org.uk.invalid> wrote:
>>>> On 30/04/12 22:49, Claus Aßmann wrote:
>>>>> Mike Scott wrote:
>>>>>
>>>>>> Is there any way to alter this behaviour?
>>>>>
>>>>> See the fine documentation...
>>>>>
>>>>> libmilter/docs/xxfi_negotiate.html
>>>>>
>>>>> * SMFIP_RCPT_REJ: By setting this bit, a milter can request that the
>>>>> MTA should
>>>>> also send RCPT commands that have been rejected because the user is
>>>>> unknown
>>>> ...
>>>> Thanks. I didn't know the magic words to search for.
>>>>
>>>> Anyway, the upshot seems to be that mimedefang doesn't support this
>>>> feature. Back to the drawing board :-{

(snip)

>> I wonder if there's something odd about the freebsd build; more likely
>> there's something obvious I've missed.
>>
>
> Talking about something obvious ....
> How is the mimedefang daemon started? Mimedefang wants to see unknown
> recipients by default but this can be switched off.
>
> From the mimedefang(8) manpage:
> -N Normally, mimedefang sees all envelope recipients, even ones that
> Sendmail knows to be invalid. If you don't want Sendmail to
> perform a milter callback for recipients it knows to be invalid,
> invoke mimedefang with the -N flag. Please note that this flag
> only works with Sendmail and Milter 8.14.0 and newer. It has no
> effect if you're running an older version of Sendmail or Milter.

Thanks, good point, but that doesn't seem to be the problem. I'm using
freebsd's default flags, which don't seem to include -N:

data# ps axww |grep mimed

86054 ?? I 0:00.01 /usr/local/bin/mimedefang-multiplexor -p
/var/spool/MIMEDefang/mimedefang-multiplexor.pid -z
/var/spool/MIMEDefang -m 2 -x 10 -U mailnull -b 600 -R 100000 -M 300000
-l -y 0 -s /var/spool/MIMEDefang/mimedefang-multiplexor.sock

86056 ?? I 0:12.77 /usr/local/bin/perl
/usr/local/bin/mimedefang.pl -server

86072 ?? I 0:00.07 /usr/local/bin/mimedefang -P
/var/spool/MIMEDefang/mimedefang.pid -R -1 -m
/var/spool/MIMEDefang/mimedefang-multiplexor.sock -z
/var/spool/MIMEDefang -U mailnull -p /var/spool/MIMEDefang/mimedefang.sock

86075 ?? I 0:01.61 /usr/local/bin/perl
/usr/local/bin/mimedefang.pl -server


(mimedefang 2.73/fbsd 8.2/sendmail 8.14.4 )


I have that horrid feeling this is going to be down to some desperately
trivial oversight on my part :-(

[Kees Theunissen]

Mike Scott wrote:

> I have that horrid feeling this is going to be down to some desperately
> trivial oversight on my part :-(

It seems that the only effect of using the SMFIP_RCPT_REJ flag is that
the MimeDefang function "filter_recipient" will be called for rejected
recipients too (if mimdefang's -t option is used otherwise
"filter_recipient" will never be called).

Your code is in the "filter_end" function and inspects the @Recipients
array. So it looks that unknown recipients are not included in the
@Recipients array (which makes some sense).

You can tell sendmail to accept unknown recipients and deliver the
message to a some local mailbox or forward the message to an other
host where the address might be known.
Have a look at cf/README and look for LUSER_RELAY.

Putting a line like:
define(`LUSER_RELAY',`local:your_spam_box')dnl
somewhere at the beginning of your sendmail.mc will cause
sendmail to accept messages for _all_ unknown recipients.
I expect (but didn't test) that in that case unknown recipients
will show up in the @Recipients array -- either with the original
address or with the name of the spam box.

I'm not aware of an option to only accept recipients matching a certain
pattern like you seem to be doing, but such filtering that can be done
in mimedefang if needed.

[Jose-Marcio]

There are some points to take care and verify. What the OP is trying to do isn't not usual (and I'm
not convinced that it's a good idea, but this doesn't matter).

I may be wrong, but let me tell how I interpret all this.

When using the SMFIP_RCPT_REJ, sendmail will point out recipient addresses to which sendmail can't
deliver messages for some reason (unknown recipient is just one of these reasons). In order to get
this working, the milter shall check {rcpt_host} and {rcpt_addr} macros, as pointed out by Claus.

It seems to me that if you use the LUSER_RELAY feature, the unknown recipient will now be known and
deliverable. So the SMFIP_RCPT_REJ option will be useless. (I didn't checked it, although).

Another important point to take into account is that you can only add or delete recipients at the
xxfi_eom callback. So, if no recipients are valid, you won't receive the body of the message and
this callback won't be called.

[Kees Theunissen]

You can still detect unknown recipients in the call_back function after
the "RCPT TO" command (that is the filter_recipient function in
mimedefang). In that case {rcpt_host} and {rcpt_addr} will contain the
values specified in LUSER_RELAY. (Didn't test this either, although.)

Later in the xxfi_eom callback (filter_start, filter, filter_multipart
and filter_end functions in mimedefang) this information will be
available in mimedefang's %RecipientMailers hash.

> Another important point to take into account is that you can only add or
> delete recipients at the xxfi_eom callback. So, if no recipients are
> valid, you won't receive the body of the message and this callback won't
> be called.

Yes that is right. That is probably the one an only reason why the
OP's code isn't called.
If the OP wants to receive (certain) SPAM messages then he needs to have
a least one accepted recipient. And since the OP wants to process
messages addresses to (certain) unknown addresses the needs to accecept
unknown recipients.
And that is what LUSER_RELAY does: accepting unknown (local) recipients.

[Mike Scott]

On 02/05/12 08:40, Kees Theunissen wrote:
> Jose-Marcio wrote:
>> Kees Theunissen wrote:
>>> Mike Scott wrote:
>>>
>>>> I have that horrid feeling this is going to be down to some desperately
>>>> trivial oversight on my part :-(

Thanks for all the comments. It certainly looks as though I've not at
all understood the ramifications of what I was trying to do - in
mitigation, the documentation isn't exactly clear, and I'm certainly no
expert. A seemingly simple bit of code seems to have contained a few
dragons :-)

If I'm understanding correctly, unknown users are only accessible to
mimedefang in filter_recipient. They don't get as far as filter_begin,
having been rejected by sendmail before that point. Therefore I'd need
to check in filter_recipient and keep state somewhere for action later.
Messy!

LUSER_RELAY seems to offer possibilities, but I'd have to think
carefully about handling addresses that are simply mistyped.

Back to the drawing board, and again, many thanks for the comments.

[Robert Bonomi]

In article <jnqrsl$njd$1...@dont-email.me>,

I'll contribute some food for thought.

If you use LUSER_RELAY, you can detect the 'unknown user', _and_, rather
than the re-write to 'deliver' to a specific mailbox, you can have the
'-eom' function _directly_ save the message to a file, and then 'reject'
the message with the 'unknown user' status code. 'Legitimate' mail with
a mistyped addressee will result in the sender seeing the error message,
as usual. 'Spam' to an invalid address will have the error message get
ignored by the remote MTA. (Just as happens when sendmail directly rejects
the recipient as 'unknown'.)

[Andrzej Adam Filip]

Mike Scott <usen...@scottsonline.org.uk.invalid> wrote:
> [... and milter alternative follows:]

> LUSER_RELAY seems to offer possibilities, but I'd have to think
> carefully about handling addresses that are simply mistyped.
>
> Back to the drawing board, and again, many thanks for the comments.

As I understand you want to redirect *SOME* 'user unknown' based only
envelope recipient address.

How complicated is the pattern? Can it be:
a) handled by sendmail.cf R line matching patterns
b) regular expression (regex map)
c) perl code [socket map, AFAIR MIMEDefang supports it]
[ if a or b or c then I may post some hints]

Anyway I would recommend you trying "general milter" instead of
"limited only to sendmail" implementation.

[Joe Zeff]

On Wed, 02 May 2012 09:40:35 +0200, Kees Theunissen wrote:

> And that is what LUSER_RELAY does: accepting unknown (local) recipients.

Thank you. I was getting rather confused by a different meaning of the
term "luser" with which I'm far, far more familiar.

Hmmm, I wonder if my sigmonster knew what I was planning to post...

--
Joe Zeff -- The Guy With The Sideburns:
http://www.zeff.us http://www.lasfs.info
It's not what they call you, it's what you answer to.

[Mike Scott]

Thanks for the offer. It's presently just a simple match (.+-.+\.ln1@),
but I'd rather anything I do be more general. I think the real problem
is I'm out of my depth - in some ways, I'd rather fix that than solve
this particular problem! I'll post more comments in response to Robert's
reply.

[Mike Scott]

On 02/05/12 11:48, Robert Bonomi wrote:
> In article<jnqrsl$njd$1...@dont-email.me>,
> Mike Scott<usen...@scottsonline.org.uk.invalid> wrote:
>> On 02/05/12 08:40, Kees Theunissen wrote:
>>> Jose-Marcio wrote:
>>>> Kees Theunissen wrote:
>>>>> Mike Scott wrote:
>>>>>
>>>>>> I have that horrid feeling this is going to be down to some desperately
>>>>>> trivial oversight on my part :-(

....

>> If I'm understanding correctly, unknown users are only accessible to
>> mimedefang in filter_recipient. They don't get as far as filter_begin,
>> having been rejected by sendmail before that point. Therefore I'd need
>> to check in filter_recipient and keep state somewhere for action later.
>> Messy!
>>
>> LUSER_RELAY seems to offer possibilities, but I'd have to think
>> carefully about handling addresses that are simply mistyped.
>>
>> Back to the drawing board, and again, many thanks for the comments.
>
> I'll contribute some food for thought.
>
> If you use LUSER_RELAY, you can detect the 'unknown user', _and_, rather
> than the re-write to 'deliver' to a specific mailbox, you can have the
> '-eom' function _directly_ save the message to a file, and then 'reject'
> the message with the 'unknown user' status code. 'Legitimate' mail with
> a mistyped addressee will result in the sender seeing the error message,
> as usual. 'Spam' to an invalid address will have the error message get
> ignored by the remote MTA. (Just as happens when sendmail directly rejects
> the recipient as 'unknown'.)

Thanks for the comment.

As I've just been noting to Andrzej, it's probably obvious that I'm
rather out of my depth here, so please bear with me.

I see that LUSER_RELAY can be defined as something like
local:spam-receiver; if that's a valid alias, mail for all unknown users
is sent there. However, after some pottering, I'm still a bit stuck.

As far as mimedefang is concerned, mail sent to that catch-all address
is treated just as mail for any other mailbox. I can pick up on my
'spammy receiver' pattern easily enough and then save it and cause a
bounce too. But I see no way to distinguish mail for proper recipients
from other mail that should be simply bounced (short, that is, of
matching against passwd and the aliases file myself - but I can't
imagine that ought to be needed).

Is there any way for mimedefang to check whether LUSER_RELAY is
involved? If there is, I've missed it :-(

I think I need a stronger hint please!!

Thanks.

[Kees Theunissen]

See the section "FILTERING BY RECIPIENT" in the mimedefang-filter
manpage and have a look at the example "filter_recipient"
subroutine in that section.
If LUSER_RELAY is involved with the definition mentioned above
then you can test the value assigned to $rcpt_addr.
The value will be "spam-receiver" (not tested but it should be).

[Mike Scott]

On 09/05/12 12:39, Kees Theunissen wrote:
.....

>> I see that LUSER_RELAY can be defined as something like
>> local:spam-receiver; if that's a valid alias, mail for all unknown users
>> is sent there. However, after some pottering, I'm still a bit stuck.
>>
>> As far as mimedefang is concerned, mail sent to that catch-all address
>> is treated just as mail for any other mailbox. I can pick up on my
>> 'spammy receiver' pattern easily enough and then save it and cause a
>> bounce too. But I see no way to distinguish mail for proper recipients
>> from other mail that should be simply bounced (short, that is, of
>> matching against passwd and the aliases file myself - but I can't
>> imagine that ought to be needed).
>>
>> Is there any way for mimedefang to check whether LUSER_RELAY is
>> involved? If there is, I've missed it :-(
>>
>> I think I need a stronger hint please!!
>
> See the section "FILTERING BY RECIPIENT" in the mimedefang-filter
> manpage and have a look at the example "filter_recipient"
> subroutine in that section.
> If LUSER_RELAY is involved with the definition mentioned above
> then you can test the value assigned to $rcpt_addr.
> The value will be "spam-receiver" (not tested but it should be).

Thanks for the comment. I'm afraid though that $rcpt_addr is actually
set to the original address, not the spam-receiver mailbox, in both
filter_recipient() and filter_begin().

(Also, using filter_recipient was discussed iirc earlier in the thread -
I'd have to save a lot of context if I took that route. Not impossible,
but I'd rather avoid if possible.)

[Kees Theunissen]

Mike Scott wrote:
> On 09/05/12 12:39, Kees Theunissen wrote:

>> See the section "FILTERING BY RECIPIENT" in the mimedefang-filter
>> manpage and have a look at the example "filter_recipient"
>> subroutine in that section.
>> If LUSER_RELAY is involved with the definition mentioned above
>> then you can test the value assigned to $rcpt_addr.
>> The value will be "spam-receiver" (not tested but it should be).
>
> Thanks for the comment. I'm afraid though that $rcpt_addr is actually
> set to the original address, not the spam-receiver mailbox, in both
> filter_recipient() and filter_begin().

I did some testing too and it looks that you are right. Actually
$rcpt_addr will be set to the result of the rewriting in rule sets
3 and 0. If you didn't customize those rule sets then for local
addresses $rcpt_addr will be the original user name with the
"@" and the local host name stripped.
The replacement of the unknown user name with the value of
LUSER_RELAY happens later after alias expansion has been done,
but that seems not to be reflected in $rcpt_addr.

> (Also, using filter_recipient was discussed iirc earlier in the thread -
> I'd have to save a lot of context if I took that route. Not impossible,
> but I'd rather avoid if possible.)

My intention was that in filter_recipient you would simply reject the
unknown addresses accepted by sendmail that you're not interested in.
That wouldn't change your context.
Of course that would require some way to distinguish between valid
and invalid addresses. I hoped/expected that $rcpt_addr could be used
for that but that seems not to be possible.
Sorry for the wrong suggestion.

My next suggestion would be to use some customized rewriting rules
to accept certain patterns in unknown addresses. This has already
been suggested by Andrzej Adam Filip.
Depending on your pattern this can be done by simple rewriting
rules (if the pattern fits properly in sendmail's rules to split
an address in "tokens" and to test the tokens with $*, $- or $+) or
you'll need a "regex map" to detect the pattern.
Your current pattern requires a regex map.

If Andrzej is already working on this I won't spent my time only
to more or less duplicate his efforts, but I'm willing to work
this out if needed. I's not too complicated.

[Mike Scott]

On 09/05/12 19:14, Kees Theunissen wrote:
....

> My next suggestion would be to use some customized rewriting rules
> to accept certain patterns in unknown addresses. This has already
> been suggested by Andrzej Adam Filip.
> Depending on your pattern this can be done by simple rewriting
> rules (if the pattern fits properly in sendmail's rules to split
> an address in "tokens" and to test the tokens with $*, $- or $+) or
> you'll need a "regex map" to detect the pattern.
> Your current pattern requires a regex map.
>
> If Andrzej is already working on this I won't spent my time only
> to more or less duplicate his efforts, but I'm willing to work
> this out if needed. I's not too complicated.

I'm grateful for the comments and offer. But not so keen on maintaining
sendmail rules, which I've always regarded as something of a dark art :-)

Anyway, I've been pottering a bit more, and think I see an easier way
out by defining a new mailer in the config file.

At present, I've a simple perl script as a mailer that logs the message
and exits with status 67 (EX_NOUSER).

My .mc file now has (give or take line wrap)
define(`LUSER_RELAY', `unknownuser:localhost')dnl

together with
Munknownuser, P=/usr/plumtree/bin/unknownuser.pl, m=1, F=l,
T=DNS/RFC822/X-Unix, A=xyzzy -z $u


With that in place, any email to an unknown user is passed to my script
(which currently just logs everything), and the mail is bounced with an
appropriate message. Obviously, I can later make the script save only
what I need, and I'll have to take care over multiple copies being active.

Does this seem a reasonable approach? It has the merit at least of
putting all the coding in an easily testable place.

[Kees Theunissen]

That will probably generate lots of "backscatter".
You really want to reject senders/recipients/entire_messages during the
SMTP-dialog -- and let the sending host do the error handling.

Permanent failures returned by delivery agents (mailers) can only be
handled by sending back a "delivery failure notification" mail because
the message has already been accepted by your sendmail daemon.
In case of spam (currently the most likely source of invalid recipient
addresses) you'll almost certain send your failure notification to
a spoofed sender address.

I did a quick test to show this.

I created a ".procmailrc" file for a user with the following contents:

# --
# First set the EXITCODE. Code 67 is unknown user.
# --
EXITCODE=67
# --
# And finaly trash the message. Note that this recipe ALLWAYS
# succeeds. Any rule placed after this recipe wil NEVER be
# reached.
# --
:0:
/dev/null


And started an SMTP session.

kees@lankhmar:~$ telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 lankhmar.remmin.home ESMTP Sendmail 8.14.4/8.14.4; Thu, 10 May 2012
23:22:12 +0200
ehlo localhost
250-lankhmar.remmin.home Hello IDENT:1000@localhost [127.0.0.1], pleased
to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-EXPN
250-VERB
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH DIGEST-MD5 CRAM-MD5
250-DELIVERBY
250 HELP
mail from:<kees@localhost>
250 2.1.0 <kees@localhost>... Sender ok
rcpt to:<cjt@localhost>
250 2.1.5 <cjt@localhost>... Recipient ok
data
354 Enter mail, end with "." on a line by itself
from: <kees@localhost>
to: <cjt@localhost>
subject: bounce test.

blah
.
250 2.0.0 q4ALMCSg002170 Message accepted for delivery
quit
221 2.0.0 lankhmar.remmin.home closing connection
Connection closed by foreign host.
kees@lankhmar:~$


The smtp conversation shown above indicates that the message has been
accepted by sendmail. But the "local mailer" (procmail in my case)
rejected the message afterwards and a DSN message was sent as shown
by the mail logs:

May 10 23:24:00 lankhmar sm-mta[2170]: q4ALMCSg002170:
from=<kees@localhost>, size=71, class=0, nrcpts=1,
msgid=<201205102123....@lankhmar.remmin.home>, proto=ESMTP,
daemon=MTA, relay=IDENT:1000@localhost [127.0.0.1]
May 10 23:24:00 lankhmar sm-mta[2176]: q4ALMCSg002170:
to=<cjt@localhost>, ctladdr=<kees@localhost> (1000/100), delay=00:01:00,
xdelay=00:00:00, mailer=local, pri=30396, dsn=5.1.1, stat=User unknown
May 10 23:24:00 lankhmar sm-mta[2176]: q4ALMCSg002170: q4ALO0Sg002176:
DSN: User unknown
May 10 23:24:07 lankhmar sm-mta[2176]: q4ALO0Sg002176:
to=<kees@localhost>, delay=00:00:07, xdelay=00:00:07, mailer=local,
pri=31420, dsn=2.0.0, stat=Sent



So a "local mailer" doesn't seem to be the right place to handle
delivery attempts to unknown users.

[Mike Scott]

On 10/05/2012 23:32, Kees Theunissen wrote:
> Mike Scott wrote:

...

>> At present, I've a simple perl script as a mailer that logs the message
>> and exits with status 67 (EX_NOUSER).
>>
>> My .mc file now has (give or take line wrap)
>> define(`LUSER_RELAY', `unknownuser:localhost')dnl
>>
>> together with
>> Munknownuser, P=/usr/plumtree/bin/unknownuser.pl, m=1, F=l,
>> T=DNS/RFC822/X-Unix, A=xyzzy -z $u
>>
>>
>> With that in place, any email to an unknown user is passed to my script
>> (which currently just logs everything), and the mail is bounced with an
>> appropriate message. Obviously, I can later make the script save only
>> what I need, and I'll have to take care over multiple copies being
>> active.
>>
>> Does this seem a reasonable approach? It has the merit at least of
>> putting all the coding in an easily testable place.
>
> That will probably generate lots of "backscatter".
> You really want to reject senders/recipients/entire_messages during the
> SMTP-dialog -- and let the sending host do the error handling.

...

> So a "local mailer" doesn't seem to be the right place to handle
> delivery attempts to unknown users.

Oh dear; you're right of course. And I should have twigged. Many thanks
for pointing this out.

Now where's my drawing board? :-(

[Mike Scott]

On 10/05/12 23:32, Kees Theunissen wrote:
> Mike Scott wrote:
>> On 09/05/12 19:14, Kees Theunissen wrote:
>> ....
>>> My next suggestion would be to use some customized rewriting rules
>>> to accept certain patterns in unknown addresses. This has already
>>> been suggested by Andrzej Adam Filip.
>>> Depending on your pattern this can be done by simple rewriting
>>> rules (if the pattern fits properly in sendmail's rules to split
>>> an address in "tokens" and to test the tokens with $*, $- or $+) or
>>> you'll need a "regex map" to detect the pattern.
>>> Your current pattern requires a regex map.

....

Ok, my final attempt :-)

in my .mc file:-
LOCAL_RULE_0
R$+.ln1 <@$+> spam-rx<@$2>

should (and seems to) rewrite anything.ln1@somewhere to be
spam-rx@somewhere.

Seems to redirect matching addresses as needed, and without touching any
headers (which is a bonus). Have I missed anything please?


Incidentally, the Bat Book has a lovely 'question for the student'
concerning rules: "Is it possible to have text in the LHS that is
literally $+ , rather than an operator with the same characters?". I'm
dying to know the answer - can't find it anywhere! :-(

[Andrzej Adam Filip]

Mike Scott <usen...@scottsonline.org.uk.invalid> wrote:
> On 10/05/12 23:32, Kees Theunissen wrote:
>> Mike Scott wrote:
>>> On 09/05/12 19:14, Kees Theunissen wrote:
>>> ....
>>>> My next suggestion would be to use some customized rewriting rules
>>>> to accept certain patterns in unknown addresses. This has already
>>>> been suggested by Andrzej Adam Filip.
>>>> Depending on your pattern this can be done by simple rewriting
>>>> rules (if the pattern fits properly in sendmail's rules to split
>>>> an address in "tokens" and to test the tokens with $*, $- or $+) or
>>>> you'll need a "regex map" to detect the pattern.
>>>> Your current pattern requires a regex map.
> ....
>
> Ok, my final attempt :-)
>
> in my .mc file:-
> LOCAL_RULE_0
> R$+.ln1 <@$+> spam-rx<@$2>
>
> should (and seems to) rewrite anything.ln1@somewhere to be
> spam-rx@somewhere.
>
> Seems to redirect matching addresses as needed, and without touching
> any headers (which is a bonus). Have I missed anything please?

0) Limit your rule to local email domains only.
R$+.ln1 <@$=w> spam-rx<@$2>

Test it with the command below:
echo '3,0 x.ln1@localhost' | sendmail -bt -d21.12

1) Do you want to get original envelope recipient addresses?
[ YES => Are you ready to use procmail script in /etc/procmailrcs]

[Andrzej Adam Filip]

FEATURE(`anfi/mrs') would allow you to pass addresses "catched" by
LUSER_RELAY via your rule set.
[ FEATURE(`anfi/mrs') requires a few lines patch of m4/proto.m4 file]

[Mike Scott]

Thanks - I'd actually caught that one before reading your post (slight
hiccup with TB and news being slow getting through :-( ).

What I currently have is

# read in aliases that always mean spam
F{spamrx} /etc/mail/spamrx

LOCAL_RULE_0
# Divert known spam recipient address to a common mailbox.
# match fixed stuff
R$={spamrx} < @ $=w . > $: spam-rx < @ $2 . >
# match stuf with variable part, special-cased
R$+.ln1 < @ $=w . > $: spam-rx < @ $2 . >
R$+.ntl.com < @ $=w . > $: spam-rx < @ $2 . >

which actually does seem to do the right things, rather to my surprise.
I know the $: isn't strictly needed.

>
> Test it with the command below:
> echo '3,0 x.ln1@localhost' | sendmail -bt -d21.12

..... then finally
parse returns: $# local $: spam-rx

while
echo '3,0 x.ln1@remotelhost' | sendmail -bt -d21.12
gives
...
parse returns: $# relay $@ XXXX . XXXX . co . uk $: x . ln1 <
@ remotelhost >


Ta muchly.

>
> 1) Do you want to get original envelope recipient addresses?
> [ YES => Are you ready to use procmail script in /etc/procmailrcs]
>
>> Incidentally, the Bat Book has a lovely 'question for the student'
>> concerning rules: "Is it possible to have text in the LHS that is
>> literally $+ , rather than an operator with the same characters?". I'm
>> dying to know the answer - can't find it anywhere! :-(

To answer my own question after a bit of trial and error, $$ seems to
mean a literal $. Weird it doesn't seem to be documented anywhere.

[Mike Scott]

Thanks, although that's a bit over my head. But the problem is I think
sufficiently solved in the other subthread by extra rules in ruleset 0.

[Kees Theunissen]

Mike Scott wrote:

>>> Incidentally, the Bat Book has a lovely 'question for the student'
>>> concerning rules: "Is it possible to have text in the LHS that is
>>> literally $+ , rather than an operator with the same characters?". I'm
>>> dying to know the answer - can't find it anywhere! :-(
>
> To answer my own question after a bit of trial and error, $$ seems to
> mean a literal $. Weird it doesn't seem to be documented anywhere.

Paragraph 5.1 of the "Sendmail Installation and Operation Guide".
That guide can be found in the sendmail source tree in the doc/op
directory. Your distribution might have installed it somewhere under
/usr/doc or /usr/share/doc.

[Mike Scott]

Too obvious :-) I've pored over the Bat Book cover to cover, and
probably overheated google's servers searching.... anyway, thank you.


(And thanks again particularly to you and to Andrzej for the advice
given. I've learned a fair amount about rules and macros through that.)