info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
dns lookups on to: message header causing mail to get stuck in mail queue
528 views
Skip to first unread message
Michael Grant
Dec 25, 2015, 8:38:51 AM12/25/15
to
I'm seeing mail come in with multiple recipients in to/cc headers like:
to: us...@my.box, unsub...@togethersingles.com
The us...@my.box being valid and destined to a user on my box, the other being bogus.
I use procmail for local delivery and some users ! forward mail to their gmail acct. Sendmail tries to do a dns lookup on each host in the to: field again even though it doesn't need to.
I get messages like this going to postmaster:
timeout writing message to [127.0.0.1]
and mailq command reports errors like:
(host map: lookup (togethersingles.com): deferred)
and the message gets stuck indefinitely in the mail queue.
To be clear, this is the To:/Cc: headers, not the envelope RCTP TO line. It seems wrong to me to do DNS queries on these names which are not consequential to message delivery.
I recently updated to Debian's version of 8.15.2. I do not recall seeing this before, so perhaps something has changed recently?
Michael Grant
to: us...@my.box, unsub...@togethersingles.com
The us...@my.box being valid and destined to a user on my box, the other being bogus.
I use procmail for local delivery and some users ! forward mail to their gmail acct. Sendmail tries to do a dns lookup on each host in the to: field again even though it doesn't need to.
I get messages like this going to postmaster:
timeout writing message to [127.0.0.1]
and mailq command reports errors like:
(host map: lookup (togethersingles.com): deferred)
and the message gets stuck indefinitely in the mail queue.
To be clear, this is the To:/Cc: headers, not the envelope RCTP TO line. It seems wrong to me to do DNS queries on these names which are not consequential to message delivery.
I recently updated to Debian's version of 8.15.2. I do not recall seeing this before, so perhaps something has changed recently?
Michael Grant
Claus Aßmann
Dec 25, 2015, 9:05:03 AM12/25/15
to
Michael Grant wrote:
> I use procmail for local delivery and some users ! forward mail to their gmail acct. Sendmail tries to do a dns
> lookup on each host in the to: field again even though it doesn't need to.
Did you tell sendmail that "it doesn't need to"?
> I use procmail for local delivery and some users ! forward mail to their gmail acct. Sendmail tries to do a dns
> lookup on each host in the to: field again even though it doesn't need to.
> To be clear, this is the To:/Cc: headers, not the envelope RCTP TO line. It seems wrong to me to do DNS queries
> on these names which are not consequential to message delivery.
It's in the fine documentation...
> on these names which are not consequential to message delivery.
> I recently updated to Debian's version of 8.15.2. I do not recall seeing this before, so perhaps something has
> changed recently?
8.15.1/8.15.1 2014/12/06
...
If header rewriting fails due to a temporary map lookup failure,
queue the mail for later retry instead of sending it
without rewriting the header. Note: this is done
while the mail is being sent and hence the transaction
is aborted, which only works for SMTP/LMTP mailers
hence the handling of temporary map failures is
suppressed for other mailers. SMTP/LMTP servers may
complain about aborted transactions when this problem
occurs.
See also "DNS Lookups" in sendmail/TUNING.
It might be a good idea to read that...
--
Note: please read the netiquette before posting. I will almost never
reply to top-postings which include a full copy of the previous
article(s) at the end because it's annoying, shows that the poster
is too lazy to trim his article, and it's wasting the time of all readers.
Michael Grant
Dec 25, 2015, 6:54:41 PM12/25/15
to
Claus, thanks for the clues.
I seem to have fixed this by adding this line in submit.mc:
define(`confDIRECT_SUBMISSION_MODIFIERS', `C')dnl
and this line in sendmail.mc:
FEATURE(`nocanonify', `canonify_hosts')
I must say the documentation could be a little bit clearer about which files these lines go in. Also, it would be helpful to state that these modifiers (the `C') are documented under the documentation for DaemonPortOptions.
It's still not clear to me why it is useful for sendmail to do dns lookups for hosts with an already fqdn which it's not going to contact. Does this have something to do with canonification for addresses with generic TLDs that might not have a dot in them at all?
I seem to have fixed this by adding this line in submit.mc:
define(`confDIRECT_SUBMISSION_MODIFIERS', `C')dnl
and this line in sendmail.mc:
FEATURE(`nocanonify', `canonify_hosts')
I must say the documentation could be a little bit clearer about which files these lines go in. Also, it would be helpful to state that these modifiers (the `C') are documented under the documentation for DaemonPortOptions.
It's still not clear to me why it is useful for sendmail to do dns lookups for hosts with an already fqdn which it's not going to contact. Does this have something to do with canonification for addresses with generic TLDs that might not have a dot in them at all?
JonB
Jan 11, 2016, 6:32:08 AM1/11/16
to
On Friday, 25 December 2015 23:54:41 UTC, Michael Grant wrote:
> Claus, thanks for the clues.
>
> I seem to have fixed this by adding this line in submit.mc:
>
> define(`confDIRECT_SUBMISSION_MODIFIERS', `C')dnl
>
> and this line in sendmail.mc:
>
> FEATURE(`nocanonify', `canonify_hosts')
>
I've just run into this issue as well - I have a bunch of junk mail queued on our primary MX which cannot be sent to the backend mail server, as it's deferred by the sending sendmail process at delivery time:
> Claus, thanks for the clues.
>
> I seem to have fixed this by adding this line in submit.mc:
>
> define(`confDIRECT_SUBMISSION_MODIFIERS', `C')dnl
>
> and this line in sendmail.mc:
>
> FEATURE(`nocanonify', `canonify_hosts')
>
Talking to mailboxes.ourdomain.com...
>>> MAIL From:<som...@domainthatdoesexist.com> SIZE=1526
250 2.1.5 <reci...@domain.com> Recipient ok
>>> DATA
354 Enter mail, end with "." on a line by itself
bogusdomain-in-from-header.com: Name server timeout
timeout writing message to mailboxes.ourdomain.com
This also annoyingly leaves it in the primary MX's queue tagged as 'deferred: Name server: mailboxes.ourdomain.com: host name lookup failure' - which is a little confusing! :(
Adding the canonfiy options above doesn't seem to fix mail *already* in the queue?
The email has envelope from domains that exist - but the 'From:' headers in our case are for bogus domains - so as it is sendmail accepts the initial message but then cannot forward onto our mailboxes machine as when it goes to deliver it it 'trips over' the from: headers bogus domain.
We'd rather push those into the mailboxes (and let the antispam stuff there handle it) than reject them at this stage (having originally accepted the email).
JonB
Jan 11, 2016, 6:45:22 AM1/11/16
to
On Monday, 11 January 2016 11:32:08 UTC, JonB wrote:
> This also annoyingly leaves it in the primary MX's queue tagged as 'deferred: Name server: mailboxes.ourdomain.com: host name lookup failure' - which is a little confusing! :(
Ok - having checked again this it doesn't mail on the primary MX is just deferred queued with 'Deferred' as the reason.
> This also annoyingly leaves it in the primary MX's queue tagged as 'deferred: Name server: mailboxes.ourdomain.com: host name lookup failure' - which is a little confusing! :(
Only at delivery time is the error "deferred: Name server: mailboxes.ourdomain.com: host name lookup failure" spat out.
Claus Aßmann
Jan 11, 2016, 7:10:03 AM1/11/16
to
Michael Grant wrote:
> It's still not clear to me why it is useful for sendmail to do dns lookups for hosts with an already fqdn which
How does sendmail know it is the _canonical_ "fqdn"?
> It's still not clear to me why it is useful for sendmail to do dns lookups for hosts with an already fqdn which
Note: this is basically sendmail operating in "fix up everything that
is thrown at me" mode -- which you can turn off (and should) on an MTA.
Claus Aßmann
Jan 11, 2016, 7:10:03 AM1/11/16
to
JonB wrote:
> Adding the canonfiy options above doesn't seem to fix mail *already* in the queue?
Did you change the correct cf file? Did you restart sendmail
> Adding the canonfiy options above doesn't seem to fix mail *already* in the queue?
so the queue runners are affected?
What happens if you run one queued item (in verbose mode)?
See also TRACEFLAGS for useful debugging flags, e.g.,
-d21.4 -d8.8 -d38.20
JonB
Jan 11, 2016, 9:42:28 AM1/11/16
to
On Monday, 11 January 2016 12:10:03 UTC, Claus Aßmann wrote:
> JonB wrote:
>
> > Adding the canonfiy options above doesn't seem to fix mail *already* in the queue?
>
> Did you change the correct cf file? Did you restart sendmail
> so the queue runners are affected?
I am sure I changed the correct file (.mc -> rebuild .cf and copy in place) - I 'tested' it by running sendmail from the CLI i.e. 'sendmail -qIqueue-id -v'.
> JonB wrote:
>
> > Adding the canonfiy options above doesn't seem to fix mail *already* in the queue?
>
> Did you change the correct cf file? Did you restart sendmail
> so the queue runners are affected?
> What happens if you run one queued item (in verbose mode)?
> See also TRACEFLAGS for useful debugging flags, e.g.,
> -d21.4 -d8.8 -d38.20
Thanks.
JonB
Jan 11, 2016, 10:09:29 AM1/11/16
to
On Monday, 11 January 2016 12:10:03 UTC, Claus Aßmann wrote:
> What happens if you run one queued item (in verbose mode)?
> See also TRACEFLAGS for useful debugging flags, e.g.,
> -d21.4 -d8.8 -d38.20
I kept some of the queue files from earlier - running one with the above, I get:
> See also TRACEFLAGS for useful debugging flags, e.g.,
> -d21.4 -d8.8 -d38.20
354 Enter mail, end with "." on a line by itself
rewritten as: "HeaderFrom Name" < HeaderFrom_Name @ dodgydomain . com > < @ >
rewritten as: "HeaderFrom Name" < HeaderFrom_Name @ dodgydomain . com >
rewritten as: < "HeaderFrom Name" < HeaderFrom_Name @ dodgydomain . com > >
rewritten as: < HeaderFrom_Name @ dodgydomain . com > >
rewritten as: < HeaderFrom_Name @ dodgydomain . com >
rewritten as: HeaderFrom_Name @ dodgydomain . com
rewritten as: HeaderFrom_Name < @ dodgydomain . com >
rewrite: ruleset Canonify2 input: HeaderFrom_Name < @ dodgydomain . com >
rewrite: RHS $&{daemon_flags} => "b h"
rewritten as: b h $| HeaderFrom_Name < @ dodgydomain . com >
getcanonname(ellgens.com), trying dns
... much DNS chatter logged and timeouts ensue ...
The 'header from' address for this queue entry is:
From: "HeaderFrom Name" <HeaderF...@dodgydomain.com>
(Obviously obfuscated)
This is with "define(`nocanonify',`canonify_hosts')" in the .mc (compiled to active .cf) file.
Thanks.
Claus Aßmann
Jan 11, 2016, 12:20:04 PM1/11/16
to
JonB wrote:
> On Monday, 11 January 2016 12:10:03 UTC, Claus Aßmann wrote:
> rewrite: ruleset Canonify2 input: HeaderFrom_Name < @ dodgydomain . com >
> rewrite: RHS $&{daemon_flags} => "b h"
> rewritten as: b h $| HeaderFrom_Name < @ dodgydomain . com >
> getcanonname(ellgens.com), trying dns
> ... much DNS chatter logged and timeouts ensue ...
Well, obfuscating the data is NOT helpful.
> On Monday, 11 January 2016 12:10:03 UTC, Claus Aßmann wrote:
> rewrite: ruleset Canonify2 input: HeaderFrom_Name < @ dodgydomain . com >
> rewrite: RHS $&{daemon_flags} => "b h"
> rewritten as: b h $| HeaderFrom_Name < @ dodgydomain . com >
> getcanonname(ellgens.com), trying dns
> ... much DNS chatter logged and timeouts ensue ...
Moreover, you messed it up (see above)
> This is with "define(`nocanonify',`canonify_hosts')" in the .mc (compiled to active .cf) file.
See the earlier posting and the fine documentation.
If you actually used the FEATURE() then compare the entire
Canonify2
ruleset from your cf file with the rules shown below
or run
echo 'Canonify2 x<@ellgens.com>' | sendmail -bt -d21.4
and check what's going on...
SCanonify2=96
# handle special cases for local names
R$* < @ localhost > $* $: $1 < @ $j . > $2 no domain at all
R$* < @ localhost . $m > $* $: $1 < @ $j . > $2 local domain
R$* < @ localhost . UUCP > $* $: $1 < @ $j . > $2 .UUCP domain
# check for IPv4/IPv6 domain literal
R$* < @ [ $+ ] > $* $: $1 < @@ [ $2 ] > $3 mark [addr]
R$* < @@ $=w > $* $: $1 < @ $j . > $3 self-literal
R$* < @@ $+ > $* $@ $1 < @ $2 > $3 canon IP addr
# if really UUCP, handle it immediately
# hostnames ending in class P are always canonical
R$* < @ $* $=P > $* $: $1 < @ $2 $3 . > $4
R$* < @ $* $~P > $* $: $&{daemon_flags} $| $1 < @ $2 $3 > $4
# pass to name server to make hostname canonical if in class {Canonify}
R$* $| $* < @ $* $={Canonify} > $* $: $2 < @ $[ $3 $4 $] > $5
# pass to name server to make hostname canonical if requested
R$* c $* $| $* < @ $* > $* $: $3 < @ $[ $4 $] > $5
R$* $| $* < @ $+ . > $* $: $2 < @ $3 . > $4
# add a trailing dot to qualified hostnames so other rules will work
R$* $| $* < @ $+.$+ > $* $: $2 < @ $3.$4 . > $5
# lookup unqualified hostnames
R$* $| $* < @ $* > $* $: $2 < @ $[ $3 $] > $4
R$* $| $* $: $2
# local host aliases and pseudo-domains are always canonical
R$* < @ $=w > $* $: $1 < @ $2 . > $3
R$* < @ $=M > $* $: $1 < @ $2 . > $3
R$* < @ $* . . > $* $1 < @ $2 . > $3
JonB
Jan 12, 2016, 6:43:32 AM1/12/16
to
On Monday, 11 January 2016 17:20:04 UTC, Claus Aßmann wrote:
> > This is with "define(`nocanonify',`canonify_hosts')" in the .mc (compiled to active .cf) file.
>
> If that's what you did then it's "obvious" what's wrong...
> See the earlier posting and the fine documentation.
Ok - yes I used define not FEATURE - unfortunately nothing complains about it - and it still churns out a CF file which 'works' (just not in the way intended). So hand up to that one.
> > This is with "define(`nocanonify',`canonify_hosts')" in the .mc (compiled to active .cf) file.
>
> If that's what you did then it's "obvious" what's wrong...
> See the earlier posting and the fine documentation.
> If you actually used the FEATURE() then compare the entire
I also notice - by default now the configuration causes backscatter. As mail is accepted without the nocanonify option but not delivered forward. So if not caught - days later it's returned to the envelope FROM address / return path, because the From header had an unresolvable / junk domain in it :( Just something to be mindful of perhaps.
Anyway - thankyou for the help.
Jon
0 new messages