Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

8.15.1, about header rewriting fails due to a temporary map lookup failure

96 views
Skip to first unread message

Sergey

unread,
Dec 5, 2016, 8:02:08 AM12/5/16
to
Hello.

RELEASE_NOTES of 8.15.1 contains warning:

| If header rewriting fails due to a temporary map lookup failure,
| queue the mail for later retry instead of sending it
| without rewriting the header.

It seems to me that it's not a good change for a some of cases.
Can it be disabled in sendmail.cf or in compile time ?

Additionally. Is it difficult to specify exactly what domain is
causing the error ? The error message is not informative now:

Dec 2 10:18:23 mail sm-msp-queue[1402]: uAT1Nkk3567536: to=ma...@example.com, delay=3+05:23:07, xdelay=00:00:04, mailer=relay, pri=
138992339, relay=localhost.localdomain. [127.0.0.1], dsn=4.0.0, stat=Deferred: Name server: localhost.localdomain.: host name lookup failure

--
Regards, Sergey

Claus Aßmann

unread,
Dec 5, 2016, 9:50:03 AM12/5/16
to
Sergey wrote:

> RELEASE_NOTES of 8.15.1 contains warning:

> | If header rewriting fails due to a temporary map lookup failure,
> | queue the mail for later retry instead of sending it
> | without rewriting the header.

There's more in that entry, esp.

See also "DNS Lookups" in sendmail/TUNING.

> It seems to me that it's not a good change for a some of cases.
> Can it be disabled in sendmail.cf or in compile time ?

follow the hint to "see also" and you'll find:

Note: starting with 8.15, sendmail will not ignore temporary map
lookup failures during header rewriting, which means that DNS lookup
problems even for headers will cause messages to stay in the queue.
Hence it is strongly suggested to use the nocanonify feature;
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

BTW: there could also be some trick to redefine the host map
to ignore temp.errors, e.g., quoting op.*

-t Normally, when a map attempts to do a lookup
and the server fails (e.g., sendmail
couldn't contact any name server; this is
not the same as an entry not being found in
the map), the message being processed is
queued for future processing. The -t flag
turns off this behavior, letting the tempo-
rary failure (server down) act as though it
were a permanent failure (entry not found).
It is particularly useful for DNS lookups,
where someone else's misconfigured name
server can cause problems on your machine.
However, care must be taken to ensure that
you don't bounce mail that would be resolved
correctly if you tried again. A common
strategy is to forward such mail to another,
possibly better connected, mail server.


> Additionally. Is it difficult to specify exactly what domain is
> causing the error ? The error message is not informative now:

If someone has a patch, please send it.

--
Note: please read the netiquette before posting. I will almost never
reply to top-postings which include a full copy of the previous
article(s) at the end because it's annoying, shows that the poster
is too lazy to trim his article, and it's wasting the time of all readers.

Sergey

unread,
Dec 6, 2016, 9:24:10 AM12/6/16
to
Claus Aßmann wrote:

>> RELEASE_NOTES of 8.15.1 contains warning:
>
>> | If header rewriting fails due to a temporary map lookup failure,
>> | queue the mail for later retry instead of sending it
>> | without rewriting the header.
>
> There's more in that entry, esp.
>
> See also "DNS Lookups" in sendmail/TUNING.

Sorry. Somehow I thought about op/TUNING.

> Note: starting with 8.15, sendmail will not ignore temporary map
> lookup failures during header rewriting, which means that DNS lookup
> problems even for headers will cause messages to stay in the queue.
> Hence it is strongly suggested to use the nocanonify feature;
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I seems that I already read and used this some years ago... Hm.
I try it now (added it to mc and rebuilt sendmail.cf), but it's
not working for me.

About the situation in more detail. Some messages (which received
by MTA) are redirected to special archive mailbox via bin/sendmail
by milter. Some of it stays in spool/clientmqueue due to a problem
with one of the e-mail in "To" field. Somewhere there is a difference
in the delivery from Internet and from localhost.

I use 8.15.2 now.

--
Regards, Sergey

Claus Aßmann

unread,
Dec 6, 2016, 11:50:03 AM12/6/16
to
Sergey wrote:
> Claus Aßmann wrote:

> > Hence it is strongly suggested to use the nocanonify feature;

> About the situation in more detail. Some messages (which received
> by MTA) are redirected to special archive mailbox via bin/sendmail
> by milter. Some of it stays in spool/clientmqueue due to a problem
> with one of the e-mail in "To" field. Somewhere there is a difference
> in the delivery from Internet and from localhost.

Looks like your submit.mc needs the same feature?

Sergey

unread,
Dec 7, 2016, 9:33:13 AM12/7/16
to
Claus Aßmann wrote:

>> > Hence it is strongly suggested to use the nocanonify feature;
>
>> About the situation in more detail. Some messages (which received
>> by MTA) are redirected to special archive mailbox via bin/sendmail
>> by milter. Some of it stays in spool/clientmqueue due to a problem
>> with one of the e-mail in "To" field. Somewhere there is a difference
>> in the delivery from Internet and from localhost.
>
> Looks like your submit.mc needs the same feature?

Why (but I tried it. unsuccessfully) ? bin/sendmail accepted mail and
sm-msp-queue attempt to process it. MTA process receive but drop connection:

Dec 7 18:19:32 sendmail[9251]: uB7EJQOI009251: --- 250 2.1.5 <arh@mail>... Recipient ok
Dec 7 18:19:32 sendmail[9251]: uB7EJQOI009251: <-- DATA
Dec 7 18:19:32 sendmail[9251]: uB7EJQOI009251: --- 354 Enter mail, end with "." on a line by itself
Dec 7 18:19:37 sm-msp-queue[9243]: uB61gokx022204: SYSERR(root): timeout writing message to localhost.localdomain.
Dec 7 18:19:37 sm-msp-queue[9243]: uB61gokx022204: to=arh@mail, delay=1+12:36:47, xdelay=00:00:11,
mailer=relay, pri=28023025, relay=localhost.localdomain. [127.0.0.1], dsn=4.0.0,
stat=Deferred: Name server: localhost.localdomain.: host name lookup failure
Dec 7 18:19:37 sendmail[9251]: uB7EJQOI009251: collect: premature EOM: unexpected close
Dec 7 18:19:37 sendmail[9251]: uB7EJQOI009251: collect: unexpected close on connection from localhost.localdomain, sender=<>

Ou, possible I need to increase timeouts in submit.mc.
I'll try later today or tomorrow morning.

--
Regards, Sergey

Claus Aßmann

unread,
Dec 7, 2016, 11:40:03 AM12/7/16
to
Sergey wrote:

> Dec 7 18:19:37 sm-msp-queue[9243]: uB61gokx022204: to=arh@mail, delay=1+12:36:47, xdelay=00:00:11,
> mailer=relay, pri=28023025, relay=localhost.localdomain. [127.0.0.1], dsn=4.0.0,
> stat=Deferred: Name server: localhost.localdomain.: host name lookup failure

This seems to indicate that sm-msp-queue has the "problem".
So you could run it in verbose mode with some debugging
and see what's going on, e.g., something like:

sendmail -Ac -qIuB61gokx022204 -v -d8.9 -d9.1

(see TRACEFLAGS for more "interesting" debug settings).

Sergey

unread,
Dec 8, 2016, 12:41:57 AM12/8/16
to
Claus Aßmann wrote:

> sendmail -Ac -qIuB61gokx022204 -v -d8.9 -d9.1
>
> (see TRACEFLAGS for more "interesting" debug settings).

The problem domain was fixed tonight. I lost opportunity
to try, I will have to wait next case.

--
Regards, Sergey

Sergey

unread,
Jan 17, 2017, 4:03:52 PM1/17/17
to
Claus Aßmann wrote:

> sendmail -Ac -qIuB61gokx022204 -v -d8.9 -d9.1
>.
> (see TRACEFLAGS for more "interesting" debug settings).

It happens again.

# sendmail -Ac -qIv0GDk8nw007690 -v -d8.9 -d9.1
...

host_map_lookup(gulfish.ru) => dns_getcanonname(gulfish.ru, trymx=1)
dns_getcanonname: trying gulfish.ru. (AAAA)
NO: errno=110, h_errno=2
dns_getcanonname: trying gulfish.ru. (A)
NO: errno=110, h_errno=2
dns_getcanonname: trying gulfish.ru. (MX)
NO: errno=110, h_errno=2
dns_getcanonname: trying gulfish.ru.mail (AAAA)
NO: errno=0, h_errno=1
FAIL (2)
gulfish.ru: Name server timeout
timeout writing message to localhost.localdomain.
arhiv@mail... Deferred: Name server: localhost.localdomain.: host name lookup failure
Closing connection to localhost.localdomain.

I think that the error message should contain gulfish.ru instead of
localhost.localdomain.

But the main problem is disable check. FEATURE(`nocanonify') used
but problem exists.

--
Regards, Sergey.

Claus Aßmann

unread,
Jan 17, 2017, 8:04:34 PM1/17/17
to
Sergey wrote:

> # sendmail -Ac -qIv0GDk8nw007690 -v -d8.9 -d9.1

> host_map_lookup(gulfish.ru) => dns_getcanonname(gulfish.ru, trymx=1)

> gulfish.ru: Name server timeout

> arhiv@mail... Deferred: Name server: localhost.localdomain.: host name lookup failure

> I think that the error message should contain gulfish.ru instead of
> localhost.localdomain.

I agree. Got a patch?

> But the main problem is disable check. FEATURE(`nocanonify') used
> but problem exists.

Do you really have that FEATURE in submit.mc?

fgrep nocanonify.m4 /etc/mail/submit.cf

and

fgrep '$[' /etc/mail/submit.cf

Sergey

unread,
Jan 18, 2017, 12:45:19 AM1/18/17
to
Claus Aßmann wrote:

>> arhiv@mail... Deferred: Name server: localhost.localdomain.: host name lookup failure
>
>> I think that the error message should contain gulfish.ru instead of
>> localhost.localdomain.
>
> I agree. Got a patch?

I haven't really thought about it. A patch should be for the Sendmail's C
code or for the cf ?

>> But the main problem is disable check. FEATURE(`nocanonify') used
>> but problem exists.
>
> Do you really have that FEATURE in submit.mc?
>
> fgrep nocanonify.m4 /etc/mail/submit.cf

# fgrep nocanonify.m4 /etc/mail/submit.cf
##### $Id: nocanonify.m4,v 8.13 2013-11-22 20:51:11 ca Exp $ #####

> and
>
> fgrep '$[' /etc/mail/submit.cf

# fgrep '$[' /etc/mail/submit.cf
R$* $| $* < @ $* $={Canonify} > $* $: $2 < @ $[ $3 $4 $] > $5
R$* c $* $| $* < @ $* > $* $: $3 < @ $[ $4 $] > $5
R<@> $+ $:<?> $[ $1 $]

--
Regards, Sergey

Claus Aßmann

unread,
Jan 18, 2017, 8:23:14 AM1/18/17
to
Sergey wrote:
> Claus Aßmann wrote:

> >> arhiv@mail... Deferred: Name server: localhost.localdomain.: host name lookup failure

> > I agree. Got a patch?

> I haven't really thought about it. A patch should be for the Sendmail's C
> code or for the cf ?

C.

> # fgrep nocanonify.m4 /etc/mail/submit.cf
> ##### $Id: nocanonify.m4,v 8.13 2013-11-22 20:51:11 ca Exp $ #####


> > fgrep '$[' /etc/mail/submit.cf

> R$* $| $* < @ $* $={Canonify} > $* $: $2 < @ $[ $3 $4 $] > $5
> R$* c $* $| $* < @ $* > $* $: $3 < @ $[ $4 $] > $5
> R<@> $+ $:<?> $[ $1 $]

Hmm, that's weird. Because I can reproduce the problem only
without that feature:
dns_getcanonname(gulfish.ru, trymx=1)
dns_getcanonname: trying gulfish.ru. (A)
NO: errno=60, h_errno=2
dns_getcanonname: trying gulfish.ru. (MX)
NO: errno=60, h_errno=2
dns_getcanonname: trying gulfish.ru.esmtp.org (A)
NO: errno=0, h_errno=1
With nocanonify those lines do not show up.

What's your complete mc file (without the comment lines if there
are many)?

What's the output of:
sendmail -bt -C /etc/mail/submit.cf -d8.20
${daemon_flags}
$={Canonify}
/tryflags hr
/try relay x...@gulfish.ru
-d21.16
Canonify2 x<@gulfish.ru>
/quit

Sergey

unread,
Jan 20, 2017, 1:13:42 AM1/20/17
to
Claus Aßmann wrote:

>> I haven't really thought about it. A patch should be for the
>> Sendmail's C code or for the cf ?
>
> C.

Perhaps I try it. But I don't know whether the result.

> What's your complete mc file (without the comment lines if there
> are many)?

It was old submit.mc based on 8.13. I changed it now to current
from sendmail-8.15.2/cf/cf/:

VERSIONID(`$Id: submit.mc,v 8.15 2013-11-22 20:51:08 ca Exp $')
OSTYPE(`linux')dnl
define(`confCF_VERSION', `Submit')dnl
define(`__OSTYPE__',`')dnl dirty hack to keep proto.m4 from complaining
define(`_USE_DECNET_SYNTAX_', `1')dnl support DECnet
define(`confTIME_ZONE', `USE_TZ')dnl
define(`confDONT_INIT_GROUPS', `True')dnl
FEATURE(`nocanonify')dnl
FEATURE(`msp', `[127.0.0.1]')dnl

> What's the output of:
> sendmail -bt -C /etc/mail/submit.cf -d8.20
> ${daemon_flags}
> $={Canonify}
> /tryflags hr
> /try relay x...@gulfish.ru
> -d21.16
> Canonify2 x<@gulfish.ru>
> /quit

# sendmail -bt -C /etc/mail/submit.cf -d8.20
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter <ruleset> <address>
> ${daemon_flags}
Undefined
> $={Canonify}
> /tryflags hr
> /try relay x...@gulfish.ru
Trying header recipient address x...@gulfish.ru for mailer relay
canonify input: x @ gulfish . ru
Canonify2 input: x < @ gulfish . ru >
Canonify2 returns: x < @ gulfish . ru . >
canonify returns: x < @ gulfish . ru . >
2 input: x < @ gulfish . ru . >
2 returns: x < @ gulfish . ru . >
MasqSMTP input: x < @ gulfish . ru . >
MasqSMTP returns: x < @ gulfish . ru . >
final input: x < @ gulfish . ru . >
final returns: x @ gulfish . ru
Rcode = 0, addr = x...@gulfish.ru
> -d21.16
> Canonify2 x<@gulfish.ru>
Canonify2 input: x < @ gulfish . ru >
-----trying rule (line 680): $* < @ localhost > $*
----- rule fails
-----trying rule (line 681): $* < @ localhost . fish-king . ru > $*
----- rule fails
-----trying rule (line 682): $* < @ localhost . UUCP > $*
----- rule fails
-----trying rule (line 685): $* < @ [ $+ ] > $*
----- rule fails
-----trying rule (line 686): $* < @ @ $=w > $*
----- rule fails
-----trying rule (line 687): $* < @ @ $+ > $*
----- rule fails
-----trying rule (line 697): $* < @ $* $=P > $*
----- rule fails
-----trying rule (line 698): $* < @ $* $~P > $*
-----rule matches: $: $&{daemon_flags} $| $1 < @ $2 $3 > $4
rewrite: RHS $&{daemon_flags} => "(NULL)"
$1: 0xbfd0a7dc="x"
$2: 0xbfd0a7e2="gulfish" 0xbfd0a7ea="."
$3: 0xbfd0a7ec="ru"
$4:
rewritten as: $| x < @ gulfish . ru >
-----trying rule (line 700): $* $| $* < @ $* $={Canonify} > $*
----- rule fails
-----trying rule (line 702): $* c $* $| $* < @ $* > $*
----- rule fails
-----trying rule (line 703): $* $| $* < @ $+ . > $*
----- rule fails
-----trying rule (line 705): $* $| $* < @ $+ . $+ > $*
-----rule matches: $: $2 < @ $3 . $4 . > $5
$2: 0xbfd0a7dc="x"
$3: 0xbfd0a7e2="gulfish"
$4: 0xbfd0a7ec="ru"
$5:
rewritten as: x < @ gulfish . ru . >
-----trying rule (line 706): $* $| $*
----- rule fails
-----trying rule (line 709): $* < @ $=w > $*
----- rule fails
-----trying rule (line 710): $* < @ $=M > $*
----- rule fails
-----trying rule (line 711): $* < @ $* . . > $*
----- rule fails
Canonify2 returns: x < @ gulfish . ru . >
> /quit

--
Regards, Sergey

Claus Aßmann

unread,
Jan 20, 2017, 9:34:33 AM1/20/17
to
Sergey wrote:

> It was old submit.mc based on 8.13. I changed it now to current
> from sendmail-8.15.2/cf/cf/:

and that resolves the problem:

> > /try relay x...@gulfish.ru
> Trying header recipient address x...@gulfish.ru for mailer relay

> Canonify2 input: x < @ gulfish . ru >
> Canonify2 returns: x < @ gulfish . ru . >

No DNS lookup anymore -- as configured and expected.

Sergey

unread,
Jan 20, 2017, 11:08:24 AM1/20/17
to
Claus Aßmann wrote:

>> It was old submit.mc based on 8.13. I changed it now to current
>> from sendmail-8.15.2/cf/cf/:
>
> and that resolves the problem:

No. "sendmail -Ac -qIv0GDk8nw007690 -v -d8.9 -d9.1" ends as before:

FAIL (2)
gulfish.ru: Name server timeout
timeout writing message to [127.0.0.1]
arhiv@mail... Deferred: Name server: [127.0.0.1]: host name lookup failure
Closing connection to [127.0.0.1]

I would have written if replacement submit.mc has helped... :-(

--
Regards, Sergey.

Claus Aßmann

unread,
Jan 20, 2017, 12:01:08 PM1/20/17
to
Sergey wrote:

> gulfish.ru: Name server timeout

Where in the message is gulfish.ru used?
envelope / header?
sender / recipient?

Run the test outlined before with the matching /tryflags:
/tryflags flags :set flags used by parsing. The flags can be `H' for
Header or `E' for Envelope, and `S' for Sender or `R'
for Recipient. These can be combined, `HR' sets
flags for header recipients.
/try mailer addr :rewrite address into the form it will have when
presented to the indicated mailer.

You can also run that queue entry with more debugging flags,
e.g., -d21.16 in addition to the others.

You have to find out why $[ ... $] is used despite the nocanonify
feature.

Sergey

unread,
Jan 21, 2017, 12:46:06 AM1/21/17
to
Claus Aßmann wrote:

>> gulfish.ru: Name server timeout
>
> Where in the message is gulfish.ru used?
> envelope / header?
> sender / recipient?

I have it reproduced thus:

==== test.eml ====
From: test <root>
To: x...@gulfish.ru

test
==================

cat test.eml | /usr/sbin/sendmail -f"<>" my@e-mail

> You have to find out why $[ ... $] is used despite the nocanonify
> feature.

I are trying it, but unsuccessful yet.

--
Regards, Sergey.

Claus Aßmann

unread,
Jan 21, 2017, 7:45:21 AM1/21/17
to
Sergey wrote:

> From: test <root>

Use a fully qualified address to avoid extra rewriting.

> To: x...@gulfish.ru

> cat test.eml | /usr/sbin/sendmail -f"<>" my@e-mail

Useless use of cat...

/usr/sbin/sendmail -f"<>" my@e-mail < test.eml

> > You have to find out why $[ ... $] is used despite the nocanonify
> > feature.

> I are trying it, but unsuccessful yet.

You ignored my comment about the debug flags: why?
Ok, one more (last?) time:

/usr/sbin/sendmail -D/tmp/d.log -d89.9 -d8.9 -d9.9 -d38.20 -d39.1 -d60.1 -d21.16 -f"<>" my@e-mail < test.eml
(see also TRACEFLAGS)

Then look at /tmp/d.log and find the places rules
match and apply $[ $] to gulfish.ru.

Sergey

unread,
Jan 21, 2017, 12:47:17 PM1/21/17
to
Claus Aßmann wrote:

>> From: test <root>
>
> Use a fully qualified address to avoid extra rewriting.

It's not essential for this problem, is not it ?

>> cat test.eml | /usr/sbin/sendmail -f"<>" my@e-mail
>
> Useless use of cat...
>
> /usr/sbin/sendmail -f"<>" my@e-mail < test.eml

I don't like this syntax: this makes it difficult to change parameters
of main command ("Up", then "BS"). :-)

>> > You have to find out why $[ ... $] is used despite the nocanonify
>> > feature.
>
>> I are trying it, but unsuccessful yet.
>
> You ignored my comment about the debug flags: why?

I haven't ignored, but it takes time (I'm not very familiar with CF
syntax). I postponed it, probably until monday. I rarely spend my
weekends at the computer lately.

> Ok, one more (last?) time:
>
> /usr/sbin/sendmail -D/tmp/d.log -d89.9 -d8.9 -d9.9 -d38.20 -d39.1 -d60.1
> -d21.16 -f"<>" my@e-mail < test.eml (see also TRACEFLAGS)
>
> Then look at /tmp/d.log and find the places rules
> match and apply $[ $] to gulfish.ru.

45354 d.log - this will take time too...
I'll send it now personally.

--
Regards, Sergey.

Claus Aßmann

unread,
Jan 21, 2017, 1:09:23 PM1/21/17
to
Sergey wrote:

> >> From: test <root>

> > Use a fully qualified address to avoid extra rewriting.

> It's not essential for this problem, is not it ?

It makes the debug log larger and hence harder to analyze.

> > You ignored my comment about the debug flags: why?

> 45354 d.log - this will take time too...

See above...

Just look for $[ in that file and the broken domain and then
go backwards from there to find the ruleset and match that
to your submit.cf file.

Claus Aßmann

unread,
Jan 22, 2017, 9:07:50 AM1/22/17
to
sendmail/TUNING:

* DNS Lookups
-----------------------------------------------

sendmail performs by default host name canonifications by using
host name lookups. This process is meant to replace unqualified
host name with qualified host names, and CNAMEs with the non-aliased
name. However, these lookups can take a while for large address
lists, e.g., mailing lists. If you can assure by other means that
host names are canonical, you should use

FEATURE(`nocanonify', `canonify_hosts')

in your .mc file. For further information on this feature and
additional options see cf/README. If sendmail is invoked directly
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
to send e-mail then either the -G option should be used or
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

define(`confDIRECT_SUBMISSION_MODIFIERS', `C')

should be added to the .mc file.


So add
define(`confDIRECT_SUBMISSION_MODIFIERS', `C')
to your submit.mc file, recreate the cf file, and the problem
should be solved (for newly submitted mail, for mail in the queue
you can specify the corresponding option on the command line).


BTW: sendmail/TUNING:
As a last resort, it is possible to override the host map to ignore
temporary failures, e.g.,
Khost host -t
However, this can cause inconsistent header rewriting.

Sergey

unread,
Jan 24, 2017, 3:16:32 PM1/24/17
to
Claus Aßmann wrote:

> So add
> define(`confDIRECT_SUBMISSION_MODIFIERS', `C')
> to your submit.mc file, recreate the cf file, and the problem
> should be solved (for newly submitted mail, for mail in the queue
> you can specify the corresponding option on the command line).

I seems it works, thanks.

--
Regards, Sergey.
0 new messages