Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Long delay in SMTP handshake
474 views
Skip to first unread message
Urs Grützner
Feb 3, 2014, 11:19:31 AM2/3/14
to
We have Problems with one mail sender SMTP to receive mails on our CGP 5.1.16. On OSX.5.8 The sender goes always into a time out, while trying to make the handshake
The IT responsible always measures 27 secs delay until our server answers with 220 (in between the peer has closed connection, because his time out limit is lower)
When I make myself a Telnet connection from different sites, I get the 220 always immediate.
1. I do not understand these contradictory results
2. If this delay as the sender measures is reality: is a wrong setup in our server responsible for that? Which one?
Thanks for help
[ursg@Urs-MacBook-Air-Mountain-Lion|~]% time telnet mail.ems.ch 25 12:14 ttys000 |
Trying 194.209.14.153...
Connected to mail.ems.ch.
Escape character is '^]'.
220 ems.ch ESMTP CommuniGate Pro 5.1.16 is glad to see you!
quit
221 ems.ch CommuniGate Pro SMTP closing connection
Connection closed by foreign host.
telnet mail.ems.ch 25 0.03s user 0.01s system 1% cpu 1.920 total
[ursg@Urs-MacBook-Air-Mountain-Lion|~]% 12:14 ttys000 |
Thomas Bleek
Feb 3, 2014, 11:50:53 AM2/3/14
to
Settings > Mail > SMTP > Receiving
delay prompt for non client senders
tb
--
Dr. Thomas Bleek, Netzwerkadministrator
Helmholtz-Zentrum Potsdam
Deutsches GeoForschungsZentrum
Telegrafenberg A20/225
D-14473 Potsdam
Tel.: +49 331 288- 1818/1681 Fax.: 1730 Mobil: +49 172 1543233
E-Mail: b...@gfz-potsdam.de
Dr. Thomas Bleek, Netzwerkadministrator
Helmholtz-Zentrum Potsdam
Deutsches GeoForschungsZentrum
Telegrafenberg A20/225
D-14473 Potsdam
Tel.: +49 331 288- 1818/1681 Fax.: 1730 Mobil: +49 172 1543233
E-Mail: b...@gfz-potsdam.de
Mark J Strawcutter
Feb 3, 2014, 1:11:01 PM2/3/14
to
And a sender who won't wait 27 seconds for the [intentionally-delayed]
SMTP prompt is broken and needs to fix/reconfigure his server.
Mark
#############################################################
This message is sent to you because you are subscribed to
the mailing list <CGat...@mail.stalker.com>.
To unsubscribe, E-mail to: <CGateP...@mail.stalker.com>
To switch to the DIGEST mode, E-mail to <CGatePr...@mail.stalker.com>
To switch to the INDEX mode, E-mail to <CGatePr...@mail.stalker.com>
Send administrative queries to <CGatePro...@mail.stalker.com>
SMTP prompt is broken and needs to fix/reconfigure his server.
Mark
This message is sent to you because you are subscribed to
the mailing list <CGat...@mail.stalker.com>.
To unsubscribe, E-mail to: <CGateP...@mail.stalker.com>
To switch to the DIGEST mode, E-mail to <CGatePr...@mail.stalker.com>
To switch to the INDEX mode, E-mail to <CGatePr...@mail.stalker.com>
Send administrative queries to <CGatePro...@mail.stalker.com>
Bill Cole
Feb 3, 2014, 11:44:11 PM2/3/14
to
On 3 Feb 2014, at 11:19, Urs Grützner wrote:
> We have Problems with one mail sender SMTP to receive mails on our CGP
> 5.1.16. On OSX.5.8 The sender goes always into a time out, while
> trying to make the handshake
>
> The IT responsible always measures 27 secs delay until our server
> answers with 220 (in between the peer has closed connection, because
> his time out limit is lower)
Any timeout less than 300 seconds conflicts with the reaffirmations in
> We have Problems with one mail sender SMTP to receive mails on our CGP
> 5.1.16. On OSX.5.8 The sender goes always into a time out, while
> trying to make the handshake
>
> The IT responsible always measures 27 secs delay until our server
> answers with 220 (in between the peer has closed connection, because
> his time out limit is lower)
2008 (RFC5321 http://tools.ietf.org/html/rfc5321#section-4.5.3.2) and
2001 (RFC2821 http://tools.ietf.org/html/rfc2821#section-4.5.3.2) of the
1989 (RFC1123 http://tools.ietf.org/html/rfc1123#page-62) specification
of minimum SMTP timeouts. A piece of software that fails to wait 27
seconds for a 220 reply is not an implementation of SMTP, it is a
broken-by-design toy. Anyone trying to use such a thing for sending mail
on the open net is like a child trying to drive a toy car on an open
highway.
> When I make myself a Telnet connection from different sites, I get the
> 220 always immediate.
>
> 1. I do not understand these contradictory results
non-client sources is broken reverse DNS for the source IP address that
results in DNS timeouts (rather than faster explicit DNS failures.)
However, that does not seem to be the cause in your case...
> 2. If this delay as the sender measures is reality: is a wrong setup
> in our server responsible for that? Which one?
CGP that applies only to non-client IP addresses. That delay is a very
useful low-cost way to catch "spambot" senders, but anything more than 5
seconds adds little to the bot identification rate while delaying nearly
all mail.
There's nothing formally wrong with setting a long delay, but there are
an amazing number of children in toy cars trying to navigate the
Internet and you may want to accommodate some of them. It also can be
annoying to have every message delayed and even more annoying to
accommodate the large number of waiting SMTP connections you can pile up
by imposing a long delay.
> Thanks for help
>
>
> [ursg@Urs-MacBook-Air-Mountain-Lion|~]% time telnet mail.ems.ch 25
> 12:14 ttys000 |
> Trying 194.209.14.153...
> Connected to mail.ems.ch.
> Escape character is '^]'.
> 220 ems.ch ESMTP CommuniGate Pro 5.1.16 is glad to see you!
IP. That could include an IP that you've recently authenticated to CGP
from.
I didn't do this from your laptop :) --
$ time telnet mail.ems.ch 25
quit
477 you did not wait for a prompt
Connection closed by foreign host.
real 0m27.548s
user 0m0.016s
sys 0m0.006s
The 477 reply is a sure sign that your server is intentionally delaying
the initial prompt and turning away clients that talk too fast.
Tom Rymes
Feb 4, 2014, 4:01:09 PM2/4/14
to
On 02/04/2014 2:38 PM, Urs Grützner wrote:
> If I understand correctly: when I do the Telnet from may laptop, after
> I was connecting with my mail program, my temporarily IP number is
> detected as client, so I do not get a waiting time.
That's it.
> However the prompt delay was and is set to 0 sec.
>
> I am still wondering where this 27 sec delay is coming from? It seems
> to be constantly 27 sec, no random variation. The delay prompt for
> non-clients seems not to be the culprit.
Perhaps this setting can be configured both system-wide and per-domain?
Tom
> If I understand correctly: when I do the Telnet from may laptop, after
> I was connecting with my mail program, my temporarily IP number is
> detected as client, so I do not get a waiting time.
That's it.
> However the prompt delay was and is set to 0 sec.
>
> I am still wondering where this 27 sec delay is coming from? It seems
> to be constantly 27 sec, no random variation. The delay prompt for
> non-clients seems not to be the culprit.
Perhaps this setting can be configured both system-wide and per-domain?
Tom
Bill Cole
Feb 5, 2014, 1:41:29 PM2/5/14
to
On 4 Feb 2014, at 14:38, Urs Grützner wrote:
> Thanks a lot for this elaborate and helpful answer
I rarely avoid "elaborate" and am always glad to find that I've managed
"helpful" as well.
> If I understand correctly: when I do the Telnet from may laptop,
> after I was connecting with
> my mail program, my temporarily IP number is detected as client, so I
> do not get a waiting time.
Right. You can tell what class of SMTP client CGP sees you as (normal,
banned, or client) by reading the text of the initial SMTP prompt. That
is useful when diagnosing problems that may only exist for non-clients:
if you see " glad to see you!" then CGP is treating you as a client.
> However the prompt delay was and is set to 0 sec.
>
> I am still wondering where this 27 sec delay is coming from? It seems
> to be constantly 27 sec, no random variation. The delay prompt for
> non-clients seems not to be the culprit.
Which conflicts with the functional tests:
1. When seen by CGP as connecting from a client IP, you got an instant
220 prompt.
2. When seen by CGP as a normal unknown client IP, I got an
idiosyncratic "477" reply that I don't believe CGP ever sends unless it
has intentionally delayed the prompt.
I have 2 theories that could explain this:
1. You are using the "Detect Clients by DNS Name" feature and there is
something wrong with your DNS configuration on the CGP host that is
causing the DNS lookups needed to determine client status to take a long
time. A common cause of DNS lookups having fixed delays is if you have
multiple DNS servers configured for the system's resolver but the first
one (or more) are not actually answering your queries. This will result
in normal queries always taking a fixed long time as the system resolver
tries the configured DNS servers in order with a timeout for each
unresponsive server. Because there's a system default timeout and
programs like CGP can specify their own timeouts on each query, it isn't
unusual in such cases to see unwavering unusual values for the DNS
delay. The same sort of problem can also occur with less visibility if
you are using a DNS server that is configured to forward queries to
other DNS servers, but in that case you will usually not see the problem
persistently because such DNS servers usually only exist for their
caching (i.e. a second connection soon after a delayed one the same IP
won't get delayed because the DNS records are cached.) If this is the
correct explanation, it indicates that the "477" reply I got CGP to send
is NOT only sent in conjunction with intentional delays, but also is
sent when CGP is waiting on DNS to figure out which prompt text to send.
2. The "Delay Prompt For Non-Clients" feature is actually set but it was
somehow set to 27 seconds, a value that CGP doesn't include in the web
admin menu for that setting. This is possible because settings can also
be changed via the CGP CLI (port 106) or by manually editing the config
files. This would raise the question of who did such a thing, since you
are not aware of it, but it is easily checked:
/var/CommuniGate/Settings/SMTP.settings is a plain text file with
settings in an easily-read format and DelayPrompt is the setting name.
To fix this without restarting CGP or using the CLI, you could just set
the delay to some non-zero value in the web admin interface. I recommend
5 seconds (even though my boss likes 10...) because my experience with
mail servers where the timing of "fast talker" violations is measured
tells me >99% of the bots that get caught for that do so in under 5
seconds, ~95% in under 1 second.
> Another question: Isn't it possible to see in the log the
> "knock-knock" of a incoming connection. I only see the answer of our
> server and the connection closed py peer entry, so I cannot verify the
> waiting time here (and may be the reason for it)
I'm not sure, since the log settings have evolved over time and I have
not used 5.1.x, but you might be able to see the details you need by
setting the log levels for SMTP and server internals (Settings->General
page) to "Low Level" temporarily. You might need "All Info" to find the
hard evidence, but I doubt it. Another option would be to find an IP
that you can test this from (or that sends you mail regularly) and put
it into the list at Settings->Network->Debug IPs to capture everything
related to that IP in the log. Normally the depth of logging from "All
Info" or "Debug IPs" inclusion is more than you need or want (and may
trigger privacy concerns) but sometimes using those temporarily is the
only way to know what CGP is doing.
> Thanks a lot for this elaborate and helpful answer
I rarely avoid "elaborate" and am always glad to find that I've managed
"helpful" as well.
> If I understand correctly: when I do the Telnet from may laptop,
> after I was connecting with
> my mail program, my temporarily IP number is detected as client, so I
> do not get a waiting time.
banned, or client) by reading the text of the initial SMTP prompt. That
is useful when diagnosing problems that may only exist for non-clients:
if you see " glad to see you!" then CGP is treating you as a client.
> However the prompt delay was and is set to 0 sec.
>
> I am still wondering where this 27 sec delay is coming from? It seems
> to be constantly 27 sec, no random variation. The delay prompt for
> non-clients seems not to be the culprit.
1. When seen by CGP as connecting from a client IP, you got an instant
220 prompt.
2. When seen by CGP as a normal unknown client IP, I got an
idiosyncratic "477" reply that I don't believe CGP ever sends unless it
has intentionally delayed the prompt.
I have 2 theories that could explain this:
1. You are using the "Detect Clients by DNS Name" feature and there is
something wrong with your DNS configuration on the CGP host that is
causing the DNS lookups needed to determine client status to take a long
time. A common cause of DNS lookups having fixed delays is if you have
multiple DNS servers configured for the system's resolver but the first
one (or more) are not actually answering your queries. This will result
in normal queries always taking a fixed long time as the system resolver
tries the configured DNS servers in order with a timeout for each
unresponsive server. Because there's a system default timeout and
programs like CGP can specify their own timeouts on each query, it isn't
unusual in such cases to see unwavering unusual values for the DNS
delay. The same sort of problem can also occur with less visibility if
you are using a DNS server that is configured to forward queries to
other DNS servers, but in that case you will usually not see the problem
persistently because such DNS servers usually only exist for their
caching (i.e. a second connection soon after a delayed one the same IP
won't get delayed because the DNS records are cached.) If this is the
correct explanation, it indicates that the "477" reply I got CGP to send
is NOT only sent in conjunction with intentional delays, but also is
sent when CGP is waiting on DNS to figure out which prompt text to send.
2. The "Delay Prompt For Non-Clients" feature is actually set but it was
somehow set to 27 seconds, a value that CGP doesn't include in the web
admin menu for that setting. This is possible because settings can also
be changed via the CGP CLI (port 106) or by manually editing the config
files. This would raise the question of who did such a thing, since you
are not aware of it, but it is easily checked:
/var/CommuniGate/Settings/SMTP.settings is a plain text file with
settings in an easily-read format and DelayPrompt is the setting name.
To fix this without restarting CGP or using the CLI, you could just set
the delay to some non-zero value in the web admin interface. I recommend
5 seconds (even though my boss likes 10...) because my experience with
mail servers where the timing of "fast talker" violations is measured
tells me >99% of the bots that get caught for that do so in under 5
seconds, ~95% in under 1 second.
> Another question: Isn't it possible to see in the log the
> "knock-knock" of a incoming connection. I only see the answer of our
> server and the connection closed py peer entry, so I cannot verify the
> waiting time here (and may be the reason for it)
I'm not sure, since the log settings have evolved over time and I have
not used 5.1.x, but you might be able to see the details you need by
setting the log levels for SMTP and server internals (Settings->General
page) to "Low Level" temporarily. You might need "All Info" to find the
hard evidence, but I doubt it. Another option would be to find an IP
that you can test this from (or that sends you mail regularly) and put
it into the list at Settings->Network->Debug IPs to capture everything
related to that IP in the log. Normally the depth of logging from "All
Info" or "Debug IPs" inclusion is more than you need or want (and may
trigger privacy concerns) but sometimes using those temporarily is the
only way to know what CGP is doing.
0 new messages