Listing IP addresses in mail queue
Duncan B.
Hi,
Firstly this is my first post to the list, so apologies if I've not
correctly followed any procedures.
I'm a new user to Postfix (ex Qmail user) and love it. However, there is
one feature of "qmqtool" that was very useful: qmqtool -i, to list queue
entries by IP address. Is there any way to see which IP addresses in the
postfix queues are the most dominant?
Also I am seeing a lot of bounces in the deferred queue from
MAILER-DAEMON@my-host. Is there a way to instantly destroy double bounces
rather than defer them, or is this bad practice/against RFC standards?
Finally, is there a way to emulate the 'greet pause' feature of Sendmail,
whereby Postfix will kill the client connection if they send any commands
before the 220 prompt? This seemed to be working with a 2.4 version of
Postfix that I was testing, but not with 2.6/2.7. The docs mention a
change to the way this works, but I don't fully understand this.
I have the following in main.cf
smtpd_client_restrictions = sleep 3, reject_unauth_pipelining,
permit_mynetworks, reject
smtpd_delay_reject = no
Thanks for any help
Cheers,
Duncan Baxter
Portsmouth.
Noel Jones
>
> Hi,
>
> Firstly this is my first post to the list, so apologies if I've not
> correctly followed any procedures.
>
> I'm a new user to Postfix (ex Qmail user) and love it. However, there is
> one feature of "qmqtool" that was very useful: qmqtool -i, to list queue
> entries by IP address. Is there any way to see which IP addresses in the
> postfix queues are the most dominant?
Postfix doesn't store IPs in the queue file, so there is no
such tool for postfix.
However, the 'qshape' tool will list the queue by destination
domain.
http://www.postfix.org/QSHAPE_README.html#qshape
>
> Also I am seeing a lot of bounces in the deferred queue from
> MAILER-DAEMON@my-host. Is there a way to instantly destroy double bounces
> rather than defer them, or is this bad practice/against RFC standards?
These aren't double bounces, just regular non-delivery reports.
It's far better to find out why you have a bunch of bounces in
your queue and fix the problem.
Generally this is a result of accepting undeliverable mail.
Don't do that. Use proper recipient validation so postfix can
reject unknown recipients automatically.
http://www.postfix.org/ADDRESS_CLASS_README.html
http://www.postfix.org/LOCAL_RECIPIENT_README.html
>
> Finally, is there a way to emulate the 'greet pause' feature of
> Sendmail, whereby Postfix will kill the client connection if they send
> any commands before the 220 prompt? This seemed to be working with a 2.4
> version of Postfix that I was testing, but not with 2.6/2.7. The docs
> mention a change to the way this works, but I don't fully understand this.
I believe this feature is deprecated, but it should still work.
>
> I have the following in main.cf
>
> smtpd_client_restrictions = sleep 3, reject_unauth_pipelining,
> permit_mynetworks, reject
You probably don't want "reject" at the end.
> smtpd_delay_reject = no
This is no longer needed for proper detection of unauth
pipelining, and it causes other complications. Suggest you
remove it.
>
>
> Thanks for any help
Welcome to postfix.
-- Noel Jones
Duncan B.
>> I'm a new user to Postfix (ex Qmail user) and love it. However, there is
>> one feature of "qmqtool" that was very useful: qmqtool -i, to list queue
>> entries by IP address. Is there any way to see which IP addresses in the
>> postfix queues are the most dominant?
>
> Postfix doesn't store IPs in the queue file, so there is no such tool for
> postfix.
>
> However, the 'qshape' tool will list the queue by destination domain.
> http://www.postfix.org/QSHAPE_README.html#qshape
Thanks, this is handy but I still find it quite hard to determine which
box the mail has originated from.
>> Also I am seeing a lot of bounces in the deferred queue from
>> MAILER-DAEMON@my-host. Is there a way to instantly destroy double bounces
>> rather than defer them, or is this bad practice/against RFC standards?
>
> These aren't double bounces, just regular non-delivery reports.
>
> It's far better to find out why you have a bunch of bounces in your queue and
> fix the problem.
>
> Generally this is a result of accepting undeliverable mail. Don't do that.
> Use proper recipient validation so postfix can reject unknown recipients
> automatically.
> http://www.postfix.org/ADDRESS_CLASS_README.html
> http://www.postfix.org/LOCAL_RECIPIENT_README.html
This box is a mail relay for broadband users, so will accept all mail from
pre-defined IP ranges, and relay the mail on. I've implemented as much
checking as possible, but of course checking recipients at the end
addresses is unfortunately impossible (without doing read-ahead
checking?).
The main problems seem to be with Yahoo/BT Internet etc, they have such
strict blacklisting policies nowadays, the server spends most of it's
time deferring mail to Yahoo. Yahoo receive a few invalid recipients and
reject connections from the box. I'm not too sure how to get around this,
other than tracking down the offending senders (which I *am* doing also)!
>> Finally, is there a way to emulate the 'greet pause' feature of
>> Sendmail, whereby Postfix will kill the client connection if they send
>> any commands before the 220 prompt? This seemed to be working with a 2.4
>> version of Postfix that I was testing, but not with 2.6/2.7. The docs
>> mention a change to the way this works, but I don't fully understand this.
>
> I believe this feature is deprecated, but it should still work.
Hmm, it just seemed to stop working after I upgraded to 2.7 :( I wonder
if anyone else has experienced this?
>> I have the following in main.cf
>>
>> smtpd_client_restrictions = sleep 3, reject_unauth_pipelining,
>> permit_mynetworks, reject
>
> You probably don't want "reject" at the end.
The mailserver is firewalled off to everything other than our IP ranges,
just thought I'd be an extra measure to prevent connections from invalid
IPs should there be a firewall breach some how :-) I'll remove it though.
>> smtpd_delay_reject = no
>
> This is no longer needed for proper detection of unauth pipelining, and it
> causes other complications. Suggest you remove it.
Ok cool, thanks.
> Welcome to postfix.
Pleasure to be on board, thanks for the quick response!
Cheers
Duncan
Duncan B.
Wietse Venema
> On 9/8/2009 6:20 AM, Duncan B. wrote:
> >
> > Hi,
> >
> > Firstly this is my first post to the list, so apologies if I've not
> > correctly followed any procedures.
> >
> > one feature of "qmqtool" that was very useful: qmqtool -i, to list queue
> > entries by IP address. Is there any way to see which IP addresses in the
> > postfix queues are the most dominant?
>
> Postfix doesn't store IPs in the queue file, so there is no
> such tool for postfix.
Client information records were added late in the Postfix life
cycle, and they are used mainly by for xforward and milters.
> However, the 'qshape' tool will list the queue by destination
> domain.
> http://www.postfix.org/QSHAPE_README.html#qshape
Indeed, qshape targets outflow. Inflow analysis is easily done
based on logfile records.
Wietse
Duncan B.
On Tue, 8 Sep 2009, Wietse Venema wrote:
>> Postfix doesn't store IPs in the queue file, so there is no
>> such tool for postfix.
>
> Client information records were added late in the Postfix life
> cycle, and they are used mainly by for xforward and milters.
>
>> However, the 'qshape' tool will list the queue by destination
>> domain.
>> http://www.postfix.org/QSHAPE_README.html#qshape
>
> Indeed, qshape targets outflow. Inflow analysis is easily done
> based on logfile records.
Are you able to recommend any methods / tools to extract this information
from the log files, Wietse, or is it just a case of writing a script to do
so?
Thanks,
Duncan.
Wietse Venema
I understand that you are actually doing from-to analysis: for
example, who sends mail to Yahoo that is getting stuck in the queue.
In that case, "qshape -s" (statistics by sender domain) could help.
(Assuming that there is a relationship between sender domain and
sender IP address).
Otherwise, a qshape option for stats by IP address might be an idea.
Wietse
Noel Jones
>
>>> I'm a new user to Postfix (ex Qmail user) and love it. However, there is
>>> one feature of "qmqtool" that was very useful: qmqtool -i, to list queue
>>> entries by IP address. Is there any way to see which IP addresses in the
>>> postfix queues are the most dominant?
>>
>> for postfix.
>>
>> http://www.postfix.org/QSHAPE_README.html#qshape
>
> box the mail has originated from.
Ah, I was thinking destination rather than origin.
There are several tools to extract historical information from
the logs, I suggest postfix-logwatch and/or pflogsumm.
Or I suppose it's possible to hack qshape to retrieve client
information from queue files if you want to try that.
>
>
>>> Also I am seeing a lot of bounces in the deferred queue from
>>> MAILER-DAEMON@my-host. Is there a way to instantly destroy double
>>> bounces
>>> rather than defer them, or is this bad practice/against RFC standards?
>>
>> These aren't double bounces, just regular non-delivery reports.
>>
>> It's far better to find out why you have a bunch of bounces in your
>> queue and fix the problem.
>>
>> Generally this is a result of accepting undeliverable mail. Don't do
>> that. Use proper recipient validation so postfix can reject unknown
>> recipients automatically.
>> http://www.postfix.org/ADDRESS_CLASS_README.html
>> http://www.postfix.org/LOCAL_RECIPIENT_README.html
>
> This box is a mail relay for broadband users, so will accept all mail
> from pre-defined IP ranges, and relay the mail on. I've implemented as
> much checking as possible, but of course checking recipients at the end
> addresses is unfortunately impossible (without doing read-ahead checking?).
OK, you didn't explain the problem fully. If your users are
sending lots of undeliverable mail, you'll need to take that
up with your users.
Postfix has a bounce_queue_lifetime parameter that can adjust
how long a bounce hangs around in the queue. The default is 5
days - you can set is somewhat smaller, but I would hesitate
to set it less than one or two days.
http://www.postfix.org/postconf.5.html#bounce_queue_lifetime
>>> Finally, is there a way to emulate the 'greet pause' feature of
>>> Sendmail, whereby Postfix will kill the client connection if they send
>>> any commands before the 220 prompt? This seemed to be working with a 2.4
>>> version of Postfix that I was testing, but not with 2.6/2.7. The docs
>>> mention a change to the way this works, but I don't fully understand
>>> this.
>>
>> I believe this feature is deprecated, but it should still work.
>
> Hmm, it just seemed to stop working after I upgraded to 2.7 :( I wonder
> if anyone else has experienced this?
(I wonder if anyone else is using it.) This feature is a
great tool for self-DoS.
>
>
>>> I have the following in main.cf
>>>
>>> smtpd_client_restrictions = sleep 3, reject_unauth_pipelining,
>>> permit_mynetworks, reject
>>
>> You probably don't want "reject" at the end.
>
> The mailserver is firewalled off to everything other than our IP ranges,
> just thought I'd be an extra measure to prevent connections from invalid
> IPs should there be a firewall breach some how :-) I'll remove it though.
I was assuming this was a general purpose MTA; apparently it's
a user submission point only - MSA. In that case, the final
reject is fine, and probably a good idea.
-- Noel Jones
Duncan B.
I think a qshape option to view stats per IP address would be brilliant,
if possible!!
Cheers,
Duncan.
Wietse Venema
Why are "stats by sender domain" not sufficient?
Wietse
Victor Duchovni
The "qshape" script is relatively straight-forward, if somewhat
under-commented, Perl code. It would not be difficult to parse "attribute"
records and extract "client" information. Also useful could be stats
by the full sender address *with* the local part, so that an outbound
system administrator can pin-point the *user* causing congestion when
all senders have the same domain, but differ in the local-part of the
address.
--
Viktor.
Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.
To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:majo...@postfix.org?body=unsubscribe%20postfix-users>
If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Duncan B.
>>>> Are you able to recommend any methods / tools to extract this information
>>>> from the log files, Wietse, or is it just a case of writing a script to do
>>>> so?
>>>
>>> I understand that you are actually doing from-to analysis: for
>>> example, who sends mail to Yahoo that is getting stuck in the queue.
>>>
>>> In that case, "qshape -s" (statistics by sender domain) could help.
>>> (Assuming that there is a relationship between sender domain and
>>> sender IP address).
>>>
>>> Otherwise, a qshape option for stats by IP address might be an idea.
>>
>> I think a qshape option to view stats per IP address would be brilliant,
>> if possible!!
>
> Why are "stats by sender domain" not sufficient?
They are certainly very useful, but maybe the stats by IP address would be
easier to see which machines are sending through the relay at a glance (As
an IP address can send from any domain, in theory? The translation from
sender domain to actual client IP address would mean trawling through the
logs (as it wouldn't necessarily be the same as the domain's MX record) ?
If it can't be done easily, then that's fine. It is the *only* feature of
Qmail I do miss a little though :-) But then the positive features
outweigh that tenfold.
Cheers
Duncan