Understanding rate limiting with anvil.
Jaroslaw Grzabel
I had a problem with rate limiting in postfix in last week, and I had to
disable it. I don't know why but I set up rate limiting per IP and it
looks like it didn't count rates correctly. Let me shed a lit a bit for
my problem.
My company decided to limit a number of messages per day per IP. That's
why we run anvil and limit 1000msg/24h/IP, so I put into main.cf:
smtpd_client_event_limit_exceptions = /etc/postfix/ratelimiting
smtpd_client_message_rate_limit = 1000
anvil_rate_time_unit=1d
anvil_status_update_time=600s
File /etc/postfix/ratelimiting contains:
!aaa.bbb.ccc.ddd/29
!xxx.yyy.zzz.qqq/28
...
...
...
!mmm.nnn.ppp.ooo
!ddd.ee.f.xxx
0.0.0.0/0
What means limit everyone besides IP's with "!" on the front because I
excluded all my clients, and put 0.0.0.0/0 on the end what should mean
count everyone besides... The problem which occurred in last week was
that one of our customers complains that he received a message during
connection that connection limit has been exceeded. What shouldn't be
right because as I counted "manually" he sends about 350 messages per
day and the limit was 1000. The worst problem was that he was included
to EXCLUDE list, so his IP shouldn't be taken into consideration and
counted.
I presume syntax in exclude list is incorrect or I don't understand how
anvil works at all.
Thank you for any help in advance.
Regards,
JG
Stefan Förster
> I had a problem with rate limiting in postfix in last week, and I had to
> disable it. I don't know why but I set up rate limiting per IP and it
> looks like it didn't count rates correctly. Let me shed a lit a bit for
> my problem.
>
> My company decided to limit a number of messages per day per IP. That's
> why we run anvil and limit 1000msg/24h/IP, so I put into main.cf:
>
> smtpd_client_event_limit_exceptions = /etc/postfix/ratelimiting
"Clients that are excluded from connection count, connection rate,
or"... "excluded".
> !aaa.bbb.ccc.ddd/29
This means: Don't exclude aaa.bbb.ccc.ddd/29 from rate limiting, i.e.
enforce rate limiting for aaa.bbb.ccc.ddd/29.
This means: Exclude the whole world from rate limiting.
> I presume syntax in exclude list is incorrect or I don't understand how
> anvil works at all.
I think you need Postfix 2.4 or later to work with that bang-style
notation.
Cheers
Stefan
Jaroslaw Grzabel
> "Clients that are excluded from connection count, connection rate,
> or"... "excluded".
>
from the list besides rest of the world! :).
> I think you need Postfix 2.4 or later to work with that bang-style
> notation.
>
I think it works with postfix 2.3.3. I've checked that and I think it works.
Last question. I need to limit each IP and from issues I had I suppose
that 192.168.0.1 = 0.0.0.0/0 and 192.168.2.222 = 0.0.0.0/0 ? Am I right ?
I need to limit messages per unique IP and I don't want to set 1000 for
everyone else than excluded IP's on the list.
>
> Cheers
> Stefan
>
Thank you for your help.
Regards,
JG
Wietse Venema
> My company decided to limit a number of messages per day per IP. That's
> why we run anvil and limit 1000msg/24h/IP, so I put into main.cf:
Don't use anvil(8) for long-term statistics.
ANVIL(8) ANVIL(8)
...
The anvil(8) server maintains an in-memory table with information about
recent clients requests. No persistent state is kept because standard
system library routines are not sufficiently robust for update-inten-
sive applications.
Although the in-memory state is kept only temporarily, this may require
a lot of memory on systems that handle connections from many remote
clients. To reduce memory usage, reduce the time unit over which state
is kept.
Instead, use a policy daemon with persistent database.
Wietse
Jaroslaw Grzabel
Wietse Venema wrote:
> Instead, use a policy daemon with persistent database.
>
As far as I remember I tried once http://www.policyd.org, but it didn't
work for me. As I remember there was a problem with message accounting,
and I have got an email from Nigel from policyd where he confirmed that
for example 192.168.0.2 is 0.0.0.0/0 and 192.111.111.111 is also
0.0.0.0/0 so IP is not unique in accounting as long as you will not
specify each IP address you want to limit.
He's saying:
"
Adding of "external" acl lists is definitely something that will be
implemented. This requires the caching to work, which will with a bit of
tuning.
I'd say the ETA on that feature is about 3 months or so, maybe 6.
Just waiting until I have a bit of time to implement it, or if someone
is willing to speed up development by sponsoring :D
-N
"
> Wietse
>
Or maybe you have different policy daemon in mind ?
Regards,
Jarek
Noel Jones
> Hi,
>
> Wietse Venema wrote:
>> Instead, use a policy daemon with persistent database.
>>
> As far as I remember I tried once http://www.policyd.org, but it didn't
> work for me. As I remember there was a problem with message accounting,
> Or maybe you have different policy daemon in mind ?
>
The point is that you are using anvil in a manner contrary to
its documented intent; don't be surprised if it doesn't work
the way you expect it to.
As documented:
- anvil must not be used for quota or traffic shaping; it's
intent is a last-resort DoS protection tool.
- anvil keeps the table of known IPs and counts in memory;
this is not suitable for more than a few minutes worth of data.
The proper solution for your requested policy is to use an
external policy server or milter that keeps a database of
connections, or maybe a traffic-shaping firewall feature.
I don't have a suggestion for what you should use, but I know
anvil just isn't suitable.
-- Noel Jones
Ing. Davy Leon
2.3.3 with amavisd-new and clamav running.
Cheers
David
Eduardo Júnior
On Wed, Aug 5, 2009 at 1:33 PM, Ing. Davy Leon<da...@scu.escambray.com.cu> w=
rote:
I use postfwd [1] for that
And it works well for me.
To build the rules is necessary time, just read the documentation [2]
[1] - http://postfwd.org/
[2] - http://postfwd.org/doc.html
[]'s
--=20
Eduardo J=FAnior
GNU/Linux user #423272
:wq
Eduardo Júnior
2009/8/5 Eduardo J=FAnior <ihtr...@gmail.com>:
> Hi,
>
>
> On Wed, Aug 5, 2009 at 1:33 PM, Ing. Davy Leon<da...@scu.escambray.com.cu>=
wrote:
>> Wich policy daemon would you guys recommend to install? I have a Postfix
>> 2.3.3 with amavisd-new and clamav running.
>>
>> Cheers
>>
>> David
>
>
> I use postfwd [1] for that
> And it works well for me.
> To build the rules is necessary time, just read the documentation [2]
>
> [1] - http://postfwd.org/
> [2] - http://postfwd.org/doc.html
just a detail:
With postfwd, i didn't get control how many messages a IP can to send.
But my control have done through number of connections from host.
If you consider that a connection is equal a message, this is ok.
Jaroslaw Grzabel
> > Hi,
>
Hi,
> > just a detail:
> >
> > With postfwd, i didn't get control how many messages a IP can to send.
> > But my control have done through number of connections from host.
> > If you consider that a connection is equal a message, this is ok.
> >
>
That may be a big problem when using greylisting because in this case
number of connections will never be the same as sent messages :( . You
can't even assume that number_of_connection/2 = number_of_sent_messages.
I'm also looking for some good solution which will solve my problem and
safe RAM :) .
> > []'s
>
P.S. Sorry Eduardo I replied to priv.
Regards,
Jarek