Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
BIND Performance with Huge RPZ
1,331 views
Skip to first unread message
Arie L. Putra
Jul 12, 2013, 6:11:15 AM7/12/13
to bind-...@lists.isc.org
Dear All,
We are building a server for recursive DNS Server, this server will be acted as a cache for our network. (several user-side DNS Server will forward to this server)
Using Ubuntu Server with latest BIND version, we are trying to have RPZ incuded in this BIND, with around 800k blacklisted sites.
Has anyone have experience, how RPZ with huge list will impact BIND performance, will it reduce DNS response time? we have six DNS server that will point to this server, each server is serving about 15Mbps of DNS Traffic on peak hour.
this server is a Ubuntu box with 2 Xeon (total of 12 core, 24 if include HT), 16GB RAM.
Appreciate any information given.
Best Regards,
Arie Lendra Putra
Steven Carr
Jul 12, 2013, 6:29:32 AM7/12/13
to bind-users
On 12 July 2013 11:11, Arie L. Putra <ari...@smartfren.com> wrote:
>
> Has anyone have experience, how RPZ with huge list will impact BIND performance, will it reduce DNS response time? we have six DNS server that will point to this server, each server is serving about 15Mbps of DNS Traffic on peak hour.
>
> this server is a Ubuntu box with 2 Xeon (total of 12 core, 24 if include HT), 16GB RAM.
>
> Has anyone have experience, how RPZ with huge list will impact BIND performance, will it reduce DNS response time? we have six DNS server that will point to this server, each server is serving about 15Mbps of DNS Traffic on peak hour.
>
> this server is a Ubuntu box with 2 Xeon (total of 12 core, 24 if include HT), 16GB RAM.
It's very difficult to predict the impact on performance in general. But every query will hit the RPZ zone to be checked against, so make sure you have enough RAM to hold all 800k records in memory then that will significantly speed things up.
The guidance figures that I've seen banded around by a BIND based DNS appliance is that it will have ~30% impact on query performance per RPZ feed that has to be looked up against.
SteveThe guidance figures that I've seen banded around by a BIND based DNS appliance is that it will have ~30% impact on query performance per RPZ feed that has to be looked up against.
Phil Mayers
Jul 12, 2013, 6:34:43 AM7/12/13
to bind-...@lists.isc.org
On 12/07/13 11:11, Arie L. Putra wrote:
> Has anyone have experience, how RPZ with huge list will impact BIND
> performance, will it reduce DNS response time? we have six DNS server
> that will point to this server, each server is serving about 15Mbps of
> DNS Traffic on peak hour.
We don't have that kind of load, but we do have a large (~550k) RPZ
> Has anyone have experience, how RPZ with huge list will impact BIND
> performance, will it reduce DNS response time? we have six DNS server
> that will point to this server, each server is serving about 15Mbps of
> DNS Traffic on peak hour.
setup. It doesn't seem to have any noticeable performance impact,
although I should note we're running bind 9.9.2 with the RRL+RPZ
patches, which contain some RPZ performance improvements in certain configs:
http://ss.vix.su/~vjs/rrlrpz.html
Our query load is in the 400-800qps range, with occasional spikes to
>1500qps.
I had a few problems with RPZ in the past, and it was suggested that our
using bind 9.8 (at the time) might have been an issue; we never
determined the exact cause, but they don't seem to have recurred on 9.9.
G.W. Haywood
Jul 12, 2013, 11:11:50 AM7/12/13
to bind-...@lists.isc.org
Hi there,
On Fri, 12 Jul 2013, Arie L. Putra wrote:
> We are building a server for recursive DNS Server, this server will
> be acted as a cache for our network. (several user-side DNS Server
> will forward to this server) Using Ubuntu Server with latest BIND
> version, we are trying to have RPZ incuded in this BIND, with around
> 800k blacklisted sites.
>
Using the ipsets extension to iptables, on very modest hardware, we
routinely block over one thousand million IP addresses with negligible
impact on performance. I understand that it's not the same thing at
all, but I still wonder if it might be of some use to you.
--
73,
Ged.
On Fri, 12 Jul 2013, Arie L. Putra wrote:
> We are building a server for recursive DNS Server, this server will
> be acted as a cache for our network. (several user-side DNS Server
> will forward to this server) Using Ubuntu Server with latest BIND
> version, we are trying to have RPZ incuded in this BIND, with around
> 800k blacklisted sites.
>
> Has anyone have experience, how RPZ with huge list will impact BIND
> performance, will it reduce DNS response time? we have six DNS
> server that will point to this server, each server is serving about
> 15Mbps of DNS Traffic on peak hour.
I wonder if you've considered using iptables in addition to BIND/RPZ?
> performance, will it reduce DNS response time? we have six DNS
> server that will point to this server, each server is serving about
> 15Mbps of DNS Traffic on peak hour.
Using the ipsets extension to iptables, on very modest hardware, we
routinely block over one thousand million IP addresses with negligible
impact on performance. I understand that it's not the same thing at
all, but I still wonder if it might be of some use to you.
--
73,
Ged.
Chris Buxton
Jul 12, 2013, 12:13:50 PM7/12/13
to Arie L. Putra, bind-...@lists.isc.org
On Jul 12, 2013, at 3:11 AM, Arie L. Putra <ari...@smartfren.com> wrote:
> We are building a server for recursive DNS Server, this server will be acted as a cache for our network. (several user-side DNS Server will forward to this server)
> Using Ubuntu Server with latest BIND version, we are trying to have RPZ incuded in this BIND, with around 800k blacklisted sites.
>
> Has anyone have experience, how RPZ with huge list will impact BIND performance, will it reduce DNS response time? we have six DNS server that will point to this server, each server is serving about 15Mbps of DNS Traffic on peak hour.
>
> We are building a server for recursive DNS Server, this server will be acted as a cache for our network. (several user-side DNS Server will forward to this server)
> Using Ubuntu Server with latest BIND version, we are trying to have RPZ incuded in this BIND, with around 800k blacklisted sites.
>
> Has anyone have experience, how RPZ with huge list will impact BIND performance, will it reduce DNS response time? we have six DNS server that will point to this server, each server is serving about 15Mbps of DNS Traffic on peak hour.
>
> this server is a Ubuntu box with 2 Xeon (total of 12 core, 24 if include HT), 16GB RAM.
I've seen well over 1 million entries in an RPZ. The performance impact with BIND 9.8 was noticeable but not horrible. The memory requirements were roughly 300 MB for this one zone, compared to over 3 GB for the equivalent in the form of somewhere north of 500 thousand individual zones (two A records each, for the zone apex and a wildcard, all loading from the same file).
I'm not used to considering DNS traffic in terms of Mb/s (nor MB/s). I'm more used to considering q/s. The servers with the aforementioned RPZ each handled a relatively large number of queries, possibly as high as 20Kq/s. In my experience, it's impossible to know how a given server will perform without seeing all of the configuration, as lots of configuration settings can impact performance. Once such example is query logging to file (instead of to syslog), which can completely gut performance.
Regards,
Chris Buxton
BLUECAT
Vernon Schryver
Jul 12, 2013, 12:31:05 PM7/12/13
to bind-...@lists.isc.org
> From: Steven Carr <sjc...@gmail.com>
> It's very difficult to predict the impact on performance in general.
Yes, the reasonable tactic is to build a representative list of
queries from your query logs and use queryperf to hit a test server
with those 800K policy zone labels.
> But
> every query will hit the RPZ zone to be checked against, so make sure you
> have enough RAM to hold all 800k records in memory then that will
> significantly speed things up.
RAM for all 800K records would be sufficient, but might not unnecessary.
Names in policy zones are kept in the same kind of red/black trees
that are used for ordinary DNS zones, because policy zones are ordinary
BIND9 DNS zones. Policy zone IP addresses are kept in radix or patricia
trees. Sufficent RAM is the size of the working set. For example,
if you have 800K labels below obscure.example.com and almost never
resolve any of them, then keeping them in RAM would not help your
server's average performance.
> The guidance figures that I've seen banded around by a BIND based DNS
> appliance is that it will have ~30% impact on query performance per RPZ
> feed that has to be looked up against.
RPZ performance with zones with NSIP or NSDNAME policies is significantly
improved in versions of the RPZ code with the new min-ns-dots parameter
with a default value 1. This turns off checking the many server names
and IP addresses of TLDs.
The version after that has min-ns-dots and also no longer hits every
policy zone with every IP address and name in every response, but
instead hits a summary red/black tree of names or radix tree of IP
addresses summarizing all policy zones once for each address and name.
This radically improves RPZ performance with multiple policy zones.
The BIND9 tests in bin/tests/system/rpz include a quick and dirty
performance test to ensure that nothing is not unexpectedly broken.
This is typical output from that test:
I:checking performance with RPZ
I:checking performance without RPZ
I:17758 qps with RPZ is 75% of 23596 qps without RPZ
Previous versions of that test did not turn off some very expensive
run time checks and so produced much smaller numbers for both cases.
Patches for both of those versions of RPZ speed improvements for some
BIND9 releases can be with the BIND RRL patches by following the link
labeled "Patch files for BIND9" on http://www.redbarn.org/dns/ratelimits
Both of those versions are or will be in official BIND releases.
I've lost track of which releases have or will have which of those
two RPZ sets of performance improvements.
Vernon Schryver v...@rhyolite.com
> It's very difficult to predict the impact on performance in general.
queries from your query logs and use queryperf to hit a test server
with those 800K policy zone labels.
> But
> every query will hit the RPZ zone to be checked against, so make sure you
> have enough RAM to hold all 800k records in memory then that will
> significantly speed things up.
Names in policy zones are kept in the same kind of red/black trees
that are used for ordinary DNS zones, because policy zones are ordinary
BIND9 DNS zones. Policy zone IP addresses are kept in radix or patricia
trees. Sufficent RAM is the size of the working set. For example,
if you have 800K labels below obscure.example.com and almost never
resolve any of them, then keeping them in RAM would not help your
server's average performance.
> The guidance figures that I've seen banded around by a BIND based DNS
> appliance is that it will have ~30% impact on query performance per RPZ
> feed that has to be looked up against.
improved in versions of the RPZ code with the new min-ns-dots parameter
with a default value 1. This turns off checking the many server names
and IP addresses of TLDs.
The version after that has min-ns-dots and also no longer hits every
policy zone with every IP address and name in every response, but
instead hits a summary red/black tree of names or radix tree of IP
addresses summarizing all policy zones once for each address and name.
This radically improves RPZ performance with multiple policy zones.
The BIND9 tests in bin/tests/system/rpz include a quick and dirty
performance test to ensure that nothing is not unexpectedly broken.
This is typical output from that test:
I:checking performance with RPZ
I:checking performance without RPZ
I:17758 qps with RPZ is 75% of 23596 qps without RPZ
Previous versions of that test did not turn off some very expensive
run time checks and so produced much smaller numbers for both cases.
Patches for both of those versions of RPZ speed improvements for some
BIND9 releases can be with the BIND RRL patches by following the link
labeled "Patch files for BIND9" on http://www.redbarn.org/dns/ratelimits
Both of those versions are or will be in official BIND releases.
I've lost track of which releases have or will have which of those
two RPZ sets of performance improvements.
Vernon Schryver v...@rhyolite.com
Noel Butler
Jul 12, 2013, 10:50:35 PM7/12/13
to bind-...@lists.isc.org
On Fri, 2013-07-12 at 16:31 +0000, Vernon Schryver wrote:
Patches for both of those versions of RPZ speed improvements for some BIND9 releases can be with the BIND RRL patches by following the link labeled "Patch files for BIND9" on http://www.redbarn.org/dns/ratelimits Both of those versions are or will be in official BIND releases. I've lost track of which releases have or will have which of those two RPZ sets of performance improvements.
9.9.4 will have them
Arie L. Putra
Jul 12, 2013, 10:58:35 PM7/12/13
to Chris Buxton, bind-...@lists.isc.org
Hi,
Thanks for the information, really appreciate it,
In qps term, each of my server around 10k qps.
Currentlt now we are just using simple query logging to file, no syslog yet :(
BR,
Arie L. Putra
陈维文
Thanks for the information, really appreciate it,
In qps term, each of my server around 10k qps.
Currentlt now we are just using simple query logging to file, no syslog yet :(
BR,
Arie L. Putra
陈维文
From: Chris Buxton
Sent: 7/12/2013 11:12 PM
To: Arie L.Putra
Cc: bind-...@lists.isc.org
Subject: Re: BIND Performance with Huge RPZ
Vernon Schryver
Jul 13, 2013, 1:01:02 AM7/13/13
to bind-...@lists.isc.org
> From: Noel Butler <noel....@ausics.net>
> > BIND9 releases can be with the BIND RRL patches by following the link
> > labeled "Patch files for BIND9" on http://www.redbarn.org/dns/ratelimits
> >
> > Both of those versions are or will be in official BIND releases.
> > I've lost track of which releases have or will have which of those
> > two RPZ sets of performance improvements.
>
> 9.9.4 will have them
Not exactly.
Checking source finds the min-ns-dots speed-up in 9.8.5-P1, 9.9.3-P1,
and 9.9.4b1 and so in 9.9.4. On the other hand, I think the more
substantial set of RPZ speed improvements for multiple policy zones
is in none of those and so will not be in 9.9.4. My bet would be
on 9.10 along with client IP address triggers and "drop" and
"truncate" actions. I think the multiple zone speed-up is in the
subscription-only 9.9.4-S and so will be in 9.9.4-S1.
Vernon Schryver v...@rhyolite.com
> > BIND9 releases can be with the BIND RRL patches by following the link
> > labeled "Patch files for BIND9" on http://www.redbarn.org/dns/ratelimits
> >
> > Both of those versions are or will be in official BIND releases.
> > I've lost track of which releases have or will have which of those
> > two RPZ sets of performance improvements.
>
> 9.9.4 will have them
Checking source finds the min-ns-dots speed-up in 9.8.5-P1, 9.9.3-P1,
and 9.9.4b1 and so in 9.9.4. On the other hand, I think the more
substantial set of RPZ speed improvements for multiple policy zones
is in none of those and so will not be in 9.9.4. My bet would be
on 9.10 along with client IP address triggers and "drop" and
"truncate" actions. I think the multiple zone speed-up is in the
subscription-only 9.9.4-S and so will be in 9.9.4-S1.
Vernon Schryver v...@rhyolite.com
0 new messages