hardware requirements per hits
Alans
--===============6433942104918916377==
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_002E_01CA1E7C.A18D7220"
Content-Language: en-us
This is a multi-part message in MIME format.
------=_NextPart_000_002E_01CA1E7C.A18D7220
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
Hi,
I want to know when we need hardware upgrade.
How many queries will use 50% of cpu and memory?
Regards,
Alans
------=_NextPart_000_002E_01CA1E7C.A18D7220
Content-Type: text/html;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
xmlns=3D"http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3DContent-Type content=3D"text/html; =
charset=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3DEN-US link=3Dblue vlink=3Dpurple>
<div class=3DSection1>
<p class=3DMsoNormal>Hi,<o:p></o:p></p>
<p class=3DMsoNormal><o:p> </o:p></p>
<p class=3DMsoNormal>I want to know when we need hardware =
upgrade.<o:p></o:p></p>
<p class=3DMsoNormal>How many queries will use 50% of cpu and =
memory?<o:p></o:p></p>
<p class=3DMsoNormal><o:p> </o:p></p>
<p class=3DMsoNormal>Regards,<o:p></o:p></p>
<p class=3DMsoNormal>Alans<o:p></o:p></p>
</div>
</body>
</html>
------=_NextPart_000_002E_01CA1E7C.A18D7220--
--===============6433942104918916377==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
bind-users mailing list
bind-...@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
--===============6433942104918916377==--
Alans
Doug Barton
> Hi,
>
>
>
> I want to know when we need hardware upgrade.
>
> How many queries will use 50% of cpu and memory?
FYI this question is impossible to answer without a lot more details.
Doug
Alans
I guess number of hits is related to increase in CPU usage, for example 1000
simultaneously hits will make cpu usage 10% and so on, I need that number.
Regards,
Alans
Doug Barton
Alans
I guess number of hits is related to increase in CPU usage, for example 1000
simultaneously hits will make cpu usage 10% and so on, I need that number.
Regards,
Alans
-----Original Message-----
From: Doug Barton [mailto:do...@dougbarton.us]
Sent: Monday, August 17, 2009 7:08 AM
To: Alans
Cc: bind-...@lists.isc.org
Subject: Re: hardware requirements per hits
Doug
_______________________________________________
Matus UHLAR - fantomas
> Ok, please tell me what details you need.
>
> I guess number of hits is related to increase in CPU usage, for example 1000
> simultaneously hits will make cpu usage 10% and so on, I need that number.
I'm afraid this ias jusst useless question. The only usefull question is
what hardware you need to be able to process your traffic.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Support bacteria - they're the only culture some people have.
Matus UHLAR - fantomas
Bill Larson
> @Matus: let me put it in this way, if I want to create a budget for next
> year for example, then I should know what upgrades I need for next year
> (estimated needs), and let's assume dns queries increase monthly by x hits,
> now, if I know how many hits will make me upgrade cpu and memory then I can
> find out my cpu and memory needs for next year, hope this explain to you
why
> my question is not "usless", at least for me.
> I'll be happy if you tell me another way to know my needs for next year.
I won't necessarily go so far as to say this question is useless, but it is
almost impossible to answer.
You aren't telling us what you current situation is; what you are seeing for
a query load, whether you are running a server for authoritative data or a
resolving server, if you are using DNSSEC, etc. This is to say nothing
about your current hardware; CPU, memory, network/Internet connectivity,
etc. Also, are your running anything else on the same platform as your DNS
server?
My best suggestion is to test your environment yourself. Run "queryperf"
(or, Nominum's "dnsperf", for authoritative servers, and "resperf", for
caching resolvers, tools) to determine what sort of level your servers can
support. Then, compare this result to the level of service that you are
currently seeing. This will give you an idea of what level of service your
systems can provide. If the maximum performance cannot meet your
expectations, then you need an upgrade. If they do meed your expectations,
then you are ok. Simple enough.
But, some questions that only you can answer for your situation. Who is
querying your server? What queries are you currently receiving and
answering? How are your servers currently performing? What is the bottle
neck that you are seeing? Is it CPU? Memory? Disk I/O? Network?
(Bottleneck is sort of a bad term here. A better phrase is "What part of
the system is limiting your performance?") How long does it take for your
server to completely start AND is this a problem for you? (More zones, and
large zones, makes for a slower startup. But, if it starts fast enough for
you then it is acceptable.)
I used to run a major DNS server on a microVAX II handling 500 queries per
second (a LONG time ago). BIND doesn't necessarily require much CPU. It
can require lots of memory. (And if you are running other services on the
same system, then these other services have to share memory with your DNS
server.) Rarely, is disk I/O an issue, unless you are seeing excessive
swapping/paging, which says you need more memory. Network or Internet
connectivity isn't normally an issue either (DNS traffic doesn't have to be
excessive if your systems are reasonably configured.) If you are using
DNSSEC, or are planning on it, you can expect to use more CPU.
Again, all of these things are things that you can determine yourself by
testing your server performance. I remember a quote from somewhere a long
time ago. "If you don't know how your system is running now when things are
good, how do you expect to be able to say what is causing a problem in the
future when things are bad?" (I suspect that this cam from "System
Performance Tuning" by Mike Loukides, O'Reilly & Assc. My copy is quite old
but still useful.) Know how your system is performing BEFORE there is a
problem.
Bill Larson
Alans
year for example, then I should know what upgrades I need for next year
(estimated needs), and let's assume dns queries increase monthly by x hits,
now, if I know how many hits will make me upgrade cpu and memory then I can
find out my cpu and memory needs for next year, hope this explain to you why
my question is not "usless", at least for me.
I'll be happy if you tell me another way to know my needs for next year.
Alans
-----Original Message-----
From: bind-user...@lists.isc.org
[mailto:bind-user...@lists.isc.org] On Behalf Of Matus UHLAR -
fantomas
Sent: Monday, August 17, 2009 12:18 PM
To: bind-...@lists.isc.org
Subject: Re: hardware requirements per hits
On 17.08.09 08:43, Alans wrote:
> Ok, please tell me what details you need.
>
> I guess number of hits is related to increase in CPU usage, for example
1000
> simultaneously hits will make cpu usage 10% and so on, I need that number.
I'm afraid this ias jusst useless question. The only usefull question is
what hardware you need to be able to process your traffic.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Support bacteria - they're the only culture some people have.
Alans
Bill Larson
> @Matus: let me put it in this way, if I want to create a budget for next
> year for example, then I should know what upgrades I need for next year
> (estimated needs), and let's assume dns queries increase monthly by x hits,
> now, if I know how many hits will make me upgrade cpu and memory then I can
> find out my cpu and memory needs for next year, hope this explain to you
why
> my question is not "usless", at least for me.
> I'll be happy if you tell me another way to know my needs for next year.
I won't necessarily go so far as to say this question is useless, but it is
Kevin Darcy
There are just so many variables here. Are all the clients looking up
essentially the *same* names? Or are the new clients looking up
*different* names than the old ones? This has an impact on cache hit
ratio, which then has an cascading impact on network traffic, query
residency, latency, etc.
Will the next big Internet fad involve 5 DNS names? Or 50? Or 250? How
frequently will those names be looked up, and, is the target app
conducive to DNS-based load-balancing with low TTL values?
What OS are you running on? Does it scale well with volume? How about
the network implementation? Is it sockets-based? How well does it manage
its buffers? Do you have competent administrators watching your systems,
tuning them for performance? Proactively? Reactively?
Would you consider jumping ship to another OS that scales better?
Consider the migration costs and logistics.
What about the option of buying more servers as volume increases, rather
than trying to upgrade the existing servers? Do you have rack space?
Electrical? Air conditioning? Backup generator? Site licenses? How's
your monitoring and event-management infrastructure? Can it handle an
influx of new servers? Are your client apps well-behaved enough that
they can tolerate a failover from one resolver to another in case of a
single-point failure? How about two failovers, in the case of a
multiple-point failure? Are your network architects competent enough to
implement "anycast"?
I could go on...
- Kevin
Alans wrote:
> @Matus: let me put it in this way, if I want to create a budget for next
> year for example, then I should know what upgrades I need for next year
> (estimated needs), and let's assume dns queries increase monthly by x hits,
> now, if I know how many hits will make me upgrade cpu and memory then I can
> find out my cpu and memory needs for next year, hope this explain to you why
> my question is not "usless", at least for me.
> I'll be happy if you tell me another way to know my needs for next year.
>
> Alans
>
> -----Original Message-----
> From: bind-user...@lists.isc.org
> [mailto:bind-user...@lists.isc.org] On Behalf Of Matus UHLAR -
> fantomas
> Sent: Monday, August 17, 2009 12:18 PM
> To: bind-...@lists.isc.org
> Subject: Re: hardware requirements per hits
>
> On 17.08.09 08:43, Alans wrote:
>
>> Ok, please tell me what details you need.
>>
>> I guess number of hits is related to increase in CPU usage, for example
>>
> 1000
>
>> simultaneously hits will make cpu usage 10% and so on, I need that number.
>>
>
> I'm afraid this ias jusst useless question. The only usefull question is
> what hardware you need to be able to process your traffic.
>
>
_______________________________________________
Kevin Darcy
Fajar A. Nugraha
> @Matus: let me put it in this way, if I want to create a budget for next
> year for example, then I should know what upgrades I need for next year
> (estimated needs), and let's assume dns queries increase monthly by x hits,
> now, if I know how many hits will make me upgrade cpu and memory then I can
> find out my cpu and memory needs for next year, hope this explain to you why
> my question is not "usless", at least for me.
> I'll be happy if you tell me another way to know my needs for next year.
I'm assuming you already have a running DNS server? In that case I'd
simply gather stats from it. What kind of hardware it currently has,
how much is current CPU and disk load, how many queries per second it
currently serves, etc. Based on that you can have a rough estimate as
to what you'd need to upgrade.
Here are some pointers from my experience though:
- syslog query logging is expensive. NEVER enable it. If you need to
log client queries, log it directly to file instead.
- disk I/O can be a serious bottleneck. If that's the case consider
disable logging.
- BIND would generally work better with faster CPU compared to
multiple CPUs/cores, e.g. 1 x 3GHZ CPU could outperform 2 x 1.5GHz
CPU.
- memory cache can speedup things to a point. Try allocating about
2-4G when you're handing lots of clients.
Those are very general pointers though, YMMV. You might find it easier
to simply add aonther server instead of upgrading.
--
Fajar
Fajar A. Nugraha
--
Fajar
Alans
Thanks for your interesting explanations.
I assumed that Bind (alone and regardless of other variables), I assumed it
has a limitation for a certain number of hits per hardware.
After reading your reply, now I understand that Bind can handle same load on
less hardware requirement but in different environment, please correct me if
I'm wrong.
I admit when I said hit I didn't think of difference between cached and
other types so thanks again.
Regards,
Alans
-----Original Message-----
From: Fajar A. Nugraha [mailto:fa...@fajar.net]
Sent: Tuesday, August 18, 2009 6:15 AM
To: Alans
Cc: bind-...@lists.isc.org
Subject: Re: hardware requirements per hits
Alans
Regards,
Alans
--
Fajar
_______________________________________________
sth...@nethelp.no
> debating sending query logs to a remote syslog server to enhance some
> security tools. We are running BIND 9.6.1-P1 with multithreading enabled
> on RHEL 4 (2 dual-core 2.8 GHz Opterons with 1MB cache, 4G of RAM). I
> have run some tests and while there is some queries/sec hit, the RTTs
> are not terrible.
For small query rates it probably won't matter. However, remote syslog
could just about double the number of packets transmitted from your name
server. For higher query rates, this likely to be noticeable.
You might want to consider running a packet sniffer (tcpdump, wireshark
etc) instead, to capture the DNS queries and answers. Advantages:
- You get both queries and answers
- The actual DNS decoding can be done offline, as needed
- If you mirror the traffic from a switch, the whole process can be
completely offloaded from the name server
- The name server isn't forced to do something it wasn't built for
Steinar Haug, Nethelp consulting, sth...@nethelp.no
sth...@nethelp.no
Subhan Malick
> Here are some pointers from my experience though:
> - syslog query logging is expensive. NEVER enable it. If you need to
> log client queries, log it directly to file instead.
I would like to hear more about why this is so. We are currently
debating sending query logs to a remote syslog server to enhance some
security tools. We are running BIND 9.6.1-P1 with multithreading enabled
on RHEL 4 (2 dual-core 2.8 GHz Opterons with 1MB cache, 4G of RAM). I
have run some tests and while there is some queries/sec hit, the RTTs
are not terrible.
---------------------------------------------------------------------
queryperf -d input/queries.log.34 -s hidden.server -q 200 -l 300 -b 32
running 9.6.1-P1 with querylogging on and directed to remote syslog
Statistics:
Parse input file: multiple times
Run time limit: 300 seconds
Ran through file: 0 times
Queries sent: 746189 queries
Queries completed: 739501 queries
Queries lost: 6688 queries
Queries delayed(?): 0 queries
RTT max: 5.003000 sec
RTT min: 0.000197 sec
RTT average: 0.036471 sec
RTT std deviation: 0.204566 sec
RTT out of range: 1 queries
Percentage completed: 99.10%
Percentage lost: 0.90%
Started at: Tue Aug 18 11:22:50 2009
Finished at: Tue Aug 18 11:27:55 2009
Ran for: 304.900344 seconds
Queries per second: 2425.385916 qps
---------------------------------------------------------------------
cache flushed
queryperf -d input/queries.log.34 -s hidden.server -q 200 -l 300 -b 32
running 9.6.1-P1 with querylogging on and directed to local disk
Statistics:
Parse input file: multiple times
Run time limit: 300 seconds
Ran through file: 0 times
Queries sent: 982436 queries
Queries completed: 973645 queries
Queries lost: 8791 queries
Queries delayed(?): 0 queries
RTT max: 4.999350 sec
RTT min: 0.000219 sec
RTT average: 0.016778 sec
RTT std deviation: 0.152307 sec
RTT out of range: 0 queries
Percentage completed: 99.11%
Percentage lost: 0.89%
Started at: Tue Aug 18 11:29:08 2009
Finished at: Tue Aug 18 11:34:13 2009
Ran for: 304.979150 seconds
Queries per second: 3192.496930 qps
---------------------------------------------------------------------
cache flushed
queryperf -d input/queries.log.34 -s hidden.server -q 200 -l 300 -b 32
running 9.6.1-P1 with querylogging off
Statistics:
Parse input file: multiple times
Run time limit: 300 seconds
Ran through file: 0 times
Queries sent: 1027578 queries
Queries completed: 1018243 queries
Queries lost: 9335 queries
Queries delayed(?): 0 queries
RTT max: 5.043680 sec
RTT min: 0.000008 sec
RTT average: 0.013455 sec
RTT std deviation: 0.142308 sec
RTT out of range: 1 queries
Percentage completed: 99.09%
Percentage lost: 0.91%
Started at: Tue Aug 18 11:35:27 2009
Finished at: Tue Aug 18 11:40:32 2009
Ran for: 304.932400 seconds
Queries per second: 3339.241747 qps
---------------------------------------------------------------------
This server is a caching-forwarder with max-cache-ttl (and
max-ncache-ttl) set to 15 mins. It has 4G of memory with no limit
enforced in named.conf.
version: 9.6.1-P1
CPUs found: 4
worker threads: 4
number of zones: 12
debug level: 0
xfers running: 0
xfers deferred: 0
soa queries in progress: 0
query logging is ON
recursive clients: 0/5900/6000
tcp clients: 0/100
server is up and running
Subhan Malick
---------------------------------------------------------------------
Statistics:
---------------------------------------------------------------------
Statistics:
---------------------------------------------------------------------
Statistics:
---------------------------------------------------------------------
Matus UHLAR - fantomas
> >> Here are some pointers from my experience though:
> >> - syslog query logging is expensive. NEVER enable it. If you need to
> >> log client queries, log it directly to file instead.
> On Wed, Aug 19, 2009 at 12:47 AM, Subhan Malick<mal...@illinois.edu> wrote:
> > I would like to hear more about why this is so. We are currently debating
> > sending query logs to a remote syslog server to enhance some security tools.
On 19.08.09 16:11, Fajar A. Nugraha wrote:
> It depends on your requirement.
> In my case, sending query log to syslog makes disk I/O the bottleneck.
> Not really sure why logging to file directly fix this issue, perhaps
> syslog does a sync() for every line or something.
yes, unless you configure it not to do so. See man page of your syslog
version.
There's also some overhead when sending log line to another process either
via network or local socket and parsing that line in the another process.
Logging to file is just faster and more reliable unless you use remote
logging features of syslog.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"One World. One Web. One Program." - Microsoft promotional advertisement
"Ein Volk, ein Reich, ein Fuhrer!" - Adolf Hitler
Fajar A. Nugraha
> On 8/17/09 10:15 PM, Fajar A. Nugraha wrote:
>>
>> Here are some pointers from my experience though:
>> - syslog query logging is expensive. NEVER enable it. If you need to
>> log client queries, log it directly to file instead.
>
> I would like to hear more about why this is so. We are currently debating
> sending query logs to a remote syslog server to enhance some security tools.
It depends on your requirement.
In my case, sending query log to syslog makes disk I/O the bottleneck.
Not really sure why logging to file directly fix this issue, perhaps
syslog does a sync() for every line or something.
> We are running BIND 9.6.1-P1 with multithreading enabled on RHEL 4 (2
> dual-core 2.8 GHz Opterons with 1MB cache, 4G of RAM). I have run some tests
> and while there is some queries/sec hit, the RTTs are not terrible.
> Queries per second: 2425.385916 qps
I got around 6000 qps on a smiliar test. Jinmei mentioned something
about getting 24k qps on a 4-way Opteron.
Again, it depends on your requirement. If your load is low enough, you
might be able to live with performance penalty imposed by syslog.
--
Fajar
Fajar A. Nugraha
> On 8/17/09 10:15 PM, Fajar A. Nugraha wrote:
>>
>> Here are some pointers from my experience though:
>> - syslog query logging is expensive. NEVER enable it. If you need to
>> log client queries, log it directly to file instead.
>
> I would like to hear more about why this is so. We are currently debating
ls.
It depends on your requirement.
In my case, sending query log to syslog makes disk I/O the bottleneck.
Not really sure why logging to file directly fix this issue, perhaps
syslog does a sync() for every line or something.
> We are running BIND 9.6.1-P1 with multithreading enabled on RHEL 4 (2
> dual-core 2.8 GHz Opterons with 1MB cache, 4G of RAM). I have run some te=
sts
> and while there is some queries/sec hit, the RTTs are not terrible.
> =A0Queries per second: =A0 2425.385916 qps
I got around 6000 qps on a smiliar test. Jinmei mentioned something
about getting 24k qps on a 4-way Opteron.
Again, it depends on your requirement. If your load is low enough, you
might be able to live with performance penalty imposed by syslog.
-- =
Fajar
Matus UHLAR - fantomas
> >> Here are some pointers from my experience though:
> >> - syslog query logging is expensive. NEVER enable it. If you need to
> >> log client queries, log it directly to file instead.
> On Wed, Aug 19, 2009 at 12:47 AM, Subhan Malick<mal...@illinois.edu> wrote:
> > I would like to hear more about why this is so. We are currently debating
> > sending query logs to a remote syslog server to enhance some security tools.
On 19.08.09 16:11, Fajar A. Nugraha wrote:
> It depends on your requirement.
> In my case, sending query log to syslog makes disk I/O the bottleneck.
> Not really sure why logging to file directly fix this issue, perhaps
> syslog does a sync() for every line or something.
yes, unless you configure it not to do so. See man page of your syslog