Re: Recommended value for max-cache-size for cache-only shared hosts..

[Doug Barton]

On 5/31/2012 1:51 PM, blrmaani wrote:

> Question:
> what is the recommended configuration for 'max-cache-size' for optimum
> usage ?

You should not restrict the size of the cache at all if you want the
best performance. BIND will use as much memory as it needs in order to
satisfy the requests of your users.


--
If you're never wrong, you're not trying hard enough

[Michael Graff]

Hmm, I don't quite think this is a good idea. BIND 9 (since 9.5) manages memory quite well, but it will happily consume all you have and go into swap.

I'd set it high enough (on a dedicated machine) to use plenty of RAM, but low enough to not cause other OS components to swap out or BIND itself to swap. 75% or 85% range seems like a good starting point.

--Michael

> _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
>
> bind-users mailing list
> bind-...@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users

[blr maani]

Doug,

  hmmm.. 75%-85% seems too large because the host runs email application in addition to cache-and-forward-only BIND (for better local caching). So, I was wondering if there are any best/proven practice/recommendations for such shared application hosts ? 

The default value is 32MB. We have 8GB RAM. I don't know if its better to start with 1GB (1/8th of RAM)?

thanks

blr

[Michael Graff]

It's really something you'll have to set, and monitor.  I'd start with 1 GB, and see how close it gets to that in (say) a week.  If it takes a few hours, you might need to go up to 2 or 4, and see how that works.  It may never hit the memory limit.  Also note that there is 10% to 20% overhead, so if you set a 1 GB limit, it's really more like a 1.1GB to 1.2GB limit.  This is because the cache is not the only thing that uses memory, of course, and the limit is only for the cache.

Remember that the cache is only used as a cache, and is not required for operation.  Technically, BIND 9 could run with a very, very small cache.  The default of 32 MB is actually a fairly new thing.  It used to be unlimited, but that means BIND will hit some operating system imposed limit, and that is more painful than self-management.

--Michael

[Doug Barton]

On 05/31/2012 22:26, blr maani wrote:
> Doug,
> hmmm.. 75%-85% seems too large because the host runs email application
> in addition to cache-and-forward-only BIND (for better local caching).

So get more RAM, or split your services onto multiple systems. Yes, I
realize that may not be possible for financial reasons, but you asked
about *optimum* performance. The cache is there for a reason.

One thing that can help is to set the cleaning interval more
aggressively, but that can also cause performance problems for your
clients if you are CPU bound, so use that option with care, and monitor
the results after a change.

> So, I was wondering if there are any best/proven
> practice/recommendations for such shared application hosts ?

Yes, don't do that. :)

Doug

[Chris Thompson]

On Jun 1 2012, Michael Graff wrote:

> [...] The default of 32 MB is actually a fairly new thing.

Surely the default went back to 0 (effectively unlimited) long ago?

2253. [func] "max-cache-size" defaults to 32M.
"max-acache-size" defaults to 16M.

got into BIND 9.5.0, but

2457. [tuning] max-cache-size is reverted to 0, the previous
default. It should be safe because expired cache
entries are also purged. [RT #18684]

was there before 9.5.1, and AFAICS it has been like that ever since.

--
Chris Thompson
Email: ce...@cam.ac.uk

[Matus UHLAR - fantomas]

On 31.05.12 22:26, blr maani wrote:
> hmmm.. 75%-85% seems too large because the host runs email application in

>addition to cache-and-forward-only BIND (for better local caching). So, I

>was wondering if there are any best/proven practice/recommendations for
>such shared application hosts ?
>

>The default value is 32MB. We have 8GB RAM. I don't know if its better to
>start with 1GB (1/8th of RAM)?

I was thinking of this when the default was changed to 32M. I changed
it intentionally to 0 to see how much will memory usage grow.

I can tell you that on one of our servers where named uses most memory,
it currently uses 1359868 VSZ and 732852 RSS after 38 days with ~432
queries per second.

I have even increased max-ttl and max-negative-ttl to see if it affects
memory usage.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Chernobyl was an Windows 95 beta test site.

[JINMEI Tatuya / 神明達哉]

At Fri, 01 Jun 2012 03:27:22 -0700,

Doug Barton <do...@dougbarton.us> wrote:

> One thing that can help is to set the cleaning interval more
> aggressively, but that can also cause performance problems for your
> clients if you are CPU bound, so use that option with care, and monitor
> the results after a change.

cleaning interval has been effectively no-op since BIND 9.5. Tweaking
it won't improve performance, although it shouldn't cause a bad effect
either.

---
JINMEI, Tatuya
Internet Systems Consortium, Inc.

[Dan Mason]

On Fri, Jun 01, 2012 at 01:11:48PM -0700, JINMEI Tatuya / ?$B?@L@C#:H wrote:
> At Fri, 01 Jun 2012 03:27:22 -0700,

> cleaning interval has been effectively no-op since BIND 9.5. Tweaking
> it won't improve performance, although it shouldn't cause a bad effect
> either.

If your cache is too small the CPU will peg when the cleaning-interval goes. Maybe that's changed but the behavior still exists in the 9.7 branch. Setting your cache size really depends on your query load. On a resolver doing 15,000/qps having a cache of 256M will cause a problem during the cleaning-interval whereas if it's 2G you won't notice the interval at all. Also on a busy resolver expect BIND to use about twice as much as where you set your limits.


Dan

--
Daniel Mason
Senior Engineer
CenturyLink, Inc.

Internal Use Only - Disclose and distribute only to CenturyLink employees and authorized persons working for CenturyLink. Disclosure outside of CenturyLink is prohibited without authorization.

[JINMEI Tatuya / 神明達哉]

At Fri, 1 Jun 2012 21:14:06 +0000,

Dan Mason <danm...@qwest.net> wrote:

> > cleaning interval has been effectively no-op since BIND 9.5. Tweaking
> > it won't improve performance, although it shouldn't cause a bad effect
> > either.
>
> If your cache is too small the CPU will peg when the cleaning-interval goes. Maybe that's changed but the behavior still exists in the 9.7 branch. Setting your cache size really depends on your query load. On a resolver doing 15,000/qps having a cache of 256M will cause a problem during the cleaning-interval whereas if it's 2G you won't notice the interval at all. Also on a busy resolver expect BIND to use about twice as much as where you set your limits.

Hmm, looking into the code again, I realized my memory was slightly
incorrect: "cleaning interval has been effectively no-op since BIND
9.5" should have been "cleaning interval has been effectively
meaningless and therefore disabled by default since BIND 9.5", and if
you explicitly enable it by setting cleaning-interval to a non 0
value, it will still do meaningless but expensive operations.

So, in conclusion, my main point should still stand: "Tweaking it
(cleaning-interval) won't improve performance". And, it could
actually do harm.

[Doug Barton]

Thanks, I learned something today! But that sort of prompts the question
in my mind, why does the option still exist?

[JINMEI Tatuya / 神明達哉]

At Mon, 04 Jun 2012 12:53:31 -0700,

Doug Barton <do...@dougbarton.us> wrote:

> >> If your cache is too small the CPU will peg when the cleaning-interval goes. Maybe that's changed but the behavior still exists in the 9.7 branch. Setting your cache size really depends on your query load. On a resolver doing 15,000/qps having a cache of 256M will cause a problem during the cleaning-interval whereas if it's 2G you won't notice the interval at all. Also on a busy resolver expect BIND to use about twice as much as where you set your limits.
> >
> > Hmm, looking into the code again, I realized my memory was slightly
> > incorrect: "cleaning interval has been effectively no-op since BIND
> > 9.5" should have been "cleaning interval has been effectively
> > meaningless and therefore disabled by default since BIND 9.5", and if
> > you explicitly enable it by setting cleaning-interval to a non 0
> > value, it will still do meaningless but expensive operations.
> >
> > So, in conclusion, my main point should still stand: "Tweaking it
> > (cleaning-interval) won't improve performance". And, it could
> > actually do harm.
>
> Thanks, I learned something today! But that sort of prompts the question
> in my mind, why does the option still exist?

Good question, I wonder the same thing:-) I don't remember the
original plan, but I guess it was actually planned to be deprecated
but it has just been forgotten or left as a lower priority thing since
then.

[Doug Barton]

On 6/5/2012 11:30 AM, JINMEI Tatuya / 神明達哉 wrote:
> Good question, I wonder the same thing:-) I don't remember the
> original plan, but I guess it was actually planned to be deprecated
> but it has just been forgotten or left as a lower priority thing since
> then.

So, get busy! It's not like you have nothing else to do ... :)

[Mike Hoskins]

-----Original Message-----
From: Doug Barton <do...@dougbarton.us>
Organization: http://SupersetSolutions.com/
Date: Tuesday, June 5, 2012 11:49 AM
To: JINMEI Tatuya / 神明達哉 <jin...@isc.org>
Cc: <comp-protoc...@isc.org>
Subject: Re: Recommended value for max-cache-size for cache-only shared
hosts..

>On 6/5/2012 11:30 AM, JINMEI Tatuya / 神明達哉 wrote:
>> Good question, I wonder the same thing:-) I don't remember the
>> original plan, but I guess it was actually planned to be deprecated
>> but it has just been forgotten or left as a lower priority thing since
>> then.
>
>So, get busy! It's not like you have nothing else to do ... :)

sorry to waste bandwidth, but just wanted to point out this statement is
more true than expected in jest...with the double negative (nothing vs
anything).

i hate english... ;-)