minimum cache times?
Christoph Weber-Fahr
recently, I ran into a debate on the merits of negative TTL caching.
Digging a little into the issue I found that apparently
- no version of Bind currently supports min-(n)cache-ttl parameters
- MS DNS apparently has such a function
- somebody (possibly Michael Milligan) at some time put it into
Debian's BIND.
Can anybody give any more information on that?
Regards,
Christoph Weber-Fahr
Atkins, Brian (GD/VA-NSOC)
response with no real information).
>From what I've read, everyone seems to frown on over-riding cache times,
but I haven't seen any specifics as to why it's bad.
Brian
Dave Sparro
Basically, it is impolite.
If you ignore my Authoritative server's request to cache answers, you'll
end up either increasing the load on my server, or missing an update I
make to my data (depending on which direction you adjust the cache time).
Now imagine a world where everybody ignores my TTL.
--
Dave
Rob Austein
>
> I asked a similar question 2 weeks ago and got a non-response (e.g., a
> response with no real information).
>
> From what I've read, everyone seems to frown on over-riding cache times,
> but I haven't seen any specifics as to why it's bad.
Because it's a protocol violation, deliberately ignores the cache time
set by the owner of the data, and is dangerous.
Eg, you ask me for the address of my web server. I answer, saying
that the answer is good for a week, after which you need to ask again
because I might have changed something. You override the TTL time and
cache the data for two weeks. Meanwhile, I start the process of
moving my server to a different address. Protocol says I have to wait
the time I set in the TTL, then I can assume that all cached copies of
the old data are dead, at which point it's safe for me to kill the old
address. But you're ignoring the TTL. So I go ahead and move my
server, your users still see your past-expiration copy of the old
address, can't reach my server, and my help desk phone starts ringing.
Your fault, but I pay for it, because you violated the protocol.
The above is a simple example. For some real fun, throw DNSSEC into
the mix and think about signature expiration times.
"min-ttl" is a really bad idea. I first saw it proposed in the late
'80s. It was a bad idea then, and it's still a bad idea now. Every
few years somebody exhumes it, it lurches, undead, into some patch
set, and we replay this discussion again. Most likely the reason you
didn't get an immediate response is simply that playing whack-a-zombie
gets old after the first decade or so.
The TTL mechanism is part of the protocol for a reason: it's to
control how tightly consistent the data are supposed to be in the
opinion of the publisher of the data. Nobody but the publisher of the
data has enough information to know how long it's safe to keep the
data. Some publishers make silly decisions about this setting, which
causes other problems, but keeping data past its expiration time is
not the answer.
Nicholas Wheeler
_______________________________________________
bind-users mailing list
bind-...@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
Rob Austein
>
> I think Brian's OP was about a max-ttl override ... Which is the
> opposite. The only disadvantages I see is a potential waste of
> bandwidth (and it violates the protocol).
max-ttl is (very) different from min-ttl. max-ttl might (or might
not) be a waste of bandwidth, but it can't be a violation of the
protocol, because nobody can require you to cache at all, or to
preserve your cache across reboots, etc.
max-ttl has been around since at least, um, 1985, when I implemented
it in a non-BIND iterative resolver to cope with the 99999999 TTLs
that we were receiving from certain badly configured authoritative
nameservers. It's not something to use blindly, but it's definitely
legal, and is sometimes necessary. The trick with max-ttl is to set
it to a sane value for your situation. Eg, an iterative resolver
associated with a busy MTA might use a max-ttl setting equal to half
of the MTA's queue lifetime, to insure that it tried looking for an
updated MX RR at least once before giving up on a message.
Eivind Olsen
> response with no real information).
The only somewhat good reason I see to overriding (well, lowering) the
cache time is if it causes your server any memory issues. Although the
real solution then would be to buy more memory. Yes, an active DNS server
will cache a few GB, depending on usage patterns, how common DNSSEC
becomes etc, but if you run an active DNS-server I'd hope you'd be able to
get the budget for that memory.
Overriding the cache TTL by lowering it is essentially the same as what
happens when nameservers are restarted - it isn't optimal, but it happens
all the time all over the world.
Overriding the cache TTL by _increasing_ the value is something that's
bound to break many setups - if I set my TTL to a low value, it's
hopefully for a reason.
I have had to remove some cached information before it timed out by itself
due to TTL - depending on how often you need to do that and how many
servers you have, one option might be to do something like "rndc flushname
hostname.to.flush" on those servers.
Depending on your setup, you might also consider centralizing this so you
can do it once from one location (easiest solution: make a wrapper script,
running rndc on all servers in turn, over the network).
Regards
Eivind Olsen
Atkins, Brian (GD/VA-NSOC)
While I am unsure if Chrisoph's question was answered, I now understand
why most everyone thinks it is a bad idea to over-ride the TTL for
records I am not authoritive for:
1) It's not RFC compliant for the protocol
2) Changing it could potentially increase load on the DNS servers for
other domains
3) It's bad manners.
So, that being said, can anyone suggest an alternative to my issue?
Currently, we use DNS to blackhole bad domains. The list of bad domains
are provided to us from another government entity or vetted by an
enterprise security team.
The servers I manage are the DNS servers of last resort for our internal
clients before hitting up root. However, they are not the only DNS
servers available to the clients - there are several hundred internal
servers, mostly windows servers, that handle client queries. I have no
control over them.
So, when I add new domains to my block list, I am at the mercy of the
bad domain's TTL. I have had DNS cache thwarting my ability to block the
bad domain, sometimes for several days.
Basically, I want to make the block occur within a couple of hours after
implementation - hence setting the max-cache-ttl.
I realize that there are other ways of to do this, but I am limited by
my funding.
Thanks,
Brian
Eivind Olsen
> Currently, we use DNS to blackhole bad domains. The list of bad domains
> are provided to us from another government entity or vetted by an
> enterprise security team.
How do you implement this list? By putting those domains into your
named.conf (or some included configuration file) as authoritative domains,
pointing to a common dummy zonefile, and then reloading/restarting BIND?
If you do it like this and restart BIND, you'll automatically lose the old
cached information anyway.
If you instead add to named.conf and do "rndc reconfig", I don't think it
will drop previously cached information.
Depending on how you do this - is it feasible to do "rndc flushname
old.cached.domain" on these domains?
> The servers I manage are the DNS servers of last resort for our internal
> clients before hitting up root. However, they are not the only DNS
> servers available to the clients - there are several hundred internal
> servers, mostly windows servers, that handle client queries. I have no
> control over them.
Are all those DNS servers pointing to your server as their forwarder, or
will any change you do on your server still have next to no impact since
these other servers bypass you anyway?
In other words, is your setup something like this:
[clients] --> [X amount of DNS servers you don't control] --> [YOUR DNS
server] --> Internet
?
> So, when I add new domains to my block list, I am at the mercy of the
> bad domain's TTL. I have had DNS cache thwarting my ability to block the
> bad domain, sometimes for several days.
If the information is cached at your internal servers which _you_ have no
control over, you'll still be at the mercy of any long TTL.
> Basically, I want to make the block occur within a couple of hours after
> implementation - hence setting the max-cache-ttl.
> I realize that there are other ways of to do this, but I am limited by
> my funding.
As long as you don't have control over all the different DNS servers used
in your organization, you'll still have problems making a solution here.
Regards
Eivind Olsen
Atkins, Brian (GD/VA-NSOC)
conclusion we came to.
Thank you for confirming it.
Brian
Christoph Weber-Fahr
On 05.10.2010 16:45, Nicholas Wheeler wrote:
> > At Tue, 5 Oct 2010 09:19:49 -0400, Atkins, Brian (GD/VA-NSOC) wrote:
>> >> From what I've read, everyone seems to frown on over-riding cache times,
>> >> but I haven't seen any specifics as to why it's bad.
> >
> > Because it's a protocol violation, deliberately ignores the cache time
> > set by the owner of the data, and is dangerous.
> >
> > Eg, you ask me for the address of my web server. I answer, saying
> > that the answer is good for a week, after which you need to ask again
> > because I might have changed something.
Well, I was talking about minimum values, and, especially, a min-ncache-ttl,
i.e. a minimum for negative caching.
My point of view is that of a the operator of a very busy DNS resolver/cache
infrastructure.
For anecdotal evidence, I present this:
http://blog.boxedice.com/2010/09/28/watch-out-for-millions-of-ipv6-dns-aaaa-requests/
Now this ostensibly is about how bad IPv6 is for DNS (n comment),
but somewhere down comes the interesting tidbit: apparently there
are commercial DNS providers (dyn.com in this case) who recommend
and default to 60 seconds as SOA value for negative caching in their
customer zones.
RIPE's recommended default is 1 hour.
Of course they do this for a reason - they actually charge by
request, so a badly set up customer DNS improves their bottom line.
This is ridiculous and puts quite a strain on resolvers having to deal
with such data - especially if one of 2 requests is no-error/no-data
for AAAA reasons.
So, if this is a trend, we might want to have a min-ncache-ttl of 300,
just to get rid of the most obnoxious jerks.
Same goes for positive caching; sensible minimum values used to be
a matter of politeness, but folks like Akamai give us TTLs like
20 or 60. As long as Akamai is the only one doing this that's not
a problem - but should that get widespread use I'd be inclined
to clamp down on this, too.
> > The TTL mechanism is part of the protocol for a reason: it's to
> > control how tightly consistent the data are supposed to be in the
> > opinion of the publisher of the data. Nobody but the publisher of the
> > data has enough information to know how long it's safe to keep the
> > data. Some publishers make silly decisions about this setting, which
> > causes other problems, but keeping data past its expiration time is
> > not the answer.
Caching is part of the protocol, too. If there are large scale
developments sabotaging that it forces me to have much more
resolver capacity online.
And that costs *me* money. Yes, publisher should know best - but
apparently he often doesn't, and publishing bad DNS data
affect's other people's systems, too.
Regards
Christoph Weber-Fahr
Doug Barton
list address in your e-mail address book and then create a new message
to the list. By replying to someone else's message and changing the
subject you cause your message to appear "hidden" behind the message you
replied to for those of us who use threaded mail readers.
FYI,
Doug
--
Breadth of IT experience, and | Nothin' ever doesn't change,
depth of knowledge in the DNS. | but nothin' changes much.
Yours for the right price. :) | -- OK Go
http://SupersetSolutions.com/
Christoph Weber-Fahr
On 06.10.2010 01:16, Doug Barton wrote:
> If you would like to create a new thread your best bet is to
> store the list address in your e-mail address book and then
> create a new message to the list. By replying to someone
> else's message and changing the subject you cause your
> message to appear "hidden" behind the message you replied
> to for those of us who use threaded mail readers.
Hehe. Thanks, I normally don't use threading, so I never noticed.
Ok, so here's my recent reply again. Hope this is a new thread now,
and apologies to those who see it the second time.
Mark Andrews
In message <4CAD0856...@arcor.de>, Christoph Weber-Fahr writes:
> On 05.10.2010 16:45, Nicholas Wheeler wrote:
> > At Tue, 5 Oct 2010 09:19:49 -0400, Atkins, Brian (GD/VA-NSOC) wrote:
> > > From what I've read, everyone seems to frown on over-riding cache
> > > times, but I haven't seen any specifics as to why it's bad.
> >
> > Because it's a protocol violation, deliberately ignores the
> > cache time set by the owner of the data, and is dangerous.
> >
> > Eg, you ask me for the address of my web server. I answer, saying
> > that the answer is good for a week, after which you need to ask again
> > because I might have changed something.
>
> Well, I was talking about minimum values, and, especially,
> a min-ncache-ttl, i.e. a minimum for negative caching.
>
> My point of view is that of a the operator of a very busy DNS resolver/cache
> infrastructure.
>
> For anecdotal evidence, I present this:
>
> http://blog.boxedice.com/2010/09/28/watch-out-for-millions-of-ipv6-dns-aaaa-r
> equests/
>
> Now this ostensibly is about how bad IPv6 is for DNS (n comment),
> but somewhere down comes the interesting tidbit: apparently there
> are commercial DNS providers (dyn.com in this case) who recommend
> and default to 60 seconds as SOA value for negative caching in their
> customer zones.
For a dynamic DNS provider where A RRsets come and go 60 seconds
is about right. It's also pretty good evidence that it is time to
set up IPv6 for that name. There are obviously plenty of clients
out there willing to connect over IPv6 if only the server supported
it.
> RIPE's recommended default is 1 hour.
Aimed at a different user base.
> Of course they do this for a reason - they actually charge by
> request, so a badly set up customer DNS improves their bottom line.
>
> This is ridiculous and puts quite a strain on resolvers having to deal
> with such data - especially if one of 2 requests is no-error/no-data
> for AAAA reasons.
>
> So, if this is a trend, we might want to have a min-ncache-ttl of 300,
> just to get rid of the most obnoxious jerks.
Or one might actually turn on IPv6. Plenty of unsatisfied demand out
there.
> Same goes for positive caching; sensible minimum values used to be
> a matter of politeness, but folks like Akamai give us TTLs like
> 20 or 60. As long as Akamai is the only one doing this that's not
> a problem - but should that get widespread use I'd be inclined
> to clamp down on this, too.
>
> > The TTL mechanism is part of the protocol for a reason: it's to
> > control how tightly consistent the data are supposed to be in the
> > opinion of the publisher of the data. Nobody but the publisher
> > of the data has enough information to know how long it's safe to
> > keep the data. Some publishers make silly decisions about this
> > setting, which causes other problems, but keeping data past
> > its expiration time is not the answer.
>
> Caching is part of the protocol, too. If there are large scale
> developments sabotaging that it forces me to have much more
> resolver capacity online.
Well a little more bandwidth. Percentage wise DNS is small compared
to all the other traffic out there.
> And that costs *me* money. Yes, publisher should know best - but
> apparently he often doesn't, and publishing bad DNS data
> affect's other people's systems, too.
>
> Regards
>
> Christoph Weber-Fahr
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
Christoph Weber-Fahr
On 07.10.2010 02:40, Mark Andrews wrote:
> In message <4CAD0856...@arcor.de>, Christoph Weber-Fahr writes:
>> Well, I was talking about minimum values, and, especially,
>> a min-ncache-ttl, i.e. a minimum for negative caching.
>>
>> My point of view is that of a the operator of a very busy DNS resolver/cache
>> infrastructure.
>>
>> For anecdotal evidence, I present this:
>>
>> http://blog.boxedice.com/2010/09/28/watch-out-for-millions-of-ipv6-dns-aaaa-r
>> equests/
>>
>> Now this ostensibly is about how bad IPv6 is for DNS (no comment),
>> but somewhere down comes the interesting tidbit: apparently there
>> are commercial DNS providers (dyn.com in this case) who recommend
>> and default to 60 seconds as SOA value for negative caching in their
>> customer zones.
>
> For a dynamic DNS provider where A RRsets come and go 60 seconds
> is about right.
This isn' about dynamic DNS. To quote:
" Dyn Inc. is a world leader in managed DNS, providing
“rock-solid” DNS solutions for everyone. "
The quoted case is about a standard DNS customer having a
normal, hosted web server who uses dyn.com for DNS hosting.
>> RIPE's recommended default is 1 hour.
>
> Aimed at a different user base.
Actually, no. This case is exactly what RIPE recommends the 1h for.
> It's also pretty good evidence that it is time to
> set up IPv6 for that name. There are obviously plenty of clients
> out there willing to connect over IPv6 if only the server supported
> it.
But it's not my name, and I have no control over it; nor do I have
control over millions of other names customers of ours are resolving,
using our infrastructure.
Short negative caching times are convenient for Domain owners
but troublesome for cache owners; and my main question is
does or will Bind provide the means to mitigate at least
the more egregious cases.
A min-ncache-ttl might be a way to do that.
> Or one might actually turn on IPv6. Plenty of unsatisfied demand out
> there.
Correct but irrelevant.
> Well a little more bandwidth. Percentage wise DNS is small compared
> to all the other traffic out there.
Bandwidth is not the problem. DNS work is. Recursive resolving is much more
costly in terms of resolver capacity than answering from cache.
Regards,
Christoph Weber-Fahr
Mark Andrews
In message <4CADEF52...@arcor.de>, Christoph Weber-Fahr writes:
> Hello,
>
> On 07.10.2010 02:40, Mark Andrews wrote:
> > In message <4CAD0856...@arcor.de>, Christoph Weber-Fahr writes:
> >> Well, I was talking about minimum values, and, especially,
> >> a min-ncache-ttl, i.e. a minimum for negative caching.
> >>
> ache
> >> infrastructure.
> >>
> >> For anecdotal evidence, I present this:
> >>
> aaa-r
> >> equests/
> >>
> >> Now this ostensibly is about how bad IPv6 is for DNS (no comment),
> >> but somewhere down comes the interesting tidbit: apparently there
> >> are commercial DNS providers (dyn.com in this case) who recommend
> >> and default to 60 seconds as SOA value for negative caching in their
> >> customer zones.
>
> > For a dynamic DNS provider where A RRsets come and go 60 seconds
>
>
> This isn' about dynamic DNS. To quote:
>
> " Dyn Inc. is a world leader in managed DNS, providing
>
> The quoted case is about a standard DNS customer having a
> normal, hosted web server who uses dyn.com for DNS hosting.
And TTLs, both positive and negative need to be tuned for the usage
model. Some usage models require small TTLs some don't.
>From what I can see they provided feedback that there was unexpected
traffic within a short period of time which is what I would expect
from a managed service.
> >> RIPE's recommended default is 1 hour.
> >
> > Aimed at a different user base.
>
> Actually, no. This case is exactly what RIPE recommends the 1h for.
Sorry 1 hour is ridiculously long if I have a machine that is coming
and going from the net and needs to remove the A records when it is
off the net.
If applications honoured 0.0.0.0 as I exist but don't know my address
one could leave a 0.0.0.0 record in the DNS and not need a small
negative TTL.
If you just need quick change over to a different set of addresses
then a longer negative TTL is appropriate.
> > It's also pretty good evidence that it is time to
> > set up IPv6 for that name. There are obviously plenty of clients
> > out there willing to connect over IPv6 if only the server supported
> > it.
>
> But it's not my name, and I have no control over it; nor do I have
> control over millions of other names customers of ours are resolving,
> using our infrastructure.
>
> Short negative caching times are convenient for Domain owners
> but troublesome for cache owners; and my main question is
> does or will Bind provide the means to mitigate at least
> the more egregious cases.
Which assumes you have a better understanding of why the TTL is
short in the first place.
> A min-ncache-ttl might be a way to do that.
>
> > Or one might actually turn on IPv6. Plenty of unsatisfied demand out
> > there.
>
> Correct but irrelevant.
>
> > Well a little more bandwidth. Percentage wise DNS is small compared
> > to all the other traffic out there.
>
> Bandwidth is not the problem. DNS work is. Recursive resolving is much more
> costly in terms of resolver capacity than answering from cache.
>
> Regards,
>
> Christoph Weber-Fahr
> _______________________________________________
> bind-users mailing list
> bind-...@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users