Promoting slave to master DNS server with dynamic updates
Eric.BERTHIA...@banque-france.fr
Hello DNS gurus,
New on the list, I’ve been tasked by my manager to revamp our dns infrastructure. I think this list is the best place to get answers.
Bind 9.3.6-16 running on RHEL5.7
Right now everything run’s on manually editing zone files but we have recently integrated vmware orchestrator (automatic deployment of linux vm’s) in the mix and that complicates things for automated processes. Add Autosys to this and you have one big messy DNS infrastructure and of course no team wants to change their work flows (classic).
So I’ve written a basic script that would allow everyone (admin’s, vmware, autosys) to use dynamic updates with nsupdate for all tasks. Everything works dandy but a simple question remains:
If the primary goes down for whatever reason, how can we quickly continue to update our DNS records on the secondary? What are the options?
- Classic slave/master change manually editing the named.conf? What happens to everything in the jnl file on the master if it crashes before the dump and zone transfer? Lost?
- Nsupdate special ninja trick? Read something about updating the SOA through dynamic update but didn’t fully understand that process.
- Pray that we get the primary up?
The last option would normally be the logical choice but like I said our DNS infrastructure is a mess and those autosys process control DNS records for their “failover” feature (yeah I know) so we need a super-lightning-fast switch if we do not want production to stop.
Of course the best solution would be something automatic but I haven’t seen anything anywhere. If it’s manual so be it maybe I would be able to write a script that could be used by support and minimize human errors.
I’m already at least pushing for a more recent bind version and of course if a special feature exist (maybe I’ve missed it) that provides an easier solution that would be a super argument to push for something with more features.
Thanks in advance.
Eric
**************************************************************
Ce courrier électronique, y compris les pièces jointes, est à l'attention exclusive des destinataires désignés et revêt un caractère confidentiel.
Si vous recevez ce courrier électronique par erreur, merci d'en informer sans délai l'expéditeur et de supprimer son contenu et ses pièces jointes.
Le contenu de ce courrier électronique ne pourrait engager la responsabilité de la Banque de France que s'il a été émis par une personne dûment habilitée agissant dans le strict cadre des fonctions auxquelles elle est employée et à des fins non étrangères à ses attributions.
L'expéditeur de ce courrier électronique ne peut pas garantir la sécurité de l'acheminement par voie électronique et ne saurait dès lors encourir une quelconque responsabilité en cas d'altération de son contenu.
**************************************************************
This e-mail, attachments included, is intended solely for the addressees and should be considered as confidential.
Should you receive this message by error, please notify the sender immediately and destroy this e-mail and its attachments.
Banque de France cannot be considered as liable for the content of this message unless the sender has been duly authorized and has acted strictly in the course of his/her tasks as an employee.
The sender of this e-mail cannot ensure the security of its electronic transmission and consequently will not be liable should its content be altered.
**************************************************************
John Miller
But why have just a single master? If you're worried about the primary going down, just have a warm standby ready to go. To keep zone data in sync, you could use shared storage, a replicated database (BIND does support a database backend), or perhaps some sort of also-notify configuration. To do the failover itself, you could use something like Pacemaker/Corosync, ucarp, or similar. Just pass a VIP back/forth between the two -- your NS records would only use the VIP.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
bind-users mailing list
bind-...@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
--
John Miller
Systems Engineer
Brandeis University
john...@brandeis.edu
(781) 736-4619
Eric.BERTHIA...@banque-france.fr
Hello John,
Thanks for taking the time to respond.
For the journal files could i force the dump every time i do an update ? nsupdate… rndc freeze/thaw boom everything in sync. I know that normaly you wait 15 minutes and that bind does that for you but we do not have that much load and it would provide us with a safety net regarding a crash? But then again I’m pretty confident a 15 minutes lost of update data is acceptable for production.
For your main solution let me get this…
My master with VIP gets the dynamic updates and since I have an also-notify configured he pushes the updates to my warm-stand-by server running a named in what mode? Master also? But since no one has this server configured as a resolver he doesn’t get requested until I flip the VRRP VIP and restart bind? That could very well work for us BUT what if I want to push an update to this “warm stand-by server”? Since he’s also a master it should work no? And resyncing back to first master……. This is not an easy one if we do not use shared storage.
To be able to use a backend database I would have to upgrade my bind version no? It’s why people use the new map format? I’m pushing for it (want named-journalprint from 9.8 tools) and it would be nice but I’m afraid the support team will have an heart attack from too many changes at the same time (it’s a government organization so it’ssssss waaaayyyy sllllooowwww). That is also why we are running on RHEL5.7 and not 6 even if every new server deployed is on RHEL6.3.
They are planning Infoblox appliances (I’m actually using those on another project) but since it’s the production servers it might take at least a year before they start the project. Dynamic updates prepares this so their world will not fall apart in one shot.
Puppet is used but only on non-system related config… so I would have a tough time for them to accept this but at least I’m advancing towards my goals J
Thanks.
Eric
De : John Miller [mailto:john...@brandeis.edu]
Envoyé : jeudi 11 septembre 2014 17:02
À : BERTHIAUME Eric (UA 2148)
Objet : Re: Promoting slave to master DNS server with dynamic updates
Mark Andrews
In message <CAGYMsbu7TLPMGooqfajWSziO...@mail.gmail.com>, John Miller writes:
>
> Hi Eric,
>
> Depends on how long you can live without dynamic updates, and how many
> dynamic updates it's acceptable to lose in the event of a master failure.
> Journal files are synced every 15 minutes, so in the event of a master
> failure (in a single-master situation), you've lost at most 15 minutes'
> worth of updates. Stand up a new master (with config management software,
> doable in a few minutes), and you're good to go.
The journal exists to recover from the master dying unexpectedly.
It is also used as the source of IXFR data. Additionaly notify
messages are scheduled to be sent out after 3 seconds so slaves
should only be a couple of seconds behind and they also write
journals.
With UPDATE, NOTIFY and IXFR a set of DNS servers should only loose
up to a couple of seconds of transactions in the event of a master
dying in such a way that the disks are unreadable.
> But why have just a single master? If you're worried about the primary
> going down, just have a warm standby ready to go. To keep zone data in
> sync, you could use shared storage, a replicated database (BIND does
> support a database backend), or perhaps some sort of also-notify
> configuration. To do the failover itself, you could use something like
> Pacemaker/Corosync, ucarp, or similar. Just pass a VIP back/forth between
> the two -- your NS records would only use the VIP.
>
> The big issue I see with converting a slave to a master is that you'd have
> to change all of your zone definitions, change your named.conf, and do so
> under time pressure. Then, when the master came back online, you'd have to
> change your zone definitions again. With config management software, not
> that hard to do, but servers are cheap these days -- easier just to create
> a failover pair (or group) for your master, then stand up as many slaves
> (doing update forwarding) as you need. If you purchase a DNS appliance
> (Infoblox, SolidServer, Bluecat, etc.), this is how they handle things.
>
> Separate issue: you might think about trying out RHEL 6 -- it's using a
> much newer version of BIND which has some updated tools (particularly rndc
> freeze/thaw, HTTP stats interface).
>
> John
>
> >
> wrote:
>
> > Hello DNS gurus,
> >
> >
> >
> > New on the list, I=E2=80=99ve been tasked by my manager to revamp our dns
> >
> >
> >
> > Bind 9.3.6-16 running on RHEL5.7
> >
> >
> >
> > recently integrated vmware orchestrator (automatic deployment of linux
> ses. Add
> > Autosys to this and you have one big messy DNS infrastructure and of cour=
> > no team wants to change their work flows (classic).
> >
> >
> >
> =E2=80=99s, vmware,
> > works dandy but a simple question remains:
> >
> >
> >
> > If the primary goes down for whatever reason, how can we quickly continue
> > to update our DNS records on the secondary? What are the options?
> >
> >
> >
> > - Classic slave/master change manually editing the named.conf?
> > What happens to everything in the jnl file on the master if it crashes
> > before the dump and zone transfer? Lost?
> >
> > - Nsupdate special ninja trick? Read something about updating
> rocess.
> > - Pray that we get the primary up?
> >
> >
> >
> > The last option would normally be the logical choice but like I said our
> s
> > for their =E2=80=9Cfailover=E2=80=9D feature (yeah I know) so we need a
> >
> >
> >
> =80=99t
> > seen anything anywhere. If it=E2=80=99s manual so be it maybe I would be=
> > write a script that could be used by support and minimize human errors.
> >
> f course
> > if a special feature exist (maybe I=E2=80=99ve missed it) that provides a=
> > solution that would be a super argument to push for something with more
> > features.
> >
> >
> >
> > Thanks in advance.
> >
> >
> >
> > Eric
> >
> >
> >
> > **************************************************************
> =A0 l'attention
> > exclusive des destinataires d=C3=A9sign=C3=A9s et rev=C3=AAt un caract=C3=
> =A8re confidentiel.
> > Si vous recevez ce courrier =C3=A9lectronique par erreur, merci d'en info=
> rmer
> > sans d=C3=A9lai l'exp=C3=A9diteur et de supprimer son contenu et ses pi=
> =C3=A8ces jointes.
> >
> > Le contenu de ce courrier =C3=A9lectronique ne pourrait engager la
> > responsabilit=C3=A9 de la Banque de France que s'il a =C3=A9t=C3=A9 =C3=
> =A9mis par une personne
> > d=C3=BBment habilit=C3=A9e agissant dans le strict cadre des fonctions au=
> xquelles
> > elle est employ=C3=A9e et =C3=A0 des fins non =C3=A9trang=C3=A8res =C3=A0=
> ses attributions.
> >
> > L'exp=C3=A9diteur de ce courrier =C3=A9lectronique ne peut pas garantir l=
> a s=C3=A9curit=C3=A9
> > de l'acheminement par voie =C3=A9lectronique et ne saurait d=C3=A8s lors =
> encourir une
> > quelconque responsabilit=C3=A9 en cas d'alt=C3=A9ration de son contenu.
> > **************************************************************
> >
> > This e-mail, attachments included, is intended solely for the addressees
> > and should be considered as confidential.
> > Should you receive this message by error, please notify the sender
> > immediately and destroy this e-mail and its attachments.
> >
> > Banque de France cannot be considered as liable for the content of this
> > message unless the sender has been duly authorized and has acted strictly
> > in the course of his/her tasks as an employee.
> >
> > The sender of this e-mail cannot ensure the security of its electronic
> > transmission and consequently will not be liable should its content be
> > altered.
> > **************************************************************
> >
> > _______________________________________________
> > Please visit https://lists.isc.org/mailman/listinfo/bind-users to
> > unsubscribe from this list
> >
> > bind-users mailing list
> > bind-...@lists.isc.org
> > https://lists.isc.org/mailman/listinfo/bind-users
> >
>
>
>
> --089e013a079287ecf30502cb76b2
> Content-Type: text/html; charset=UTF-8
> Content-Transfer-Encoding: quoted-printable
>
> <div dir=3D"ltr"><div>Hi Eric,<br><br></div><div>Depends on how long you ca=
> n live without
> dynamic updates, and how many dynamic updates it's acceptable to lose=
> =20
> in the event of a master failure.=C2=A0 Journal files are synced every 15=
> =20
> minutes, so in the event of a master failure (in a single-master=20
> situation), you've lost at most 15 minutes' worth of updates.=C2=A0=
> Stand up a
> new master (with config management software, doable in a few minutes),=20
> and you're good to go.<br></div><div><br>But why have just a single=20
> master?=C2=A0 If you're worried about the primary going down, just have=
> a=20
> warm standby ready to go.=C2=A0 To keep zone data in sync, you could use=20
> shared storage, a replicated database (BIND does support a database=20
> backend), or perhaps some sort of also-notify configuration.=C2=A0 To do th=
> e=20
> records would only use the VIP.<br><br></div><div>The big issue I see=20
> with converting a slave to a master is that you'd have to change all of=
> =20
> your zone definitions, change your named.conf, and do so under time=20
> pressure.=C2=A0 Then, when the master came back online, you'd have to c=
> hange=20
> your zone definitions again.=C2=A0 With config management software, not tha=
> t=20
> hard to do, but servers are cheap these days -- easier just to create a=20
> failover pair (or group) for your master, then stand up as many slaves=20
> (doing update forwarding) as you need.=C2=A0 If you purchase a DNS applianc=
> e=20
> (Infoblox, SolidServer, Bluecat, etc.), this is how they handle things.<br>=
> <br></div><div>Separate
> issue: you might think about trying out RHEL 6 -- it's using a much=20
> newer version of BIND which has some updated tools (particularly rndc=20
> freeze/thaw, HTTP stats interface).<br><br></div>John</div><div class=3D"gm=
> ail_extra"><br><div class=3D"gmail_quote">On Thu, Sep 11, 2014 at 4:48 AM, =
> <span dir=3D"ltr"><<a href=3D"mailto:Eric.BERTHIAUME.external@banque-fr=
> ance.fr" target=3D"_blank">Eric.BERTHIA...@banque-france.fr</a>>=
> ;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
> .8ex;border-left:1px #ccc solid;padding-left:1ex"><div link=3D"blue" vlink=
> =3D"purple" lang=3D"FR"><div><p class=3D"MsoNormal"><span lang=3D"EN-US">He=
> llo DNS gurus,<u></u><u></u></span></p><p class=3D"MsoNormal"><span lang=3D=
> "EN-US"><u></u>=C2=A0<u></u></span></p><p class=3D"MsoNormal"><span lang=3D=
> "EN-US">New on the list, I=E2=80=99ve been tasked by my manager to revamp o=
> ur dns infrastructure.=C2=A0 I think this list is the best place to get ans=
> wers.<u></u><u></u></span></p><p class=3D"MsoNormal"><span lang=3D"EN-US"><=
> u></u>=C2=A0<u></u></span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">B=
> ind 9.3.6-16 running on RHEL5.7<u></u><u></u></span></p><p class=3D"MsoNorm=
> al"><span lang=3D"EN-US"><u></u>=C2=A0<u></u></span></p><p class=3D"MsoNorm=
> al"><span lang=3D"EN-US">Right now everything run=E2=80=99s on manually edi=
> ting zone files but we have recently integrated vmware orchestrator (automa=
> tic deployment of linux vm=E2=80=99s) in the mix and that complicates thing=
> s for automated processes.=C2=A0 Add Autosys to this and you have one big m=
> essy DNS infrastructure and of course no team wants to change their work fl=
> ows (classic).<u></u><u></u></span></p><p class=3D"MsoNormal"><span lang=3D=
> "EN-US"><u></u>=C2=A0<u></u></span></p><p class=3D"MsoNormal"><span lang=3D=
> "EN-US">So I=E2=80=99ve written a basic script that would allow everyone (a=
> dmin=E2=80=99s, vmware, autosys) to use dynamic updates with nsupdate for a=
> ll tasks. =C2=A0Everything works dandy but a simple question remains:<u></u=
> ><u></u></span></p><p class=3D"MsoNormal"><span lang=3D"EN-US"><u></u>=C2=
> =A0<u></u></span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">If the pri=
> mary goes down for whatever reason, how can we quickly continue to update o=
> ur DNS records on the secondary?=C2=A0 What are the options?<u></u><u></u><=
> /span></p><p class=3D"MsoNormal"><span lang=3D"EN-US"><u></u>=C2=A0<u></u><=
> /span></p><p><u></u><span lang=3D"EN-US"><span>-<span style=3D"font:7.0pt &=
> quot;Times New Roman"">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
> =A0=C2=A0 </span></span></span><u></u><span lang=3D"EN-US">Classic slave/ma=
> ster change manually editing the named.conf?=C2=A0 What happens to everythi=
> ng in the jnl file on the master if it crashes before the dump and zone tra=
> nsfer?=C2=A0 Lost?<u></u><u></u></span></p><p><u></u><span lang=3D"EN-US"><=
> span>-<span style=3D"font:7.0pt "Times New Roman"">=C2=A0=C2=A0=
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 </span></span></span><u></u><spa=
> n lang=3D"EN-US">Nsupdate special ninja trick?=C2=A0 Read something about u=
> pdating the SOA through dynamic update but didn=E2=80=99t fully understand =
> that process.<u></u><u></u></span></p><p><u></u><span lang=3D"EN-US"><span>=
> -<span style=3D"font:7.0pt "Times New Roman"">=C2=A0=C2=A0=C2=A0=
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 </span></span></span><u></u><span lang=
> =3D"EN-US">Pray that we get the primary up?<u></u><u></u></span></p><p clas=
> s=3D"MsoNormal"><span lang=3D"EN-US"><u></u>=C2=A0<u></u></span></p><p clas=
> s=3D"MsoNormal"><span lang=3D"EN-US">The last option would normally be the =
> logical choice but like I said our DNS infrastructure is a mess and those a=
> utosys process control DNS records for their =E2=80=9Cfailover=E2=80=9D fea=
> ture (yeah I know) so we need a super-lightning-fast switch if we do not wa=
> nt production to stop.<u></u><u></u></span></p><p class=3D"MsoNormal"><span=
> lang=3D"EN-US"><u></u>=C2=A0<u></u></span></p><p class=3D"MsoNormal"><span=
> lang=3D"EN-US">Of course the best solution would be something automatic bu=
> t I haven=E2=80=99t seen anything anywhere.=C2=A0 If it=E2=80=99s manual so=
> be it maybe I would be able to write a script that could be used by suppor=
> t and minimize human errors.<u></u><u></u></span></p><p class=3D"MsoNormal"=
> ><span lang=3D"EN-US"> <u></u><u></u></span></p><p class=3D"MsoNormal"><spa=
> n lang=3D"EN-US">I=E2=80=99m already at least pushing for a more recent bin=
> d version and of course if a special feature exist (maybe I=E2=80=99ve miss=
> ed it) that provides an easier solution that would be a super argument to p=
> ush for something with more features.<u></u><u></u></span></p><p class=3D"M=
> soNormal"><span lang=3D"EN-US"><u></u>=C2=A0<u></u></span></p><p class=3D"M=
> soNormal"><span lang=3D"EN-US">Thanks in advance.<u></u><u></u></span></p><=
> p class=3D"MsoNormal"><span lang=3D"EN-US"><u></u>=C2=A0<u></u></span></p><=
> p class=3D"MsoNormal"><span lang=3D"EN-US">Eric<u></u><u></u></span></p><p =
> class=3D"MsoNormal"><span lang=3D"EN-US"><u></u>=C2=A0<u></u></span></p></d=
> iv><p>**************************************************************<br>Ce =
> courrier =C3=A9lectronique, y compris les pi=C3=A8ces jointes, est =C3=A0 l=
> 'attention exclusive des destinataires d=C3=A9sign=C3=A9s et rev=C3=AAt=
> un caract=C3=A8re confidentiel.<br>Si vous recevez ce courrier =C3=A9lectr=
> onique par erreur, merci d'en informer sans d=C3=A9lai l'exp=C3=A9d=
> iteur et de supprimer son contenu et ses pi=C3=A8ces jointes.</p>
> <p>Le contenu de ce courrier =C3=A9lectronique ne pourrait engager la respo=
> nsabilit=C3=A9 de la Banque de France que s'il a =C3=A9t=C3=A9 =C3=A9mi=
> s par une personne d=C3=BBment habilit=C3=A9e agissant dans le strict cadre=
> des fonctions auxquelles elle est employ=C3=A9e et =C3=A0 des fins non =C3=
> =A9trang=C3=A8res =C3=A0 ses attributions.</p>
> <p>L'exp=C3=A9diteur de ce courrier =C3=A9lectronique ne peut pas garan=
> tir la s=C3=A9curit=C3=A9 de l'acheminement par voie =C3=A9lectronique =
> et ne saurait d=C3=A8s lors encourir une quelconque responsabilit=C3=A9 en =
> cas d'alt=C3=A9ration de son contenu.</p>
> <p>**************************************************************</p>
> <p>This e-mail, attachments included, is intended solely for the addressees=
> and should be considered as confidential.<br>Should you receive this messa=
> ge by error, please notify the sender immediately and destroy this e-mail a=
> nd its attachments.</p>
> <p>Banque de France cannot be considered as liable for the content of this =
> message unless the sender has been duly authorized and has acted strictly i=
> n the course of his/her tasks as an employee.</p>
> <p>The sender of this e-mail cannot ensure the security of its electronic t=
> ransmission and consequently will not be liable should its content be alter=
> ed.<br>**************************************************************</p></=
> div><br>_______________________________________________<br>
> Please visit <a href=3D"https://lists.isc.org/mailman/listinfo/bind-users" =
> target=3D"_blank">https://lists.isc.org/mailman/listinfo/bind-users</a> to =
> unsubscribe from this list<br>
> <br>
> bind-users mailing list<br>
> <a href=3D"mailto:bind-...@lists.isc.org">bind-...@lists.isc.org</a><br=
> >
> <a href=3D"https://lists.isc.org/mailman/listinfo/bind-users" target=3D"_bl=
> ank">https://lists.isc.org/mailman/listinfo/bind-users</a><br></blockquote>=
> </div><br><br clear=3D"all"><br>-- <br>John Miller<br>Systems Engineer<br>B=
> randeis University<br><a href=3D"mailto:john...@brandeis.edu">johnmill@bra=
> ndeis.edu</a><br>(781) 736-4619
> </div>
>
> --089e013a079287ecf30502cb76b2--
>
> --===============3368274548509764707==
> Content-Type: text/plain; charset="us-ascii"
> MIME-Version: 1.0
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
> _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
>
> bind-users mailing list
> bind-...@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
Stuart Browne
> -----Original Message-----
> From: bind-user...@lists.isc.org [mailto:bind-users-
> bou...@lists.isc.org] On Behalf Of Mark Andrews
> Sent: Friday, 12 September 2014 8:58 AM
> To: John Miller
> Cc: Bind Users Mailing List
> Subject: Re: Promoting slave to master DNS server with dynamic updates
>
> Journals are consolidated into the master file every 15 minutes.
> The journal exists to recover from the master dying unexpectedly.
> It is also used as the source of IXFR data. Additionaly notify
> messages are scheduled to be sent out after 3 seconds so slaves
> should only be a couple of seconds behind and they also write
> journals.
>
> With UPDATE, NOTIFY and IXFR a set of DNS servers should only loose
> up to a couple of seconds of transactions in the event of a master
> dying in such a way that the disks are unreadable.
Stuart