Problem: the AD zone _msdcs.<domain> was not found

https://blogs.msmvps.com/acefekay/2010/10/04/complete-step-by-step-to-remove-an-orphaned-domain-controller/

Web: www.mpecsinc.com

Blog: blog.mpecsinc.com

Twitter: Twitter.com/MPECSInc

Skype: MPECSInc.

Please note: Although we may sometimes respond to email, text and phone calls instantly at all hours of the day, our regular business hours are 8:00 AM - 5:00 PM, Monday thru Friday.

From: ntsys...@googlegroups.com <ntsys...@googlegroups.com> On Behalf Of Mike Leone
Sent: September 17, 2021 10:07
To: NTSysAdmin <ntsys...@googlegroups.com>
Subject: [ntsysadmin] Problem: the AD zone _msdcs.<domain> was not found

Somebody explain this one to me ... I ran the Best Practices Analyzer on my root domain DC. And one of the things it found says:

Problem:
The Active Directory integrated DNS zone _msdcs.ads.pha.phila.gov was not found.

And recommends restoring it.

Yet the zone is right there ...

Any ideas what's going on here?? Why does it think this zone doesn't exist?

--

Mike. Leone, <mailto:tur...@mike-leone.com>

PGP Fingerprint: 0AA8 DC47 CB63 AE3F C739 6BF9 9AB4 1EF6 5AA5 BCDF
Photo Gallery: <http://www.flickr.com/photos/mikeleonephotos>

This space reserved for future witticisms ...

--

Mike Leone

unread,

Sep 20, 2021, 12:21:37 PM9/20/21

to NTSysAdmin

On Sat, Sep 18, 2021 at 1:39 PM Philip Elder <Phili...@mpecsinc.ca> wrote:

It’s part of our routine to go through Sites, DNS, Trusts, and ADUC to make sure all remnants of a demoted DC are removed as there is always some.

Yeah, I'm still seeing gc and SRV/LDAP records point to non-existent DCs (they were gracefully demoted).

IIRC, the stub/delegation zone will show up after the root _MSDCS.DOMAIN.COM Forward Lookup Zone gets recreated.

The stub zone (delegation) being missing may also be a symptom of having that G00g DNS server IP in the DC’s DNS list.

https://servergurunow.wordpress.com/2017/09/29/recreate-the-_msdcs-dns-zone/

^^^

This tells us that to get to the stub/delegation we need to create the _msdcs.domain.com Forward Lookup Zone and restart NETLOGON then allow time for replication to take place.

OK, I'll see if I can get approval to add those zones (we have this new change policy, any change like this must come with some direction from MS, not from just a blog site or Reddit or mailing list ... problem is, the KB link is now dead. And searching the MS site for "missing zones" gives me nothing that looks remotely like this issue ...

Anyone know an actual MS link that basically says the same thing as the link above??

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/566765e882654362961057ebaebace60%40MPECSInc.Ca.

Jim Behning

unread,

Sep 20, 2021, 2:48:47 PM9/20/21

to ntsys...@googlegroups.com

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2BgFhiY01ZuSwGXPxbemf7LN7_4Xb9regBZ0wS5YMUyBmQ%40mail.gmail.com.

Jim Behning

unread,

Sep 20, 2021, 2:54:03 PM9/20/21

to ntsys...@googlegroups.com

My phone will not let me type if I try to do a copy and paste. We used to do swing migrations from old SBS to new SBS. Jeff Middleton had a product called Swing Migration that had step by step of numerous Microsoft articles using tools like adsiedit to clean up orphaned objects. Mariette Knapp also has some articles. Surely some searches on adsiedit would get you some Microsoft articles.

Mike Leone

unread,

Sep 20, 2021, 2:56:06 PM9/20/21

to NTSysAdmin

On Mon, Sep 20, 2021 at 2:48 PM Jim Behning <jimbe...@gmail.com> wrote:

https://blogs.msmvps.com/acefekay/2010/10/04/complete-step-by-step-to-remove-an-orphaned-domain-controller/

It's not orphaned. There are just a few DNS records left. Nothing in AD, or in Sites and Services.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CALOTwGYnmoGdxyMS-wi16LP9Hm7p5Ys0O%2B%2Bf6dimtcrQT05haQ%40mail.gmail.com.

Jim Behning

unread,

Sep 20, 2021, 3:27:28 PM9/20/21

to ntsys...@googlegroups.com

If you know they are gone then why haven't you deleted those DNS records?

You have run adsiedit searching for orphans? I thought you posted something with name resolution issues, then you also had 8.8.8.8 listed for DNS which seems odd as your internal domain should not be known to outside public DNS servers. You do not have to post ipconfig/all but my limited experience in servers the past 20 years I have seen a fair number of DNS configuration errors.

I could not swear to it as it has been a while since I have dug out stale records but I thought some things just aren't all that easy to find with gui that appear with command line/PowerShell work.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2Bg63UMMydZf2hnkYbvaQWSKFCpE_4y8HC9OQvH_Gn44oA%40mail.gmail.com.

Mike Leone

unread,

Sep 22, 2021, 10:21:10 AM9/22/21

to NTSysAdmin

On Mon, Sep 20, 2021 at 3:27 PM Jim Behning <jimbe...@gmail.com> wrote:

If you know they are gone then why haven't you deleted those DNS records?

Didn't realize they were still there, until I happened to see them, while troubleshooting this problem. :-)

The records I'm referring to are in the root domain. They don't exist in the sub-domain (which is where the retired DC was).

I didn't think to go look in the root, since the sub-domain was properly cleaned up. Silly me, I thought it would have also cleaned up the root ...

Example

You have run adsiedit searching for orphans? I thought you posted something with name resolution issues, then you also had 8.8.8.8 listed for DNS which seems odd as your internal domain should not be known to outside public DNS servers.

That was on 1 DC, in the root domain, that had that as a DNS entry on a NIC. I've removed that.

You do not have to post ipconfig/all but my limited experience in servers the past 20 years I have seen a fair number of DNS configuration errors.

I see no errors on any ipconfig /all that I looked at. (with the exception of that 1 DC that had the external DNS). I have 9 DCs at the moment ... :-)

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CALOTwGbzNT9haEiMO_OPAHddxA5OCYwKZhbrF73O%3D79v2JZpzw%40mail.gmail.com.

Mike Leone

unread,

Sep 22, 2021, 10:32:37 AM9/22/21

to NTSysAdmin

On Sat, Sep 18, 2021 at 1:39 PM Philip Elder <Phili...@mpecsinc.ca> wrote:

The stub zone (delegation) being missing may also be a symptom of having that G00g DNS server IP in the DC’s DNS list.

https://servergurunow.wordpress.com/2017/09/29/recreate-the-_msdcs-dns-zone/

^^^

This tells us that to get to the stub/delegation we need to create the _msdcs.domain.com Forward Lookup Zone and restart NETLOGON then allow time for replication to take place.

OK! (sorry, I was off for a bit) So my boss is asking:

Can this be done during hours? What is the impact to end users and systems?

I'm thinking: Yes, it can be done during normal business hours. No, no one should notice anything, especially since we are creating this zone in the root domain, and all end users and servers are in the sub-domain.

In the sub-domain, the "_msdcs" zone is not at the top level under Forward Lookup Zones, but the dcdiag errors specifically say (as an example) " _ldap._tcp.gc._msdcs.ads.pha.phila.gov" (that's the root domain) and not " _ldap._tcp.gc._msdcs.wrk.ads.pha.phila.gov", which would be the sub-domain (being a sub-domain, I guess there is no "gc" zone).

Am I right in thinking that there should be no impact to end users in the sub-domain?

Mike Leone

unread,

Nov 30, 2021, 11:36:13 AM11/30/21

to ntsys...@googlegroups.com, Ken Owens

Sorry to bring up an old post, but I *finally* convinced my boss to allow us to fix this issue.

So, to recap: I have a root and child domain config. Apparently, I need to change my DNS zones (because they were originally created during Win 2000, and now Win 2019 expects the sub-zones to be in a different location). To fix my missing records messages, I need to create in my ROOT domain, and new zone under Forward Lookup Zone, called "_msdcs.ads.pha.phila.gov" (since my current "_msdcs.ads.pha.phila.gov" is underneath the zone "ads.pha.phila.gov", and apparently Win 2019 expects it to not be under there, but to be at the same level as "ads.pha.phila.gov").

Correct? I would then have 2 zones under "Forward Lookup Zone", the _msdcs and the actual domain name ads.pha.phila.gov. And AD would auto-populate the new zone, one I restart the NETLOGON service of the DC I am doing this maintenance on.

This part confuses me -

This link https://servergurunow.wordpress.com/2017/09/29/recreate-the-_msdcs-dns-zone/ also says "Same steps need to be carried out to create Domain.com zone.".

But I already have the "ads.pha.phila.gov" zone. Does this mean I need to delete *THAT* zone? That seems very not right to me ..

Anyone?

On Sat, Sep 18, 2021 at 1:39 PM Philip Elder <Phili...@mpecsinc.ca> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/566765e882654362961057ebaebace60%40MPECSInc.Ca.

Glen Johnson

unread,

Nov 30, 2021, 11:53:26 AM11/30/21

to ntsys...@googlegroups.com

We went through that process a couple years ago and I did NOT delete the main zone, Just create the _msdcs.. zone. If I remember correctly you will need to restart the netlogon service on all your domain controllers so they all populate the needed records.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2BgnRF%2BmX%3DTFksc5jj3uLu5VL6zKivZkZ_E%3DogcA805kzg%40mail.gmail.com.

Mike Leone

unread,

Nov 30, 2021, 12:07:36 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

On Tue, Nov 30, 2021 at 11:53 AM Glen Johnson <gjohns...@email.vccs.edu> wrote:

We went through that process a couple years ago and I did NOT delete the main zone, Just create the _msdcs.. zone. If I remember correctly you will need to restart the netlogon service on all your domain controllers so they all populate the needed records.

Excellent information! I love talking to people with actual hands-on experience. LOL

I was hoping I wouldn't have to delete any zones, just create a new one. I will do a full DNS backup using the "dnscmd /zoneexport ads.pha.phila.gov ads.pha.phila.gov.ZoneExport-2021-11-30.BAK". Then just create the new zone in the parent domain, and restart the NETLOGON service. And restart the same service on all the other DCs in this domain (5 - 2 older Win 2008 R2, and 3 new Win 2019 replacement DCs), if need be.

Once that new zone is created and populated, the dcdiag tests on the Win 2019 DCs in the *child* domain should now all pass, since it will find the _msdcs.<parent_zone>" that it is expecting.

At least, that's my story, and I'm sticking to it! LOL

I'll do that tonight, after hours. I'll let you all know how it works out.

(the idea is to raise the FFL/DFL to the highest level - which is still Win 2016 - before we start making a new, single level, more sanely named AD domain, and planning a migration to it). I want no warnings or errors in DCDIAG before I start even planning a migration from one domain to another ...)

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CA%2B_GP0D1OVuqZ0Z6UXQGYxOzbnHk%3DZ7AEe2SPK%3D-1i4p6-hL5g%40mail.gmail.com.

Jonathan Raper

unread,

Nov 30, 2021, 1:54:00 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

I've personally cleared this "error" more times than I can count.

Creating the Delegation should do it. If you get an error that it already exists, then you'll have to delete the original _msdcs and then go through the steps again.

Jonboy

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2BhhV%3DK9J8kCxt7k62eeXFpZvPMwSNcohENVeb1%3DkxEWBA%40mail.gmail.com.

Philip Elder

unread,

Nov 30, 2021, 2:07:07 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

I only see the need to create the _msdcs.ads.pha.phila.gov FLZ and restart NETLOGON/DNS to get the stub zone to show up correctly (grey).

We then add any additional DCs/DNS servers to the stub zone because ADDS does not pick them up on its own.

Yes, the _msdcs domain and the ADDS domain should be at the same level.

I suspect that someone in the past figured it needed to be nested since the grey stub zone _msdcs was missing? At least that’s my guess.

Philip Elder MCTS

Senior Technical Architect

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2BgnRF%2BmX%3DTFksc5jj3uLu5VL6zKivZkZ_E%3DogcA805kzg%40mail.gmail.com.

Philip Elder

unread,

Nov 30, 2021, 2:07:18 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

Just creating the _msdcs stub zone under the main domain FLZ won’t do it as far as I remember. It will stay yellow instead of turning grey.

Philip Elder MCTS

Senior Technical Architect

https://social.technet.microsoft.com/Forums/en-US/4bd4c09b-d128-4356-8506-58d231ee8ef9/rodc-servers-dns-service-stops-and-dhcp-gets-unauthorized?forum=winserverDS

Web: www.mpecsinc.com

Blog: blog.mpecsinc.com

Twitter: Twitter.com/MPECSInc

Skype: MPECSInc.

Please note: Although we may sometimes respond to email, text and phone calls instantly at all hours of the day, our regular business hours are 8:00 AM - 5:00 PM, Monday thru Friday.

From: ntsys...@googlegroups.com <ntsys...@googlegroups.com> On Behalf Of Jonathan Raper

Sent: November 30, 2021 11:54
To: ntsys...@googlegroups.com
Cc: Ken Owens <Kennet...@pha.phila.gov>

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAAgW6%2BbNpoCoU6HuOaUtAhWV4QHoG7WE18R85MPaBssiVzNYfg%40mail.gmail.com.

Glen Johnson

unread,

Nov 30, 2021, 2:12:39 PM11/30/21

to ntsys...@googlegroups.com

Nested was how it was created if the domain was initially created in windows 2000. Ours was.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/f0590a193aee4fbc9b2ff65b51aa1a4a%40MPECSInc.Ca.

Mike Leone

unread,

Nov 30, 2021, 2:14:09 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

On Tue, Nov 30, 2021 at 2:07 PM Philip Elder <Phili...@mpecsinc.ca> wrote:

Just creating the _msdcs stub zone under the main domain FLZ won’t do it as far as I remember. It will stay yellow instead of turning grey.

So what are you sayimg? That in addition to create the new _msdcs zone under Forward Lookups, I will also have to delete the existing _msdcs that's under the domain name? Because right now, that zone is fully populated ....

Mike Leone

unread,

Nov 30, 2021, 2:17:18 PM11/30/21

to ntsys...@googlegroups.com

On Tue, Nov 30, 2021 at 2:12 PM Glen Johnson <gjohns...@email.vccs.edu> wrote:

Nested was how it was created if the domain was initially created in windows 2000. Ours was.

I believe ours was created in Win 2000 as well. This structure was in place when I got here in 2007, having been implemented in either 2003 or 2005, I was told back then ...

This site seems to agree:

It seems our issue was really caused by the fact that our DNS zone was originally created in Win2000. These devices were eventually upgraded to Win2003, and most recently, migrated to 2008 R2.

Win2000 implemented _msdcs as a subfolder of the DNS zone. The recommended config for 2003 and 2008 AD-Integrated DNS zones, is that _msdcs be moved to a separate AD-integrated primary zone as _msdcs.ForestFQDN. However, the zones created in 2000 are not changed to this config when DNS is upgraded or migrated 2003 or 2008.

To fix this you need to manually create a new "separate"active directory integrated primary zone _msdcs.ForestFQDN and remove the old subfolder under the existing primary zone. (after successful config and replication). Then run your best practices analyzer in 2008R2 and see the problem is no longer...

Philip Elder

unread,

Nov 30, 2021, 2:31:10 PM11/30/21

to ntsys...@googlegroups.com

That’s interesting. Getting Group Policy going back then was also a lot of fun.

I can see how Microsoft went, “Oops!” and restructured things possibly via Service Pack (?) or in 2003 it was correct with _msdcs.domain.local and domain.local being at the same FLZ level.

Philip Elder MCTS

Senior Technical Architect

Web: www.mpecsinc.com

Blog: blog.mpecsinc.com

Twitter: Twitter.com/MPECSInc

Skype: MPECSInc.

Please note: Although we may sometimes respond to email, text and phone calls instantly at all hours of the day, our regular business hours are 8:00 AM - 5:00 PM, Monday thru Friday.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CA%2B_GP0AdVngz_X273_fd72nncDMwdK8du7%2BmX7gsaHU7Pbs1Tw%40mail.gmail.com.

Philip Elder

unread,

Nov 30, 2021, 2:39:31 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

Your follow up e-mail to this one seems to have the right series of steps in it.

Philip Elder MCTS

Senior Technical Architect

Web: www.mpecsinc.com

Blog: blog.mpecsinc.com

Twitter: Twitter.com/MPECSInc

Skype: MPECSInc.

Please note: Although we may sometimes respond to email, text and phone calls instantly at all hours of the day, our regular business hours are 8:00 AM - 5:00 PM, Monday thru Friday.

From: ntsys...@googlegroups.com <ntsys...@googlegroups.com> On Behalf Of Mike Leone
Sent: November 30, 2021 12:14
To: ntsys...@googlegroups.com
Cc: Ken Owens <Kennet...@pha.phila.gov>
Subject: Re: [ntsysadmin] Problem: the AD zone _msdcs.<domain> was not found

On Tue, Nov 30, 2021 at 2:07 PM Philip Elder <Phili...@mpecsinc.ca> wrote:

Just creating the _msdcs stub zone under the main domain FLZ won’t do it as far as I remember. It will stay yellow instead of turning grey.

So what are you sayimg? That in addition to create the new _msdcs zone under Forward Lookups, I will also have to delete the existing _msdcs that's under the domain name? Because right now, that zone is fully populated ....

Philip Elder MCTS

Senior Technical Architect

Microsoft High Availability MVP

E-mail: Phili...@mpecsinc.ca

Phone: +1 (780) 458-2028

Web: www.mpecsinc.com

Blog: blog.mpecsinc.com

Twitter: Twitter.com/MPECSInc

Skype: MPECSInc.

Please note: Although we may sometimes respond to email, text and phone calls instantly at all hours of the day, our regular business hours are 8:00 AM - 5:00 PM, Monday thru Friday.

From: ntsys...@googlegroups.com <ntsys...@googlegroups.com> On Behalf Of Jonathan Raper
Sent: November 30, 2021 11:54
To: ntsys...@googlegroups.com
Cc: Ken Owens <Kennet...@pha.phila.gov>
Subject: Re: [ntsysadmin] Problem: the AD zone _msdcs.<domain> was not found

I've personally cleared this "error" more times than I can count.

Creating the Delegation should do it. If you get an error that it already exists, then you'll have to delete the original _msdcs and then go through the steps again.

Jonboy

--

You received this message because you are subscribed to the Google Groups "ntsysadmin" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ntsysadmin+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2BjBUjj%2B7noX3nR5EH7%2BogrLM3NwDm3pNr_RE6%3D%2ByytRfA%40mail.gmail.com.

Melvin Backus

unread,

Nov 30, 2021, 3:17:02 PM11/30/21

to ntsys...@googlegroups.com

Certainly not the only time they changed the way things worked but didn’t change them if you were already using it the old way. Then 3+ iterations later they deprecate it.

--
There are 10 kinds of people in the world...
those who understand binary and those who don't.

¯\_(ツ)_/¯

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/680d362afc58427a926a65ee8899dd95%40MPECSInc.Ca.

Mike Leone

unread,

Nov 30, 2021, 7:51:31 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

OK! So I created that new _msdcs in the Forward Lookup Zone of the root domain, and it autpoulated. But I'm getting errors in dcdiag on the *child* domain. For example, dcdiag on a DC in the *child* domain:

Error:
Missing SRV record at DNS server 10.64.7.59:
_ldap._tcp.34818731-cabd-4d11-96e6-7886efde2209.domains._msdcs.ads.pha.phila.gov
[Error details: 9003 (Type: Win32 - Description: DNS name does not exist.)]

Now, .59 is a DC in my *child* domain, but "_msdcs.ads.pha.phila.gov" is my *root* domain.

So what I'm wondering is, do I need to create the _msdcs under the Forward Lookup Zone of the *child* domain now??

AND now I'm seeing failing in test "KnowsOfRoleHolders" - RPC server is unavailable ...." (of 2 different DCs than the one referenced above). So RidManager is failing, since it says it can't contact the RID Manager DC ...

I'm tempted to just let it percolate overnight. But the repadmin shows no errors, so it's replicating ...

Ideas, anyone????

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/255c33b225a54378b814e141fa6a7339%40MPECSInc.Ca.

Mike Leone

unread,

Nov 30, 2021, 8:53:52 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

MORE:

So now, I created the new _msdcs zone properly under Forward Lookup Zones. BUT ... now there are only 3 sub-zones (dc, domains, gc". I do NOT have a "pdc" sub-zone, and I used to ...

Plus, apparently I'm still missing LDAP SRV records, and others, now for servers in the root domain, not just the child domain.

<sigh> I may have made things worse ...

Any ideas how I can fix this? I need that "pdc" sub-zone under _msdcs, but I for sure don't want to create it manually .... how can I get the system to do that? If I do a "netdom query fsmo", it know which server is the PDC, but there's no entry like that in DNS now ...

Any advice greatly appreciated!

I'm going to leave it for now, and bang away at it again in the morning ...

Henry Awad

unread,

Nov 30, 2021, 10:57:34 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

I would recommend waiting a few hours for replication to take place and if the missing records are not created by morning to call Microsoft support before making things worse by digging yourself a bigger hole when trying to create these important records yourself. It better to pay a couple of hundred dollars and get expert assistance as they would be able to see your configuration and what's missing as well as access to server logs to help you fix the issue quickly. And if you feel like the support person is not knowledgeable enough ask to escalate the case. The last thing you want is for your endpoints to lose AD authentication or connectivity due to DNS issues.

As a last ditch effort before calling MS, you could try to move one of the FSMO roles and see if the SRV record is added/changed.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2BiHSnYrYvyEYA4Wpz0XSVRofWRust4mR%2BZ7fFZFb%3D%2B2vg%40mail.gmail.com.

Philip Elder

unread,

Nov 30, 2021, 11:32:36 PM11/30/21

to ntsys...@googlegroups.com, Ken Owens

Mike,

What’s the FQDN of the domain missing the stub?

Philip Elder MCTS

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2BiHSnYrYvyEYA4Wpz0XSVRofWRust4mR%2BZ7fFZFb%3D%2B2vg%40mail.gmail.com.

Mike Leone

unread,

Dec 1, 2021, 7:52:41 AM12/1/21

to ntsys...@googlegroups.com, Ken Owens

On Tue, Nov 30, 2021 at 11:32 PM Philip Elder <Phili...@mpecsinc.ca> wrote:

Mike,

What’s the FQDN of the domain missing the stub?

ads.pha.phila.gov. This is the domain where I created a new _msdcs zone under Forward Lookup Zone.

UPDATE: The next morning, the "pdc" stub zone is still missing. I have NS and CNAME records for it in the _msdcs and A records in the actual domain zone.However, I have no _kerberos or _ldap records in the _tcp under the _sites, or under domains, and not under gc, either.

At this point, I think I will recommend officially to my boss that we pay the money to Microsoft. I am way too scared to continue futzing with this. The only good thing is ... all the users and assets are in the child domain, and we can all still login fine, access shared folders fine, etc. And I do NOT want to risk that any further (although I doubt it would, since the DNS for the child domain seems to be fine - it's resolving forward and backward. Then again, the DNS for the parent domain is also resolving forward and backward) ...

I tried restarting the NETLOGON service on the host that has the PDC role, and even ipconfig /registerdns, and nltest /dsregdns. Nothing ... I really think it's time for some professional help ....

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/7eab22feb71a48189804fa7fb1872f49%40MPECSInc.Ca.

Mike Leone

unread,

Dec 1, 2021, 10:18:19 AM12/1/21

to ntsys...@googlegroups.com, Ken Owens

UPDATE:

OK, so I created an isolated VM and created a new Win 2019 DC there, so I could look at it's DNS structure. And as I suspected, my root domain is missing the "pdc" stub zone, and it's stub zone "_tcp", and the record within - the "_ldap" SRV record.

On my root domain, I am missing this stub zone completely. I am also missing almost all records that reference the PDC in my manually created _msdcs zone (meaning: I have an NS record and a CNAME record for the host that is the PDC in _msdcs, but that's it. There are no records that reference it in any stub zone under that - no entries under _tcp under dc, or under domains, nothing.

So what I am thinking is this:

I have entries in my production DNS from previously demoted and destroyed DCs. They were never removed (I thought for sure I had checked for that, but maybe I was only looking in the child domain, not here in the root domain). I want to clean up my DNS as much as possible first- I will make sure those servers are gone in Sites and Services. I will then go through all those DNS stub zones (gc, domains, etc), and make sure there are no SRV records that reference any non-existent DCs.

THEN ... I guess we will try creating a new zone under _msdcs called pdc. Then a new stub inside THAT called _tcp (presuming it doesn't get auto-created). If I have to, then I will create the SRV record in there, that points to the PDC for that domain.

One of our onsite consultants says he had to do that for a client once - MS told him to just create the missing zone (_msdcs, in his case), and then all was well.

AND ... if the other records don't then auto-create themselves (i.e., an SRV entry under gc, under domains, etc) that point to the PDC host, then I create those SRV records, too.

Sound like a plan?

Under the actual domain name zone (ads.pha.phila.gov", I do see entries for that PDC host - I see gc, _kerberos, _passwd, _ldap records for it, just like for all the other DCs in the domain. It's only the _msdcs zone (that I created last night) that is missing almost all references to the PDC host (i.e., SRV records).

I am so sick of this domain. LOL

Glen Johnson

unread,

Dec 1, 2021, 12:11:43 PM12/1/21

to ntsys...@googlegroups.com

It doesn't appear that it needs to be gray.

Here is a pic of a test 2016 DNS that has never been modified.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/97dca0661dcf4aeebe44918ad2c6ebd0%40MPECSInc.Ca.

Mike Leone

unread,

Dec 1, 2021, 12:57:29 PM12/1/21

to ntsys...@googlegroups.com, Ken Owens

UPDATE 2:

I think it's (mostly) resolved!

I had to manually make new entries for the "pdc" and "_tcp" zones (which weirdly are created by choosing "new domain" ..). Then I had to add new records for each missing SRV - _ldap, _kerberos, etc - in all the sub-zones.

THEN the dcdiag passed, even on the actual PDC (which now was able to find itself using the Locator check).

So YAY! <whew> Finally ...

so that root domain is back to passing all it's dcdiag tests. YAY!

The child domain Win 2019 DCs are still having issues - the DNS tests of dcdiag seem to go looking for A and SRV records for child DCs in _ldap._tcp.gc._msdcs.ads.pha.phila.gov. In other words, in the _msdcs of the *root* domain, even though this is a *child* domain ... What's up with THAT??

Otherwise, except for the root PDC not passing the Dynamic DNS update test (dunno what's up with that; I suspect permissions ...), all the dcdiag tests are passing for my root domain. And the child domain Win 2019 DC is failing some DNS test, but otherwise passing ...

So that's progress!

Philip Elder

unread,

Dec 1, 2021, 3:46:17 PM12/1/21

to ntsys...@googlegroups.com, Ken Owens

In DNS then:

ads.pha.phila.gov

_msdcs.ads.pha.phila.gov

Both would be at the same level.

After restarting the DNS and NETLOGON services the _msdcs greyed stub zone folder should show up with the NS record for the DC the work was done on IIRC. Then, double click the stub and add the other NS servers for the zone.

That should do it.

Please System State Backup prior.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2BgxHZ%2B9NNZ9tenBnnYucDVeHnp1%3DSYncuQLZsme4qPsoA%40mail.gmail.com.

Philip Elder

unread,

Dec 1, 2021, 3:48:49 PM12/1/21

to ntsys...@googlegroups.com

Verify that the correct NS record(s) are seated within the folder.

If it’s not grey, something is missing.

Philip Elder MCTS

Senior Technical Architect

https://blog.mpecsinc.com/2019/05/13/ad-ds-operation-failed-directory-service-is-missing-mandatory-configuration-event-id-2091-fsmo-role-broken/

Web: www.mpecsinc.com

Blog: blog.mpecsinc.com

Twitter: Twitter.com/MPECSInc

Skype: MPECSInc.

Please note: Although we may sometimes respond to email, text and phone calls instantly at all hours of the day, our regular business hours are 8:00 AM - 5:00 PM, Monday thru Friday.

From: ntsys...@googlegroups.com <ntsys...@googlegroups.com> On Behalf Of Glen Johnson
Sent: December 1, 2021 10:11
To: ntsys...@googlegroups.com
Subject: Re: [ntsysadmin] Problem: the AD zone _msdcs.<domain> was not found

It doesn't appear that it needs to be gray.

Here is a pic of a test 2016 DNS that has never been modified.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CA%2B_GP0BBGPQvkdN4fn4feOH2rE9dpb2W3NAYrZkyN%2BTGyPzYOg%40mail.gmail.com.

Mike Leone

unread,

Dec 2, 2021, 9:26:34 AM12/2/21

to ntsys...@googlegroups.com, Ken Owens

On Wed, Dec 1, 2021 at 3:46 PM Philip Elder <Phili...@mpecsinc.ca> wrote:

In DNS then:

ads.pha.phila.gov

_msdcs.ads.pha.phila.gov

Both would be at the same level.

After restarting the DNS and NETLOGON services the _msdcs greyed stub zone folder should show up with the NS record for the DC the work was done on IIRC. Then, double click the stub and add the other NS servers for the zone.

That should do it.

This turns out not to be the case ...

Previously, I had my domain zone ads.pha.phila.gov. Under it I had _msdcs.ads.pha.phila.gov. Inside there I had "domains" zone. Inside *that*, I had 2 entries that look like GUIDs - one for the root domain, one for the child domain. Each had a _tcp zone, with various SRV records - _ldap - one for each DC in the domain.

After creating _msdcs.ads.pha.phila.gov at the same level as the domain zone ads.pha.phila.gov (and restarting all NETLOGOIN services), it created a "domains ". But then only created *1* entry inside there, for the root domain. It did *not* have the 2nd entry, that corresponds to the child domain. (i used dnscmd to backup the ads.pha.phila.gov zone before I started, and I do see those records in that file, so I know they used to exist in that location).

Additionally, inside the domains zone it did create the same GUID entry as the root domain, and it's _tcp zone, and SRV records for *almost* all the DCs in the root domain. But it did *not* create the _ldap record for the DC that holds the PDC for that domain. I had to create that one manually.

So, to make the dcdiag work for the child domain, I had to create an entry in that "domains" for the child domain (using the same GUID looking identifier as before), then a _tcp, and SRV records for the child domain DCs in there.

Glen Johnson

unread,

Dec 2, 2021, 10:03:28 AM12/2/21

to ntsys...@googlegroups.com

Verified the records are there. That is a vm stood up as a test server, nothing done to it, so if it should be gray and isn't then the DCPromp process didn't do what it was supposed to.

Imagine that. ;)

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/62b96d605e1b45d89878c659c5c7f0cb%40MPECSInc.Ca.

James Iversen

unread,

Dec 2, 2021, 10:06:32 AM12/2/21

to ntsys...@googlegroups.com

Have you verified the ACL's on the newly created zone?

I had to do something similar and found several stale records...

Some orgs delegate permissions on DNS zones to allow non DNSAdmins to make changes.

Just a thought.

James Iversen 1899 Central Plaza East Edmeston, NY 13335
Phone:	(607) 965-2706
nycm.com

From: "Glen Johnson" <gjohns...@email.vccs.edu>
To: ntsys...@googlegroups.com
Date: 12/02/2021 10:03 AM
Subject: Re: [ntsysadmin] Problem: the AD zone _msdcs.<domain> was not found

Sent by: ntsys...@googlegroups.com

ATTENTION: This email was sent from someone outside of NYCM.

.
To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CA%2B_GP0CYkFUfEKOoCOiz71PJ7YnGREAfk6%3D-_bZ92%2BaScDDzRQ%40mail.gmail.com.

Join us on Facebook at www.facebook.com/NYCMInsurance.

***CONFIDENTIALITY NOTICE***

This email and any attachments to it are confidential and intended solely for the individual or entity to whom it is addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you have received this email in error, please contact the sender by reply email and destroy all copies of the original message.

Glen Johnson

unread,

Dec 2, 2021, 10:37:31 AM12/2/21

to ntsys...@googlegroups.com

If you are asking me, no and since this is a throw-away vm, I'm not going to do any troubleshooting. I had just stood this up to look at a default DNS zone as created by running DCPromo and creating a new domain.

I might try it with a 2022 server and see what it looks like, just for grins.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/OF865CCAF3.44530963-ON8525879F.0052D517-8525879F.0052FD90%40nycm.com.

Mike Leone

unread,

Dec 2, 2021, 10:38:38 AM12/2/21

to ntsys...@googlegroups.com

No delegations have been done, that I am aware of. No, haven't had time to check ACLs.

On Thu, Dec 2, 2021 at 10:06 AM James Iversen <JIve...@nycm.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/OF865CCAF3.44530963-ON8525879F.0052D517-8525879F.0052FD90%40nycm.com.

Philip Elder

unread,

Dec 2, 2021, 12:39:10 PM12/2/21

to ntsys...@googlegroups.com, Ken Owens

While not completely in alignment this is one reason why we don’t do upgrades.

We’re in a situation where there are so many hidden bits inherited from the initial domain creation with all its flaws.

There is lots of weirdness in AD Integrated DNS that have never been “fixed”. Like records not being properly expunged when a DC is DCPromo’d out of the domain for one example. We have to go through the various AD management consoles and DNS cleaning up the records from that old DC.

This:

That is a repost of this:

http://blog.mpecsinc.ca/2011/03/ad-ds-operation-failed-directory.html

That’s ten years ago and folks are still commenting on it saying, “Yup, hit us too thanks!”

Lots of fun. ;0)

Philip Elder MCTS

Web: www.mpecsinc.com

Blog: blog.mpecsinc.com

Twitter: Twitter.com/MPECSInc

Skype: MPECSInc.

Please note: Although we may sometimes respond to email, text and phone calls instantly at all hours of the day, our regular business hours are 8:00 AM - 5:00 PM, Monday thru Friday.

From: ntsys...@googlegroups.com <ntsys...@googlegroups.com> On Behalf Of Mike Leone
Sent: December 2, 2021 07:26
To: ntsys...@googlegroups.com
Cc: Ken Owens <Kennet...@pha.phila.gov>
Subject: Re: [ntsysadmin] Problem: the AD zone _msdcs.<domain> was not found

On Wed, Dec 1, 2021 at 3:46 PM Philip Elder <Phili...@mpecsinc.ca> wrote:

--

You received this message because you are subscribed to the Google Groups "ntsysadmin" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ntsysadmin+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/CAHBr%2B%2BhAouQrv_VhcDB401L0ad-EZD5tLNE0Bth00H1bSea19Q%40mail.gmail.com.

Jonathan Raper

unread,

Dec 2, 2021, 12:50:37 PM12/2/21

to ntsys...@googlegroups.com

"...this is one reason why we don’t do upgrades."

So I'll bite....what's your methodology?

Jonboy

To view this discussion on the web visit https://groups.google.com/d/msgid/ntsysadmin/500106f845b44d6ca789baf2d56f7efd%40MPECSInc.Ca.

Philip Elder

unread,

Dec 2, 2021, 1:09:30 PM12/2/21

to ntsys...@googlegroups.com

For DCs it’s setting up new VMs and DCPromo in. DCPromo the old ones out, clean up, and verify.

For workloads it’s migrate to fresh.

It’s rare that we need to hold on to something. There’s a few accounting apps that we need to hold on to a Windows Server 2008/R2 or Windows XP/7 but not many.

If we do, the legacy setup gets VLAN’d off or physically segmented off with routing to prevent any access out to the Internet or in from it.

Philip Elder MCTS