[Samba] showrepl is showing a deleted connexion

[MORILLO Jordi]

Hi everybody,

One of my DC crash this afternoon (dead disk).
I can't remove this DC server from windows GUI (computer object from < users and computers >) and NTDS settings from < sites and services > because windows GUI error.

So i manually remove this old server :

- Clean all DNS stuff (tpc, sites, kerberos, kpasswd, srv entries.....)

- With apache directory studio, i connect to ldap and remove NTDS settings under site's tree (configuration -> sites -> my_old_site)
After that, windows GUI is good, no more DC's computer object or NTDS settings

But
A samba-tool drs showrepl gives :

==== OUTBOUND NEIGHBORS ====
....
DC=pr,DC=educationetformation,DC=fr
NTDS DN: CN=NTDS Settings\0ADEL:1e23b3de-ae49-406d-bd33-e233b168945c,CN=DC540\0ADEL:ceeb7300-2411-4e05-83e2-e4ebf521f145,CN=Servers\0ADEL:85d2165b-0a31-4f90-be71-e2b73c8eb88a,CN=SaintSaens\0ADEL:f23842e5-e22b-4ad2-9cb3-a72fe0dd73dd,CN=Sites,CN=Configuration,DC=pr,DC=educationetformation,DC=fr
DSA object GUID: 1e23b3de-ae49-406d-bd33-e233b168945c
Last attempt @ Thu Jan 21 19:44:00 2016 CET failed, result 87 (WERR_INVALID_PARAM)
1932 consecutive failure(s).
Last success @ NTTIME(0)
....

This object is not visible from ldap but is visible with ldbsearch on CONFIGURATION ldb
If I ldbdel this object, samba-tool drs showrepl failed :

==== OUTBOUND NEIGHBORS ====

ERROR(runtime): DsReplicaGetInfo of type 4294967294 failed - (8442, 'WERR_DS_DRA_INTERNAL_ERROR')

So I ldbadd this object (previously backup up), no more ERROR(runtime) but i can see again wrong connexion from samba-tool drs showrepl....
Any idea to clean drs showrepl from this deleted object ?
Thanks for all
Samba 4.3.3

--
To unsubscribe from this list go to the following URL and read the
instructions: https://lists.samba.org/mailman/options/samba

[Stefan Kania]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

You shoud remove alle DC-date with this script:
https://gallery.technet.microsoft.com/scriptcenter/d31f091f-2642-4ede-9f
97-0e1cc4d577f3
Than you can ben sure that alle the metadate is removed. Then clean
only the DNS-entries by hand

- --
Stefan Kania
Landweg 13
25693 St. Michaelisdonn


Signieren jeder E-Mail hilft Spam zu reduzieren. Signieren Sie ihre
E-Mail. Weiter Informationen unter http://www.gnupg.org

Mein Schlüssel liegt auf

hkp://subkeys.pgp.net

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iEYEARECAAYFAlah5CEACgkQ2JOGcNAHDTbmoQCfdKK0uNK5QUmqyN0B6ZW1Sqvr
0jwAoKNnsFZmSNIXitYMmP8Wqr1CBXwj
=dZgV
-----END PGP SIGNATURE-----

[MORILLO Jordi]

Solved !
Thanks for the script.
In my case, it was just too late.
I have just found a ugly but working solution:
From Configuration, Schema, Domaindnszones, forestdnszones and principal, I remove using ldbdel a "repsTo" binary object.
No more trouble with drs showrepl :-)

-----Message d'origine-----
De : samba [mailto:samba-...@lists.samba.org] De la part de Stefan Kania
Envoyé : vendredi 22 janvier 2016 09:35
À : sa...@lists.samba.org
Objet : Re: [Samba] showrepl is showing a deleted connexion

[Denis Cardon]

Hi Jordi,

> Solved !
> Thanks for the script.
> In my case, it was just too late.
> I have just found a ugly but working solution:
> From Configuration, Schema, Domaindnszones, forestdnszones and principal, I remove using ldbdel a "repsTo" binary object.
> No more trouble with drs showrepl :-)

Indeed, samba-tool drs showrepl show in fact the repsfrom/repsto
attributes. They should be created / deleted by kcc. However I have seen
lingering repsto attributes in the past too and had to ldbedit to
cleanup the mess.

Ldbdel'eting an entry in "CN=Deleted Object" should be done with care.
In your case, you still had a repsto referencing the GUID of that
object, hence among other things the crash of samba-tool drs showrepl on
the OUTBOUND NEIGHBOR part of the listing. However, I guess the initial
condition is a bug and it should be the job of the KCC (or integrity
check) to delete a repsto pointing to an object in Deleted Objects.
Should check with Douglas and the dev team...

Cheers,

Denis

Denis Cardon
Tranquil IT Systems
Les Espaces Jules Verne, bâtiment A
12 avenue Jules Verne
44230 Saint Sébastien sur Loire
tel : +33 (0) 2.40.97.57.55
http://www.tranquil-it-systems.fr

[MORILLO Jordi]

Hi Denis,

I have seen in an old post that you have tested new KCC from full mesh to bridge head at a french school.
Is your "drs showrepl" correct on such DC's ?

In my case, a drs showrepl is showing a full mesh on inbound and outbound (not good) but only 1 KCC connection objects (good)
Where is a full description of my trouble: https://lists.samba.org/archive/samba/2015-December/196844.html
Best regards


-----Message d'origine-----
De : Denis Cardon [mailto:denis....@tranquil-it-systems.fr]
Envoyé : vendredi 22 janvier 2016 14:31
À : MORILLO Jordi <J.Mo...@educationetformation.fr>; sa...@lists.samba.org

[Denis Cardon]

Hi Jordi,

How is it going up there in Normandie?

> I have seen in an old post that you have tested new KCC from full mesh to bridge head at a french school.
> Is your "drs showrepl" correct on such DC's ?
>
> In my case, a drs showrepl is showing a full mesh on inbound and outbound (not good) but only 1 KCC connection objects (good)
> Where is a full description of my trouble: https://lists.samba.org/archive/samba/2015-December/196844.html

KCC does not remove existing outdated kcc objects by itself (or didn't,
if it has been changed in more recent versions). I had a chat with
Douglas about this a while back. However it should remove your
repsfrom/repsto attribute, unless you messed up the thing (I did once).
I also had in the past repsfrom/repsto pointing to deleted NTDSDSA
entries with the \0ADEL string.

Before asking samba_kcc to buildup the connexions, you have to define
the sites, put the DC in the correct site, remove the site from de
default_ip_link, and set up a link for each remote site to main site.
Actually the bridge head thing does not seem necessary to get the thing
working. With such a configuration, samba_kcc does build only the
necessary connexions, and by reading you post, it seems that you did it
properly, so that sounds good.

If you still have spurious repsfrom/repsto, I don't know if there is
another way to get rid it other than ldbedit'ing... By the way, did you
check in the _msdcs DNS zone that you don't have leftover CNAME entries
of your old servers?

In order to finish the setup, be sure to setup the subnet properly in
order for all windows to contact their nearest DCs. After having created
the sites, double check that all the _kerberos/_ldap entries under
_sites are properly created in the DNS server (sometime, they aren't).
After, you can check on a windows desktop at different site that it
knows on which site it is located in the windows registry in
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters
, check the value DynamicSiteName, and on a cmd.exe check the env
variable LOGONSERVER

Another hint: if you set up a star topology where remote sites cannot
see each others (especially if you have DROP/no_ip_unreachable firewall
rules), then you have also to be careful that during the process of
joining a new DC, the join process reads /etc/krb5.conf file and tries
to contact all the DC that are referenced, and thus if you use DNS SRV
records to resolv kdc addresses, it will try to contact all the servers.
In that case, you have to specify manually the kdc in that
/etc/krb5.conf file and not rely on the automatic DNS discovery.

Another corner case is that when having more than 40-50 kdc in the
domain, you may encounter another bug with /etc/krb5.conf file with
automatic KDC discovery through DNS SRV records, it looks like it is
just too much for libkrb5. In that case, you should also disable DNS
automatic kerberos discovery and specify a few useful kdc addresses in
the krb5.conf file by hand.

Cheers,

Denis

[MORILLO Jordi]

> Hi Jordi,
> How is it going up there in Normandie?

Hi Denis :-) not so bad even if it's a raining day (as usual in Normandie :-) )
I will reply to your brother's mail soon, i'll copied you in

> KCC does not remove existing outdated kcc objects by itself (or didn't, if it has been changed in more recent versions). I had a chat with Douglas about this a while back. However it should remove your repsfrom/repsto attribute, unless you messed up the thing (I did once).
> I also had in the past repsfrom/repsto pointing to deleted NTDSDSA entries with the \0ADEL string.

Hum... so if it sould remove repsfrom/repsto attribute, there is a problem in my ldap attribute. I have to play more with samba_kcc debug options and perhaps i should have a look to source code

> Before asking samba_kcc to buildup the connexions, you have to define the sites, put the DC in the correct site, remove the site from de default_ip_link, and set up a link for each remote site to main site.
> Actually the bridge head thing does not seem necessary to get the thing working. With such a configuration, samba_kcc does build only the necessary connexions, and by reading you post, it seems that you did it properly, so that sounds good.

Yes I've done all this things, sounds good

> If you still have spurious repsfrom/repsto, I don't know if there is another way to get rid it other than ldbedit'ing... By the way, did you check in the _msdcs DNS zone that you don't have leftover CNAME entries of your old servers?

_msdcs DNS zone is clean. Ok for playing with ldbedit but i'm always scared to hack samba'ldb directly on production. I will try to install a test environment for playing a bit more

> In order to finish the setup, be sure to setup the subnet properly in order for all windows to contact their nearest DCs. After having created the sites, double check that all the _kerberos/_ldap entries under _sites are properly created in the DNS server (sometime, they aren't).
> After, you can check on a windows desktop at different site that it knows on which site it is located in the windows registry in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters
> , check the value DynamicSiteName, and on a cmd.exe check the env variable LOGONSERVER

I always check DNS entries after DC domain join, DNS are essential's parts of Active Directory engine, isn't it ? :-)
As wiki's says (https://wiki.samba.org/index.php/Active_Directory_Sites), there's also "nltest /dsgetsite" and "nltest /dsgetdc:samdom" great commands

> Another hint: if you set up a star topology where remote sites cannot see each others (especially if you have DROP/no_ip_unreachable firewall rules), then you have also to be careful that during the process of joining a new DC, the join process reads /etc/krb5.conf file and tries to contact all the DC that are >referenced, and thus if you use DNS SRV records to resolv kdc addresses, it will try to contact all the servers.
>In that case, you have to specify manually the kdc in that /etc/krb5.conf file and not rely on the automatic DNS discovery.

Yes i'm in a star topology but no firewall/restriction about dc's talking to each other (VPN fully routed). Star topology permits to save bandwitdh on small adsl connection (even if ldap's exchange are low).
When DC's domain join, i'm using --server for pointing bridge head DC

> Another corner case is that when having more than 40-50 kdc in the domain, you may encounter another bug with /etc/krb5.conf file with automatic KDC discovery through DNS SRV records, it looks like it is just too much for libkrb5. In that case, you should also disable DNS automatic kerberos discovery and >specify a few useful kdc addresses in the krb5.conf file by hand.

I'll put it away in a corner of my brain. We do not planned to have more than 20 KDC. Maybe in 15 years if activity will grow inordinatly

Have a nice week, and happy new year for all your team :-)