Sync LDAP Username

[Nate Dreier]

Hello!

Version: 3.4
Auth: Active Directory (previously Slapd)

Issue: Update old username

I have recently shifted auth to use Active Directory. Previously we used a basic slapd. Both are LDAP.

In the old LDAP instance there are a small handful of users that their UID does not map 1:1 with our Active Directory's UID.

Is there a simple way to update a username?

I have looked around online and there are varying answers from varying years. Wondering if there is a new updated answer for 3.4

The two things that I've ruled out are using the API. Errors out since the accounts are controlled via LDAP (makes sense). The other is reverting back to the old LDAP, updating the LDAP UID entry to match the new UID. That I hoped would work but ultimately just creates a new account.

I have seen some rumblings from posts about updating All-Users.git. I am currently looking around seeing if I can figure that out. Any direction would be greatly appreciated.

Nate

https://maystreet.com

[Andrew Grimberg]

On 7/13/21 12:36 PM, Nate Dreier wrote:
> Hello!
>
> *Version: 3.4
> Auth: Active Directory (previously Slapd)*
> *Issue: Update old username*

>
> I have recently shifted auth to use Active Directory. Previously we used
> a basic slapd. Both are LDAP.
>
> In the old LDAP instance there are a small handful of users that their
> UID does not map 1:1 with our Active Directory's UID.
>
> Is there a simple way to update a username?
>
> I have looked around online and there are varying answers from varying
> years. Wondering if there is a new updated answer for 3.4
>
> The two things that I've ruled out are using the API. Errors out since
> the accounts are controlled via LDAP (makes sense). The other is
> reverting back to the old LDAP, updating the LDAP UID entry to match the
> new UID. That I hoped would work but ultimately just creates a new account.
>

> I have seen some rumblings from posts about updating *All-Users.git*. I

> am currently looking around seeing if I can figure that out. Any
> direction would be greatly appreciated.

You're likely going to have to muck around inside All-Users to get this
fixed. To my knowledge there are currently no plugins or tools for
merging or renaming accounts that already exist.

So... the primer on user accounts in NoteDB

Your external authentication linkages all happen in the
meta/external-ids branch

Your records are indexed and stored via a shared sha1sum of the record
data type and key.

So, for LDAP you're going to have a few records. Record types that I
know you'll have:

gerrit:$uid
username:$uid


The gerrit:$uid is the primary account linkage record it will look
something like this:

[externalId "gerrit:foo"]
<hardtab>accountId = 12345
<hardtab>email = f...@example.com

The above would be stored in the following file:

df/e9e07aea260bf85aa31f08a392a2ae71f40766

That can be obtained by the following incantation:

echo -n 'gerrit:foo' | sha1sum | cut -f1 -d' ' | \
sed 's/^\(.\{2\}\)/\1\//'

The username record would look like this:

[externalId "username:foo"]
<hardtab>accountId = 12345

There might be an optional password field in there to depending on the
gerrit configuration. This one would be stored at:

4b/0d6805d13e212673b3b7ea7f9ecc464c4a9227

Which is obtained by the same incantation, just swapping out the
gerrit:foo for username:foo

So... if you're needing to change account identifiers on the back end
you basically have to move records around and update the information in
them.

Additionally the email field in the gerrit record must be unique in your
Gerrit system, if the email that is listed there _does not match_ what
your authentication backend is, it must be fixed. If you need to
maintain that email address after swapping it out then you will need to
add a mailto record which would look like:

[externalId "mailto:f...@example.com"]
<hardtab>accountId = 12345
<hardtab>email = f...@example.com

Living at 0a/401e098c5702d9ed9643f405ef3893632ad4b2

As for the account details those will be stored in files at are the
following location
users/<shard>/<accountId>

Where shard == the last too digits of the accountId (zero padded)

So in this case it would be users/45/12345 a user with accountId of 1
would be at users/01/1

For the case of just fixing up bad accounts you shouldn't have to deal
with those branches, but you _will_ want to consider doing some
consistency checks.

A more in depth email about all of this (including how to deal with
consistency issues) can be found in this [0] post I made back in 2019

-Andy-

[0] https://groups.google.com/g/repo-discuss/c/tZ1tYQwbeLY/m/xSZhIQ20EQAJ

[Nate Dreier]

Thanks Andy!

Really appreciate the big response. 

Testing the above out in my test environment. 

Thanks again!
Nate

[Nate Dreier]

At the end of the day:

track down the appropriate username:foo and gerrit:foo, edit the username and email (if needed), commit and push back to gerrit. In theory that will resolve the issue, yeah?

I was able to checkout git checkout meta/external-ids and find the appropriate entries in prod. My dev instance is missing them. Going to take some time this eve to clone prod and run through a few tests to verify they will work.

[Andrew Grimberg]

On 7/14/21 10:29 AM, Nate Dreier wrote:
> At the end of the day:
>

> track down the appropriate *username:foo *and *gerrit:foo*, edit the

> username and email (if needed), commit and push back to gerrit. In
> theory that will resolve the issue, yeah?

Pretty much. The good thing is that as long as you don't have any
consistency issue then Gerrit will validate that the change you're
making is good and only allow it in if it is.

If you have any consistency issues, you _must_ fix those first as one
giant correction before you'll be able to do any of this via standard
git mechanisms. Otherwise you're having to do "pick-axe" fixes directly
on disk with a gerrit restart after the changes.

> I was able to checkout *git checkout meta/external-ids* and find the

> appropriate entries in prod. My dev instance is missing them. Going to
> take some time this eve to clone prod and run through a few tests to
> verify they will work.

-Andy-

[Nate Dreier]

Awesome, thanks Andy.

Hopefully the last question. The below is the appropriate way to get to where I need to make the changes, yeah? In your first reply, you mentioned that the files are located in something like 4b/0d6805d13e212673b3b7ea7f9ecc464c4a9227, but when I switch to meta/external-ids all of the files (0d6805d13e212673b3b7ea7f9ecc464c4a9227) are located at the root of the repo.

git clone All-Users.git
git checkout meta/external-ids

[Andrew Grimberg]

On 7/14/21 11:09 AM, Nate Dreier wrote:
> Awesome, thanks Andy.
>
> Hopefully the last question. The below is the appropriate way to get to
> where I need to make the changes, yeah? In your first reply, you
> mentioned that the files are located in something like
> 4b/0d6805d13e212673b3b7ea7f9ecc464c4a9227, but when I switch to
> meta/external-ids all of the files
> (0d6805d13e212673b3b7ea7f9ecc464c4a9227) are located at the root of the
> repo.
>
> git clone All-Users.git
> git checkout meta/external-ids

Yes, it's just like working with the meta/config magic branch of Gerrit
repos.

If you aren't seeing sharded IDs that's because you don't have enough
objects for Gerrit to switch to a sharded setup. In that case the IDs
are the full sha1sum without the injected '/' after the first two
characters.

I have no idea what the tipping point number of objects is before it
switches to a sharded configuration!

-Andy-

> On Wednesday, July 14, 2021 at 11:05:26 AM UTC-7 grim...@gmail.com wrote:
>
> On 7/14/21 10:29 AM, Nate Dreier wrote:
> > At the end of the day:
> >
> > track down the appropriate *username:foo *and *gerrit:foo*, edit the
> > username and email (if needed), commit and push back to gerrit. In
> > theory that will resolve the issue, yeah?
>
> Pretty much. The good thing is that as long as you don't have any
> consistency issue then Gerrit will validate that the change you're
> making is good and only allow it in if it is.
>
> If you have any consistency issues, you _must_ fix those first as one
> giant correction before you'll be able to do any of this via standard
> git mechanisms. Otherwise you're having to do "pick-axe" fixes directly
> on disk with a gerrit restart after the changes.
>
> > I was able to checkout *git checkout meta/external-ids* and find the
> > appropriate entries in prod. My dev instance is missing them.
> Going to
> > take some time this eve to clone prod and run through a few tests to
> > verify they will work.
>
> -Andy-
>
>
>

> https://maystreet.com <https://maystreet.com>
>
> --
> --
> To unsubscribe, email repo-discuss...@googlegroups.com
> More info at http://groups.google.com/group/repo-discuss?hl=en
> <http://groups.google.com/group/repo-discuss?hl=en>
>
> ---
> You received this message because you are subscribed to the Google
> Groups "Repo and Gerrit Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to repo-discuss...@googlegroups.com
> <mailto:repo-discuss...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/repo-discuss/bd94d8b6-ae18-4960-b5cd-7fa71fc1ab26n%40googlegroups.com
> <https://groups.google.com/d/msgid/repo-discuss/bd94d8b6-ae18-4960-b5cd-7fa71fc1ab26n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Andrew Grimberg]

One other thing...

On 7/14/21 11:14 AM, Andrew Grimberg wrote:
> On 7/14/21 11:09 AM, Nate Dreier wrote:
>> Awesome, thanks Andy.
>>
>> Hopefully the last question. The below is the appropriate way to get to
>> where I need to make the changes, yeah? In your first reply, you
>> mentioned that the files are located in something like
>> 4b/0d6805d13e212673b3b7ea7f9ecc464c4a9227, but when I switch to
>> meta/external-ids all of the files
>> (0d6805d13e212673b3b7ea7f9ecc464c4a9227) are located at the root of the
>> repo.
>>
>> git clone All-Users.git
>> git checkout meta/external-ids
>
> Yes, it's just like working with the meta/config magic branch of Gerrit
> repos.
>
> If you aren't seeing sharded IDs that's because you don't have enough
> objects for Gerrit to switch to a sharded setup. In that case the IDs
> are the full sha1sum without the injected '/' after the first two
> characters.
>
> I have no idea what the tipping point number of objects is before it
> switches to a sharded configuration!

I'll add that the linked email in my initial writeup goes into a lot
more depth on this.

[Nate Dreier]

Ah perfect, thank you!

I gave the first email a once over, I will go through it again to have better context.

Thanks again Andy!

[Nate Dreier]

As an update:

Initial commands I ran:

git clone All-Users.git

git fetch origin refs/meta/external-ids:refs/meta/external-ids

git checkout meta/external-ids

At this point, I just grep'd for the username/email/external_ids that I needed to edit.

git add --all

git commit -m "message"

git push origin HEAD:refs/meta/external-ids

At this point, I logged into Gerrit to see the changes. I enabled development_become_any_account under auth type so that I can quickly navigate the users that were affected.

NOTE: An interesting behavior I came across, even though I adjusted the names in All-Users, they were not reflected under the Settings of the respective user. Instead when I "became" the user, their username box was empty, but editable. I added the corrected username in the UI and all seemed to be okay.

[Nate Dreier]

As a final update to this:

the SHA1 sums that were carried through the upgrade from 2.11.2 -> 3.4 were different than the SHA1 sums that were output with echo -n "gerrit:foo" | sha1sum | cut -f1 -d' '

When I was first walking through the fix, all I did was update the username in the gerrit:foo and username:foo. This only "fixed" it cosmetically. Interestingly enough, when the user logged back in it wound up creating a brand new account with the error: SHA1 of external ID 'gerrit:rnihilo' does not match note ID 's0m3l0ngsha12348912349078'. That is what reminded me that Andy already gave me the answer to this.

Removing the file that contained the old sha and adding a file with the sha that was outputted with the above command worked. 

There are two files (as Andy mentioned) associated to each user. One contains gerrit:foo and the other contains username:foo. The file that contained username:foo was correct and the SHA's matched. I did not need to update the file, however the gerrit:foo file name was incorrect and needed to be updated.