gerrit ldap authentication, specifically group membership

[Rob]

I'm pulling my hair out here. I've got an existing gerrit install tied to an openldap server, which totally works (ldap user auth, and group memberships from ldap). The users get 2-3 groups like 'registered' etc, then 2-3 like ldap/group-name.

I've moved our ldap setup to a new freeipa based server, which has a slightly diffrent layout (cn=accounts in stead of ou=people) etc. So i thought if i just alter the gerrit config it'd work. Nope !

So here's the existing config (gerrit and openldap are on the same machine currently):

[ldap]

server = ldap://127.0.0.1/

accountBase = ou=people,dc=<domain>,dc=com

accountPattern = (&(objectClass=gosaAccount)(uid=${username}))

groupBase = ou=groups,dc=<domain>,dc=com

groupPattern = (&(objectClass=posixGroup)(cn=${groupname}))

groupMemberPattern = (memberUid=${username})

sslVerify = false

accountFullName = cn

accountEmailAddress = mail

accountSshUserName = uid

And here's the altered config:

[ldap]

server = ldap://<new-ldap-ip>/

accountBase = cn=users,cn=accounts,dc=<domain>,dc=com

accountPattern = (&(objectClass=inetorgperson)(uid=${username}))

groupBase = cn=groups,cn=accounts,dc=<domain>,dc=com

groupPattern = (&(objectClass=groupofnames)(cn=${groupname}))

groupMemberPattern =  (memberUid=${username})

sslVerify = false

accountFullName = cn

accountEmailAddress = mail

accountSshUserName = uid

Which does not work - users can log in fine, but group membership is not respected.

So next up i decided to set up a test install of gerrit on my own machine, took the top ldap config (the existing, working one) and put it on my testbed and just changed the 127.0.0.1 to be the current ldap ip. Strangely i get the same behavior - users can login just fine, but group memberships are not known.

Heres what a group looks like on the current openldap ldap:

dn: cn=gerrit-commit,ou=groups,dc=<domain>,dc=com

cn: gerrit-commit

description: Commit patches to gerrit

gidNumber: 1005

objectClass: top

objectClass: posixGroup

memberUid: fubar1

memberUid: fubar2

Here's what a group looks like on the new ipa based ldap

dn: cn=gerrit-commit,cn=groups,cn=accounts,dc=<domain>,dc=com

cn: gerrit-commit

objectClass: ipaobject

objectClass: top

objectClass: ipausergroup

objectClass: posixgroup

objectClass: groupofnames

objectClass: nestedgroup

memberUid: fubar1

memberUid: fubar2

gidNumber: 1005

description: Commit patches to gerrit

ipaUniqueID: <uuid>

member: uid=fubar1,cn=users,cn=accounts,dc=<domain>,dc=com

member: uid=fubar2,cn=users,cn=accounts,dc=<domain>,dc=com

[Rob]

if anyone has any ideas on this i'd love to know. It's freaking annoying and i've no idea why it isn't working as expected. It's single handedly holding up the entire companies migration to the new auth server so it's really important to get it fixed but i've no idea why it isn't working.

[Sven Selberg]

A stab in the dark

from the non functioning gerrit.config

groupBase = cn=groups,cn=accounts,dc=<domain>,dc=com

It sounds strange that groups would be under accounts, have you tried with:

groupBase = cn=groups,dc=<domain>,dc=com

or simply:

groupBase = dc=<domain>,dc=com

/Sven

[Rob]

so i've done a fair bit of testing on this, and i've found a few things out.

- gerrit does not show user's being in a remote ldap user groups until those groups are mentioned in an access policy on a project (this is horrible UX design)

Following on from that (which incidentally lost me about a day) the behavior i'm seeing is this (after adding user groups to a access list):

- login as ldap user, login is fine. 

- go to user's groups page, no ldap groups listed.

- go to project, go to access list, make any change at all, then save

- go back to the user group information, and it now shows those groups that are in access lists.

- logout, log back in, user is listed in no groups again.

What the heck ? That has to be a bug, surely !? It must be doing ....something... at the time of saving the access policy on the project.

There's only one other thing i (now) think might be at play. If you do an anonymous bind to freeipa it'll show you the group names, but not membership information. However because i'm giving both ldap username and password below it *should* be doing a bind to ldap, right ?

here's my current config....

[auth]

type = ldap

[ldap]

        refferal = follow

        username = uid=ldap,cn=users,cn=accounts,dc=(domain),dc=com

        password = (password)

server = ldap://(ldap-ip)/

accountBase = cn=users,cn=accounts,dc=(domain),dc=com

accountPattern = (&(objectClass=inetorgperson)(uid=${username}))

groupBase = cn=groups,cn=accounts,dc=(domain),dc=com

groupPattern = (cn=${groupname})

groupMemberPattern = (memberUid=${username})

        fetchMemberOfEagerly = true

sslVerify = false

accountFullName = cn

accountEmailAddress = mail

accountSshUserName = uid

Regarding your reply, no, that's the correct ldap schema layout - both users and groups are under accounts.

[Rob]

think i can rule out the anonymous bind theory. I looked at the ldap server logs and it's doing it as the user in the config file first, then the user logged in as second (to test password, i assume).

On Wednesday, 11 May 2016 18:36:28 UTC+12, Rob wrote:

[Björn Pedersen]

Hi,

I guess what you observe is actually by design. ldap groups are lazy-loaded on demand and cached (https://gerrit-review.googlesource.com/#/c/66059/). If you really need to pre-populate the cache see e.g. https://gerrit-review.googlesource.com/#/c/75424, https://gerrit-review.googlesource.com/#/c/75636/

Björn

[Luca Milanesio]

I came up with the warm-cache script as loading groups from LDAP is the *most risky* operation for your Gerrit server, especially for large organisations.

LDAP latency for groups resolution could be *very significant*.

Luca.

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[siology.io]

well, i don't really understand why that would be a desirable design ? I start the gerrit server, and noone has any permissions to push code to gerrit as the groups aren't loaded in.

I'll try the warm cache thing tomorrow when i'm in the office. I also don't understand why this wasn't an issue using openldap but is now occuring after we moved to using freeipa ldap ?

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---

You received this message because you are subscribed to a topic in the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/repo-discuss/CsbRX6Xr6g8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to repo-discuss...@googlegroups.com.

[Luca Milanesio]

On 19 May 2016, at 10:38, siology.io <siolo...@gmail.com> wrote:

well, i don't really understand why that would be a desirable design ?

 It is a desirable design as you want to avoid to generate massive load to scan your entire LDAP structure (which could be potentially *huge*) at start-up.

I start the gerrit server, and noone has any permissions to push code to gerrit as the groups aren't loaded in.

LDAP groups are loaded "on-demand": as soon as someone pushes code, their associated LDAP group memberships will  be loaded.

I'll try the warm cache thing tomorrow when i'm in the office. I also don't understand why this wasn't an issue using openldap but is now occuring after we moved to using freeipa ldap ?

It isn't an LDAP behaviour but the implementation of the Gerrit Group Backend.

You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

[Luca Milanesio]

On 19 May 2016, at 10:52, siology.io <siolo...@gmail.com> wrote:

ah, this can't be the same effect then i guess. When i originally tried the new ldap configuration (which essentially boiled down to chaning the ldap server ip and altering the base dn as needed, then restarting) i was told by my users that they could no longer push code.

Group resolution was then broken I guess. It is not a lazy loading problem then :-(

I have noticed that sometimes LDAP group configuration can be done in different ways and they may result in different behaviours in terms of results and execution times.

I would recommend you to make some investigation on the two LDAP queries generated using ldapsearch and see the associated results and execution times.

So i guess it didn't lazy load the groups (and i did try the eager mode = True as well).

The eager mode (https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#ldap.fetchMemberOfEagerly) is when the group is fetched using the memberOf attribute of the account LDAP record which I think isn't your case.

That's why did not have any effect.

From your LDAP output I see that you have the groups to user relationship (group listing all their members) rather than the user having the list of groups he is member of.

This could be as well the difference you experienced when migrating from OpenLDAP to FreeIPA.

Luca.

I reverted that change and since then have been trying to work out what's actually happening under the hood.

[Luca Milanesio]

Why don't you try to make some experiments with some basic LDAP search?

Gerrit all it does it to trigger searches based and use the results.

If no groups were resolved it means that no groups were returned by your search.

HTH

Luca.

On 19 May 2016, at 11:17, siology.io <siolo...@gmail.com> wrote:

freeipa actually has bi directional relationships, so the users have memberOf attributes, and the groups have both memberUid and member attributes for the user id's and dn's respectively. I figured therefore that no matter what it was doing on the original ldap setup it should have worked on the new one also.  

[Rob]

well, i did that but it didn't get me very far. The ldap server logs look like this;

[21/Jun/2016:21:31:20 +0000] conn=107125 op=1 SRCH base="cn=users,cn=accounts,dc=<domain>,dc=com" scope=2 filter="(&(objectClass=inetorgperson)(uid=user1))" attrs="uid mail username cn"

[21/Jun/2016:21:31:20 +0000] conn=107126 op=0 BIND dn="uid=user1,cn=users,cn=accounts,dc=<domain>,dc=com" method=128 version=3

[21/Jun/2016:21:31:20 +0000] conn=107126 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=user1,cn=users,cn=accounts,dc=<domain>,dc=com"

[21/Jun/2016:21:31:20 +0000] conn=107125 op=2 SRCH base="cn=groups,cn=accounts,dc=<domain>,dc=com" scope=2 filter="(memberUid=user1)" attrs="1.1"

Which when i run with ldapsearch gives me this:

ldapsearch -H ldap://srv1.<domain>.com:389 -D "uid=user1,cn=users,cn=accounts,dc=<domain>,dc=com" -b 'cn=groups,cn=accounts,dc=<domain>,dc=com' -LLL "(memberUid=user1)" 1.1

dn: cn=gerrit,cn=groups,cn=accounts,dc=<domain>,dc=com

dn: cn=wiki,cn=groups,cn=accounts,dc=<domain>,dc=com

dn: cn=gerrit-commit,cn=groups,cn=accounts,dc=<domain>,dc=com

dn: cn=wikiprivate,cn=groups,cn=accounts,dc=<domain>,dc=com

dn: cn=publictracker,cn=groups,cn=accounts,dc=<domain>,dc=com

gerrit-commit is the interesting one here, because that group is used in acl's on various gerrit projects, and my gerrit user is an admin. However when i log into gerrit i don't see ldap/gerrit-commit listed as a group. And that's about the point i got stuck with debugging it.

-- 
-- 
To unsubscribe, email repo-discuss+unsub...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to a topic in the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/repo-discuss/CsbRX6Xr6g8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
-- 
To unsubscribe, email repo-discuss+unsub...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

--- 

[Rob]

i dont know if it's important, but running the same query against the old ldap server i get results like this;

cn=gerrit-commit,ou=groups,dc=<domain>,dc=com

which differ slightly from the new server's;

dn: cn=gerrit-commit,cn=groups,cn=accounts,dc=<domain>,dc=com

but i dont know how if that's significant to gerrit ? Might be a red herring.

[Rob]

posting this here in case anyone else finds this via google. we finally got to the bottom of this.

what we ended up doing was going into the all-projects ACL, removing all the mentions of the 'ldap/' groups, and then readded them. No idea what gerrit does internally but it must have tracked the groups from old and new server separately. We also needed to have bind credentials in the config as unlike openldap freeipa wont allow anonymous binds to show group membership information.

as soon as we did that it all sprung back to life and is working again !

[Rob]

gah. a footnote, this workaround does work but it kinda sucks because you have to do it for every new user to the system. With each new user gerrit wont add that user to the ldap groups, but will let them log in. This results in the user not being able to clone the repos until all the ACLs are rebuilt.

[Dawid Rabiega]

Hi Rob,

See this fix for FreeIPA backend: https://gerrit-review.googlesource.com/#/c/94925/. It should help you.

Enjoy!

DR