LDAP group search in 0.6.1

[Adam Curtis]

Hey guys, I'm having a few issues setting up LDAP in 0.6.1. I've not setup LDAP previous so don't know if this affected older versions too, but I know lots has changed in 0.6.1.

It seems to be failing on group searches. I have plenty of other applications setup and working correctly. I've tried various configuration options. Heres my config:

vault write auth/ldap/config \

    url="ldaps://ldap.test.com" \

    starttls=true \

    insecure_tls=false \

    binddn="uid=vault,ou=applications,dc=test,dc=com" \

    bindpass="**SNIP**" \

    userdn="ou=users,dc=test,dc=com" \

    userattr="uid" \

    groupdn="ou=groups,dc=test,dc=com" \

    groupattr="cn" \

    groupfilter="(&(objectClass=posixGroup)(&(cn=*)(memberUid={{.Username}})))"

The group filter is probably more complicated that it needs to be, I just wanted to replicate what other apps that authenticate against OpenLDAP. Here are the OpenLDAP logs when I try and login using vault:

57bc0ab5 conn=319658 op=3 SRCH base="ou=groups,dc=test,dc=com" scope=2 deref=0 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))"

57bc0ab5 conn=319658 op=3 SRCH attr=cn

57bc0ab5 conn=319658 op=3 SEARCH RESULT tag=101 err=32 nentries=0 text=

As you can nentries=0, so zero results are returned. However when Nexus OSS finds the groups it works:

57bc0c91 conn=319751 op=2 SRCH base="ou=groups,dc=test,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))"

57bc0c91 conn=319751 op=2 SRCH attr=cn

57bc0c91 conn=319751 op=2 SEARCH RESULT tag=101 err=0 nentries=2 text=

The only difference being being the deref value, which after further research deref=3 means always and 0 means never. I've noticed that all the other applications I use to auth against LDAP have deref=3, so I guess thats why its not working. TBH I'm a massive LDAP noob and don't really know what dereferencing is. Is there anything I can change in vault or reconfigure in OpenLDAP in order for this to work?

[Jeff Mitchell]

Hi Adam,

I actually noticed two differences: the op number and the deref. I
don't know much about OpenLDAP, so I'm mostly divulging what I found
via some google searches, and this may not be all that helpful. I've
also CC'd Oren who did much of the fantastic improvement work in
0.6.1; I don't think this is related to that work, but he may have
some ideas.

The op number is sequentially set for each operation for a connection.
For Vault there are four ops (they start at zero) and for Nexus three
ops. It might be instructive to look at what the previous ops are for
each and see if one is performing some action the other isn't.

As for deref, it looks like a configuration option we could add. If
you can built Vault yourself for testing I could tell you what to
change to add it and you could see if that fixes your problem. (Or I
can built a binary and send it if needed.)

Best,
Jeff

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/vault/issues
> IRC: #vault-tool on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Vault" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vault-tool+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/vault-tool/9b0422db-309c-415b-ab3a-0162135cbf7a%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Adam Curtis]

Hi Jeff,

Thanks for the quick reply :)

I did look at the other operations and they seem to return the correct stuff, including vault all the way up until the group searches.

I could probably build a version if required, although I'm not sure what to change :D

Thanks

Adam

[Jeff Mitchell]

Hi Adam,

What OS/arch (e.g. linux/amd64, windows/i386) do you need? I can send
a dropbox link with a version that adds in dereferencing for testing.

Best,
Jeff

On Tue, Aug 23, 2016 at 10:04 AM, 'Adam Curtis' via Vault

> https://groups.google.com/d/msgid/vault-tool/80ff8592-b62a-4988-85b5-0ec7d2bf36d0%40googlegroups.com.

[Adam Curtis]

Hi Jeff,

linux/amd64 please :)

Adam

[Jeff Mitchell]

Hi Adam,

https://www.dropbox.com/s/4ee8sg5p1tvsu9g/vault-deref-ldap.gz?dl=0

Let me know how it goes!

Best,
Jeff

On Tue, Aug 23, 2016 at 11:01 AM, 'Adam Curtis' via Vault

> https://groups.google.com/d/msgid/vault-tool/77804903-667d-4994-bc48-dde0a594164c%40googlegroups.com.

[Jeff Mitchell]

Hi Adam,

Any luck with that build?

Best,
Jeff

[Adam Curtis]

Hi Jeff,

Unfortunately not! The deref worked but it still didn't return any results which is rather odd, I guess its back to the drawing board!

Heres the full connection from vault to openldap:

57bea018 conn=351332 fd=139 ACCEPT from IP=000.000.000.000:51413 (IP=0.0.0.0:636)

57bea018 conn=351332 fd=139 TLS established tls_ssf=128 ssf=128

57bea019 conn=351332 op=0 BIND dn="uid=vault,ou=applications,dc=redacted,dc=com" method=128

57bea019 conn=351332 op=0 BIND dn="uid=vault,ou=applications,dc=redacted,dc=com" mech=SIMPLE ssf=0

57bea019 conn=351332 op=0 RESULT tag=97 err=0 text=

57bea019 conn=351332 op=1 SRCH base="ou=users,dc=redacted,dc=com" scope=2 deref=3 filter="(uid=acurtis)"

57bea019 conn=351332 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text=

57bea019 conn=351332 op=2 BIND anonymous mech=implicit ssf=0

57bea019 conn=351332 op=2 BIND dn="cn=acurtis,ou=users,dc=redacted,dc=com" method=128

57bea019 conn=351332 op=2 BIND dn="cn=acurtis,ou=users,dc=redacted,dc=com" mech=SIMPLE ssf=0

57bea019 conn=351332 op=2 RESULT tag=97 err=0 text=

57bea019 conn=351332 op=3 SRCH base="ou=groups,dc=redacted,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))"

57bea019 conn=351332 op=3 SRCH attr=cn

57bea019 conn=351332 op=3 SEARCH RESULT tag=101 err=32 nentries=0 text=

Also heres the full output from Nexus OSS2 when I login:

57bea119 conn=351384 fd=142 ACCEPT from IP=000.000.000.000:57883 (IP=0.0.0.0:636)

57bea119 conn=351384 fd=142 TLS established tls_ssf=128 ssf=128

57bea119 conn=351384 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" method=128

57bea119 conn=351384 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" mech=SIMPLE ssf=0

57bea119 conn=351384 op=0 RESULT tag=97 err=0 text=

57bea119 conn=351384 op=1 SRCH base="ou=users,dc=redacted,dc=com" scope=1 deref=3 filter="(&(objectClass=inetOrgPerson)(uid=acurtis))"

57bea119 conn=351384 op=1 SRCH attr=uid displayName mail labeledUri

57bea119 conn=351384 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text=

57bea119 conn=351384 op=2 SRCH base="ou=groups,dc=redacted,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))"

57bea119 conn=351384 op=2 SRCH attr=cn

57bea119 <= bdb_equality_candidates: (memberUid) not indexed

57bea119 conn=351384 op=2 SEARCH RESULT tag=101 err=0 nentries=2 text=

57bea119 conn=351384 op=3 UNBIND

57bea119 conn=351384 fd=142 closed

57bea119 conn=351385 fd=142 ACCEPT from IP=000.000.000.000:57900 (IP=0.0.0.0:636)

57bea119 conn=351385 fd=142 TLS established tls_ssf=128 ssf=128

57bea119 conn=351385 op=0 BIND dn="cn=acurtis,ou=users,dc=redacted,dc=com" method=128

57bea119 conn=351385 op=0 BIND dn="cn=acurtis,ou=users,dc=redacted,dc=com" mech=SIMPLE ssf=0

57bea119 conn=351385 op=0 RESULT tag=97 err=0 text=

57bea119 conn=351385 op=1 UNBIND

57bea119 conn=351385 fd=142 closed

57bea119 conn=351386 fd=142 ACCEPT from IP=000.000.000.000:57901 (IP=0.0.0.0:636)

57bea119 conn=351386 fd=142 TLS established tls_ssf=128 ssf=128

57bea119 conn=351386 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" method=128

57bea119 conn=351386 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" mech=SIMPLE ssf=0

57bea119 conn=351386 op=0 RESULT tag=97 err=0 text=

57bea119 conn=351386 op=1 SRCH base="ou=groups,dc=redacted,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))"

57bea119 conn=351386 op=1 SRCH attr=cn

57bea119 <= bdb_equality_candidates: (memberUid) not indexed

57bea119 conn=351386 op=1 SEARCH RESULT tag=101 err=0 nentries=2 text=

57bea119 conn=351386 op=2 UNBIND

57bea119 conn=351386 fd=142 closed

57bea119 conn=351387 fd=142 ACCEPT from IP=000.000.000.000:57906 (IP=0.0.0.0:636)

57bea119 conn=351387 fd=142 TLS established tls_ssf=128 ssf=128

57bea119 conn=351387 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" method=128

57bea119 conn=351387 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" mech=SIMPLE ssf=0

57bea119 conn=351387 op=0 RESULT tag=97 err=0 text=

57bea119 conn=351387 op=1 SRCH base="ou=users,dc=redacted,dc=com" scope=1 deref=3 filter="(&(objectClass=inetOrgPerson)(uid=acurtis))"

57bea119 conn=351387 op=1 SRCH attr=uid displayName mail labeledUri

57bea119 conn=351387 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text=

57bea119 conn=351387 op=2 SRCH base="ou=groups,dc=redacted,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))"

57bea119 conn=351387 op=2 SRCH attr=cn

57bea119 <= bdb_equality_candidates: (memberUid) not indexed

57bea119 conn=351387 op=2 SEARCH RESULT tag=101 err=0 nentries=2 text=

57bea119 conn=351387 op=3 UNBIND

57bea119 conn=351387 fd=142 closed

57bea119 conn=351388 fd=142 ACCEPT from IP=000.000.000.000:57914 (IP=0.0.0.0:636)

57bea119 conn=351388 fd=142 TLS established tls_ssf=128 ssf=128

57bea119 conn=351388 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" method=128

57bea119 conn=351388 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" mech=SIMPLE ssf=0

57bea119 conn=351388 op=0 RESULT tag=97 err=0 text=

57bea11a conn=351388 op=1 SRCH base="ou=users,dc=redacted,dc=com" scope=1 deref=3 filter="(&(objectClass=inetOrgPerson)(uid=acurtis))"

57bea11a conn=351388 op=1 SRCH attr=uid displayName mail labeledUri

57bea11a conn=351388 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text=

57bea11a conn=351388 op=2 SRCH base="ou=groups,dc=redacted,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))"

57bea11a conn=351388 op=2 SRCH attr=cn

57bea11a <= bdb_equality_candidates: (memberUid) not indexed

57bea11a conn=351388 op=2 SEARCH RESULT tag=101 err=0 nentries=2 text=

57bea11a conn=351388 op=3 UNBIND

57bea11a conn=351388 fd=142 closed

57bea11a conn=351389 fd=142 ACCEPT from IP=000.000.000.000:57965 (IP=0.0.0.0:636)

57bea11a conn=351389 fd=142 TLS established tls_ssf=128 ssf=128

57bea11a conn=351389 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" method=128

57bea11a conn=351389 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" mech=SIMPLE ssf=0

57bea11a conn=351389 op=0 RESULT tag=97 err=0 text=

57bea11a conn=351389 op=1 SRCH base="ou=users,dc=redacted,dc=com" scope=1 deref=3 filter="(&(objectClass=inetOrgPerson)(uid=acurtis))"

57bea11a conn=351389 op=1 SRCH attr=uid displayName mail labeledUri

57bea11a conn=351389 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text=

57bea11a conn=351389 op=2 SRCH base="ou=groups,dc=redacted,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))"

57bea11a conn=351389 op=2 SRCH attr=cn

57bea11a <= bdb_equality_candidates: (memberUid) not indexed

57bea11a conn=351389 op=2 SEARCH RESULT tag=101 err=0 nentries=2 text=

57bea11a conn=351389 op=3 UNBIND

57bea11a conn=351389 fd=142 closed

Theres certainly lots more going on in that connection, noticeably multiple connections, although I'm not quite sure why as they look similar.

This may help, this is what happens when I check the group mapping from the LDAP settings in Nexus:

57bea2f8 conn=351481 fd=142 ACCEPT from IP=000.000.000.000:41453 (IP=0.0.0.0:636)

57bea2f8 conn=351481 fd=142 TLS established tls_ssf=128 ssf=128

57bea2f8 conn=351481 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" method=128

57bea2f8 conn=351481 op=0 BIND dn="uid=nexus2,ou=applications,dc=redacted,dc=com" mech=SIMPLE ssf=0

57bea2f8 conn=351481 op=0 RESULT tag=97 err=0 text=

57bea2f8 conn=351481 op=1 SRCH base="ou=users,dc=redacted,dc=com" scope=1 deref=3 filter="(&(objectClass=inetOrgPerson)(uid=*))"

57bea2f8 conn=351481 op=1 SRCH attr=uid displayName mail labeledUri

57bea2f8 conn=351481 op=1 SEARCH RESULT tag=101 err=0 nentries=3 text=

57bea2f8 conn=351481 op=2 SRCH base="ou=groups,dc=redacted,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))"

57bea2f8 conn=351481 op=2 SRCH attr=cn

57bea2f8 <= bdb_equality_candidates: (memberUid) not indexed

57bea2f8 conn=351481 op=2 SEARCH RESULT tag=101 err=0 nentries=2 text=

57bea2f8 conn=351481 op=3 SRCH base="ou=groups,dc=redacted,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=jjackson)))"

57bea2f8 conn=351481 op=3 SRCH attr=cn

57bea2f8 <= bdb_equality_candidates: (memberUid) not indexed

57bea2f8 conn=351481 op=3 SEARCH RESULT tag=101 err=0 nentries=2 text=

57bea2f8 conn=351481 op=4 SRCH base="ou=groups,dc=redacted,dc=com" scope=2 deref=3 filter="(&(objectClass=posixGroup)(&(cn=*)(memberUid=mfenn)))"

57bea2f8 conn=351481 op=4 SRCH attr=cn

57bea2f8 <= bdb_equality_candidates: (memberUid) not indexed

57bea2f8 conn=351481 op=4 SEARCH RESULT tag=101 err=0 nentries=2 text=

57bea2f8 conn=351481 op=5 UNBIND

57bea2f8 conn=351481 fd=142 closed

Not sure if any of that helps. Unfortunately I know very little about LDAP, I pretty much figured out the bare basics to get something up and running in our org.

If theres anything you need me to test let me know :)

Adam

[Oren Shomron]

Hi there,

I have a theory - I believe this is a combination of a permissions issue and how Vault is performing authentication.

There is a substantive difference between the Nexus logs and Vault - Nexus binds back to the service user before

performing its group search, while Vault performs the group search as the authenticated user (acurtis) instead.

I am guessing that acurtis does not have sufficient permissions to run your query.

You can test my theory by running the group search using ldap_search with both users and comparing the results.

This will look something like this:

ldapsearch -x -H ldaps://ldap.test.com -b ou=groups,dc=redacted,dc=com -s sub -D cn=acurtis,ou=users,dc=redacted,dc=com -W -E pr=1000/noprompt -o ldif-wrap=no '(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))'

vs.

ldapsearch -x -H ldaps://ldap.test.com -b ou=groups,dc=redacted,dc=com -s sub -D uid=vault,ou=applications,dc=redacted,dc=com -W -E pr=1000/noprompt -o ldif-wrap=no '(&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))'

If this pans out, I believe the solution will be to have Vault rebind as the service user (if provided) after user authentication.

Let me know what you come back with!

  -  Oren

[Adam Curtis]

Your suspicions are correct!

The first query:

# extended LDIF

#

# LDAPv3

# base <ou=groups,dc=redacted,dc=com> with scope subtree

# filter: (&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))

# requesting: ALL

# with pagedResults control: size=1000

#

# search result

search: 2

result: 32 No such object

# numResponses: 1

And the second:

# extended LDIF

#

# LDAPv3

# base <ou=groups,dc=redacted,dc=com> with scope subtree

# filter: (&(objectClass=posixGroup)(&(cn=*)(memberUid=acurtis)))

# requesting: ALL

# with pagedResults control: size=1000

#

# admin, groups, mooloop.com

dn: cn=admin,ou=groups,dc=redacted,dc=com

cn: admin

gidNumber: 500

objectClass: posixGroup

objectClass: top

memberUid: acurtis

memberUid: mfenn

memberUid: jjackson

# users, groups, mooloop.com

dn: cn=users,ou=groups,dc=redacted,dc=com

cn: users

gidNumber: 501

objectClass: posixGroup

objectClass: top

memberUid: acurtis

memberUid: mfenn

memberUid: jjackson

# search result

search: 2

result: 0 Success

control: 1.2.840.113556.1.4.319 false MAUCAQAEAA==

pagedresults: cookie=

# numResponses: 3

# numEntries: 2

Rebinding vault as the service user sounds like a sensible way to go and it seems like most of the applications I've hooked up using ldap work in this way :)

Adam

[Oren Shomron]

Happy dance!

I can submit a PR over the weekend, if Jeff doesn't beat me to it.

  - Oren

[Jeff Mitchell]

Hi Oren,

If you're working on such a PR, might be useful to make the deref
value user-controllable at the same time, for advanced setups/needs.
Or is that basically just never needed?

Best,
Jeff

> https://groups.google.com/d/msgid/vault-tool/0031024f-acc8-4560-b8e8-f88c51183746%40googlegroups.com.