salt-api ldap group external_auth not working

251 views
Skip to first unread message

Dave Macias

unread,
Oct 18, 2022, 10:51:46 AM10/18/22
to Salt-users
Hello,

First-time poster here.

We have the salt master working fine with ldap authentication. I know this because `salt -T  -a ldap \* bla` works fine.

Now we want to add the api functionality.
Looking over the documentation this is what we have for the rest_cherrypy config:

rest_cherrypy:
  host: '10.x.x.x'
  port: 8000
  ssl_crt: /etc/pki/tls/certs/server.crt
  ssl_key: /etc/pki/tls/certs/server.key


Here is our auth and acl configs:

external_auth:
  ldap:
    'salt%':
      - '.*'
      - '\@jobs'
      - '\@wheel'
      - '\@runners'

publisher_acl:
  'salt%':
    - '.*'


My understanding is the `publisher_acl` uses the linux groups; while `external_auth` is used for api access.

This is how I call the API:

curl -k -sSi https://master.domain.net:8000/login -H 'Accept: application/x-yaml' -d username=blabla -d password='bla' -d eauth=ldap

Here are the debug logs from salt-master:

[DEBUG   ] LazyLoaded ldap.auth
[DEBUG   ] Attempting LDAP bind with user dn: uid=blabla,ou=people,dc=domain,dc=net
[ERROR   ] Failed to authenticate user dn via LDAP: {'binddn': 'uid=blabla,ou=people,dc=domain,dc=net', 'uri': '', 'server': 'ldap-server.domain.net', 'port': 389, 'starttls': False, 'tls': False, 'no_verify': True, 'anonymous': False, 'accountattributename': 'uid', 'activedirectory': False}
[DEBUG   ] Error authenticating user dn via LDAP:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/salt/auth/ldap.py", line 138, in __init__
    self.ldap.simple_bind_s(self.binddn, self.bindpw)
  File "/usr/local/lib/python3.9/site-packages/ldap/ldapobject.py", line 249, in simple_bind_s
    resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all=1,timeout=self.timeout)
  File "/usr/local/lib/python3.9/site-packages/ldap/ldapobject.py", line 543, in result3
    resp_type, resp_data, resp_msgid, decoded_resp_ctrls, retoid, retval = self.result4(
  File "/usr/local/lib/python3.9/site-packages/ldap/ldapobject.py", line 553, in result4
    ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
  File "/usr/local/lib/python3.9/site-packages/ldap/ldapobject.py", line 128, in _ldap_call
    result = func(*args,**kwargs)
ldap.INVALID_CREDENTIALS: {'msgtype': 97, 'msgid': 1, 'result': 49, 'desc': 'Invalid credentials', 'ctrls': []}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/salt/auth/ldap.py", line 324, in _bind
    ldap_conn = _LDAPConnection(**connargs).ldap
  File "/usr/local/lib/python3.9/site-packages/salt/auth/ldap.py", line 140, in __init__
    raise CommandExecutionError(
salt.exceptions.CommandExecutionError: Failed to bind to LDAP server ldap://ldap-server.domain.net:389 as uid=blabla,ou=people,dc=domain,dc=net: {'msgtype': 97, 'msgid': 1, 'result': 49, 'desc': 'Invalid credentials', 'ctrls': []}
[ERROR   ] LDAP _bind authentication FAILED
[WARNING ] Authentication failure of type "eauth" occurred.
[WARNING ] Authentication failure of type "eauth" occurred.


The salt-api debug is not that exciting:

[INFO    ] [api_acl] Authentication not checked for user blabla from IP 10.x.x.x
[DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://10.x.x.x:4506
[DEBUG   ] Trying to connect to: tcp://10.x.x.x:4506
[DEBUG   ] Closing AsyncZeroMQReqChannel instance
[INFO    ] 10.x.x.x - - [13/Oct/2022:20:05:35] "POST /login HTTP/1.1" 401 761 "" "curl/7.61.1"


Looking at the logs on the ldap server for some reason it is returning err=49 which means bad authentication, but that should not be the case. So im guessing i may have some misconfiguration or bug? 

63487127.3693df1e 0x7f844390f700 conn=1027 fd=12 ACCEPT from IP=[x:x:x::13]:60762 (IP=[::]:389)
63487127.369c5eeb 0x7f844390f700 conn=1027 op=0 BIND dn="uid=blabla,ou=people,dc=datacom,dc=net" method=128
63487127.36aaba96 0x7f844390f700 conn=1027 op=0 RESULT tag=97 err=49 qtime=0.000029 etime=0.001255 text=
63487127.36d00303 0x7f844390f700 conn=1027 op=1 UNBIND
63487127.36d23078 0x7f844390f700 conn=1027 fd=12 closed


Interestingly, I don't see the `mech` option when salt attempts to bind to the ldap server, like any other normal bind. (below is a successful bind when running `salt -a ldap blabla`)

Oct 13 20:30:12 server-lab slapd[950164]: conn=1004 op=0 BIND dn="uid=blabla,ou=People,dc=domain,dc=net" mech=SIMPLE bind_ssf=0 ssf=0
Oct 13 20:30:12 server-lab slapd[950164]: conn=1004 op=0 RESULT tag=97 err=0 qtime=0.000024 etime=0.026285 text=

Not sure what i am missing. but the version report showing `cherrypy: unknown` is interesting... not sure if relevant. It is `cherrypy==18.8.0`

Salt Version:
          Salt: 3004.2
 
Dependency Versions:
          cffi: 1.15.1
      cherrypy: unknown
      dateutil: Not Installed
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.4
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.15.0
        pygit2: Not Installed
        Python: 3.9.14 (main, Oct  7 2022, 02:39:02)
  python-gnupg: Not Installed
        PyYAML: 6.0
         PyZMQ: 21.0.2
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
Salt Extensions:
   salt-nornir: 0.9.0
 
System Versions:
          dist: alpine 3.16.2
        locale: utf-8
       machine: x86_64
       release: 4.18.0-348.20.1.el8_5.x86_64
        system: Linux
       version: Alpine Linux 3.16.2


What am I missing?
Any input is much appreciated!!

Best,
Dave

Simon Lundström

unread,
Oct 19, 2022, 2:42:30 AM10/19/22
to salt-...@googlegroups.com
Hey Dave!

I haven't used LDAP with Salt but I have used LDAP.

What is the LDAP-server? AD? OpenLDAP?

Can you try to do auth outside of salt via e.g. ldapwhoami(1) and see if
that works?

ldapwhoami -D uid=blabla,ou=people,dc=domain,dc=net -x -w bla

BR,
- Simon
> --
> You received this message because you are subscribed to the Google Groups "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com<mailto:salt-users+...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/ddcb8c96-9e67-4ab2-8ea2-5bf636b53349n%40googlegroups.com<https://groups.google.com/d/msgid/salt-users/ddcb8c96-9e67-4ab2-8ea2-5bf636b53349n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Dave Macias

unread,
Oct 19, 2022, 8:41:07 AM10/19/22
to salt-...@googlegroups.com
Thank you for the reply Simon.

I haven't used LDAP with Salt but I have used LDAP.

What is the LDAP-server? AD? OpenLDAP?


openldap 2.6.1
 
Can you try to do auth outside of salt via e.g. ldapwhoami(1) and see if
that works?

ldapwhoami -D uid=blabla,ou=people,dc=domain,dc=net -x -w bla


This is the results: (which works)

-> ldapwhoami -D uid=blabla,ou=people,dc=domain,dc=net -x -w 'bla'
dn:uid=blabla,ou=People,dc=domain,dc=net

the above bind produced these logs: (notice mech=SIMPLE bind_ssf=0 ssf=0 is there)

Oct 19 12:25:58 ldap-server slapd[47122]: conn=1045953 fd=66 ACCEPT from IP=[::1]:59110 (IP=[::]:389)
Oct 19 12:25:58 ldap-server slapd[47122]: conn=1045953 op=0 BIND dn="uid=blabla,ou=people,dc=domain,dc=net" method=128
Oct 19 12:25:58 ldap-server slapd[47122]: conn=1045953 op=0 BIND dn="uid=blabla,ou=People,dc=domain,dc=net" mech=SIMPLE bind_ssf=0 ssf=0
Oct 19 12:25:58 ldap-server slapd[47122]: conn=1045953 op=0 RESULT tag=97 err=0 qtime=0.000025 etime=0.027784 text=
Oct 19 12:25:58 ldap-server slapd[47122]: conn=1045953 op=1 EXT oid=1.3.6.1.4.1.4203.1.11.3
Oct 19 12:25:58 ldap-server slapd[47122]: conn=1045953 op=1 WHOAMI
Oct 19 12:25:58 ldap-server slapd[47122]: conn=1045953 op=1 RESULT oid= err=0 qtime=0.000033 etime=0.000161 text=

Like i had mentioned before i noticed that when salt-master binds to ldap when using `salt -a ldap bla` the binding is the same as shown in the logs above.
But when trying to auth through the api, the mechanism is missing and auth fails.

Oct 19 12:35:08 ldap-server slapd[47122]: conn=1046535 fd=66 ACCEPT from IP=[x:x:x]:34820 (IP=[::]:389)
Oct 19 12:35:08 ldap-server slapd[47122]: conn=1046535 op=0 BIND dn="uid=blabla,ou=people,dc=domain,dc=net" method=128
Oct 19 12:35:08 ldap-server slapd[47122]: conn=1076 op=1 ENTRY dn="uid=blabla,ou=people,dc=domain,dc=net"
Oct 19 12:35:08 ldap-server slapd[47122]: conn=1038 op=1 ENTRY dn="uid=blabla,ou=people,dc=domain,dc=net"
Oct 19 12:35:08 ldap-server slapd[47122]: conn=1046535 op=0 RESULT tag=97 err=49 qtime=0.000026 etime=0.029017 text=

Since the ldap configuration on the salt server side is the same, and it works fine with regular `salt` cmds but not with api, then either i have a misconfiguration or maybe a bug.

Thank you again for the reply

Best,
Dave

Simon Lundström

unread,
Oct 21, 2022, 2:42:55 AM10/21/22
to salt-...@googlegroups.com
Hey Dave!

Interesting!

You didn't really post your salt LDAP configuration, can you do so as
well?

The docs on it is located here:
https://docs.saltproject.io/en/latest/topics/eauth/index.html#openldap-and-similar-systems

Have a great weekend!

BR,
- Simon
> --
> You received this message because you are subscribed to the Google Groups "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com<mailto:salt-users+...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/CA%2BnFYV9hWGkJqBGjV%3DZ119cEpV6RMT4hVQrZaqmUTrhpeV1YEg%40mail.gmail.com<https://groups.google.com/d/msgid/salt-users/CA%2BnFYV9hWGkJqBGjV%3DZ119cEpV6RMT4hVQrZaqmUTrhpeV1YEg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Dave Macias

unread,
Oct 21, 2022, 8:18:20 AM10/21/22
to salt-...@googlegroups.com
Ah! how foolish

Here it is:

auth.ldap.server: ldap-server.domain.net
auth.ldap.port: 389
auth.ldap.tls: False
auth.ldap.starttls: False
auth.ldap.scope: 2
auth.ldap.basedn: dc=domain,dc=net
auth.ldap.binddn: uid={{username}},ou=people,dc=domain,dc=net
auth.ldap.filter: uid={{ username }}
auth.ldap.no_verify: True
auth.ldap.anonymous: False
auth.ldap.auth_by_group_membership_only: False
auth.ldap.groupou: 'group'
auth.ldap.groupclass: 'posixGroup'
auth.ldap.groupattribute: 'member'
auth.ldap.accountattributename: 'uid'
auth.ldap.group_basedn: ou=group,dc=domain,dc=net
auth.ldap.group_filter: '(&(member=uid={{username}},ou=people,dc=domain,dc=net)(objectClass=posixGroup))'
auth.ldap.activedirectory: False
auth.ldap.persontype: 'person'
auth.ldap.minion_stripdomains: []
auth.ldap.freeipa: True


Thank you Simon, you enjoy your weekend as well

Best,
Dave

To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/Y1I/ZybN8I2vf6N3%40meow.it.su.se.

Simon Lundström

unread,
Oct 24, 2022, 7:58:05 AM10/24/22
to salt-...@googlegroups.com
I have no idea, but you're not running FreeIPA from what I could tell so
maybe set that to False? I don't see that it would change anything
relevant in the code though but I only gave it a very shallow read.

Looks like there is at least a "Success" debug log, so add debug log and
see if you actually see that. If not some of the conditionals might not
be right so add more logging to figure out what happens:
https://github.com/saltstack/salt/blob/master/salt/auth/ldap.py#L335-L370

Maybe some salt people can help fill in the blanks?

Good luck!

BR,
- Simon

On Fri, 2022-10-21 at 14:17:57 +0200, Dave Macias wrote:
> Ah! how foolish
>
> Here it is:
>
> auth.ldap.server: ldap-server.domain.net<http://ldap-server.domain.net>
> auth.ldap.port: 389
> auth.ldap.tls: False
> auth.ldap.starttls: False
> auth.ldap.scope: 2
> auth.ldap.basedn: dc=domain,dc=net
> auth.ldap.binddn: uid={{username}},ou=people,dc=domain,dc=net
> auth.ldap.filter: uid={{ username }}
> auth.ldap.no_verify: True
> auth.ldap.anonymous: False
> auth.ldap.auth_by_group_membership_only: False
> auth.ldap.groupou: 'group'
> auth.ldap.groupclass: 'posixGroup'
> auth.ldap.groupattribute: 'member'
> auth.ldap.accountattributename: 'uid'
> auth.ldap.group_basedn: ou=group,dc=domain,dc=net
> auth.ldap.group_filter: '(&(member=uid={{username}},ou=people,dc=domain,dc=net)(objectClass=posixGroup))'
> auth.ldap.activedirectory: False
> auth.ldap.persontype: 'person'
> auth.ldap.minion_stripdomains: []
> auth.ldap.freeipa: True
>
> Thank you Simon, you enjoy your weekend as well
>
> Best,
> Dave
>
> > To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com<mailto:salt-users%2Bunsu...@googlegroups.com><mailto:salt-users+...@googlegroups.com<mailto:salt-users%2Bunsu...@googlegroups.com>>.
> To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com<mailto:salt-users%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/Y1I/ZybN8I2vf6N3%40meow.it.su.se.
>
> --
> You received this message because you are subscribed to the Google Groups "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com<mailto:salt-users+...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/CA%2BnFYV_M5_F4gGW%2BKDvRs5qz4QAa_XeojT%3Dp0ZFjGz5cFjqdNw%40mail.gmail.com<https://groups.google.com/d/msgid/salt-users/CA%2BnFYV_M5_F4gGW%2BKDvRs5qz4QAa_XeojT%3Dp0ZFjGz5cFjqdNw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Dave Macias

unread,
Oct 24, 2022, 10:36:02 AM10/24/22
to salt-...@googlegroups.com
Thank you for the feedback.
Ill try to enable more debug on the ldap module and report back.

Regarding auth.ldap.freeipa: True
Honestly, it is not very clear to me what it is supposed to do, but from the behavior I have seen, when setting to True vs False, is that for group membership, when True, it uses the "(&(member=uid=bla,ou=people,dc=domain,dc=net)(objectClass=posixGroup))" filter. Our openldap installation uses the rfc2307bis schema which uses both posfixGroup and groupOfNames objectclass, so because of this, we have to enable `auth.ldap.freeipa: True` so that the salt master can properly search through the groups. Otherwise, if set to false, the group search filter is simply "(&(uid=bla)(objectClass=posixGroup))" which is not the case when using the rfc2307bis schema. So that should be fine.

Best,
Dave




To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/Y1Z9w5qYa56onC5W%40meow.it.su.se.

Dave Macias

unread,
Oct 24, 2022, 2:19:38 PM10/24/22
to salt-...@googlegroups.com
Ok, i think i figured out the issue now....

So the good thing is that my current configuration works, so im happy about that.
The interesting issue i found was that it does not like the way I send the password using curl....

So with an account that works fine:

curl -4 -k -sSi https://salt-master.domain.net:8000/login -H 'Accept: application/x-yaml' -d username=serviceaccnt -d password=nospecialcharacterspassword -d eauth=ldap
HTTP/1.1 200 OK
Content-Type: application/x-yaml
Server: CherryPy/18.8.0
Date: Mon, 24 Oct 2022 17:44:30 GMT
Allow: GET, HEAD, POST
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: GET, POST
Access-Control-Allow-Credentials: true
X-Auth-Token: 0118e4059f9b821bf8b2166f86b2898cdec880c7
Vary: Accept-Encoding
Content-Length: 165
Set-Cookie: session_id=0118e4059f9b821bf8b2166f86b2898cdec880c7; expires=Tue, 25 Oct 2022 03:44:30 GMT; Max-Age=36000; Path=/

return:
- eauth: ldap
  expire: 1667238270.4198487
  perms: {}
  start: 1666633470.4198482
  token: 0118e4059f9b821bf8b2166f86b2898cdec880c7
  user: serviceaccnt

My personal credentials have a special character in my password. So for some reason, it does not like the way it sends it via curl: (for the SHELL to ignore the special character i placed it in 'single quotes')

> curl blabla username=bla password='mypasswordwithspecialcharacter' eauth=ldap

So tried it with python and it works fine:

>>> url = 'https://salt-master:8000'
>>> headers = {'Accept': 'application/x-yaml'}
>>> password = 'passwordwithspecialcharacter'
>>> username = 'bla'
>>> data = {'username': username, 'password': password, 'eauth': 'ldap'}
>>> import requests
>>> response = requests.post(f'{url}/login', headers=headers, json=data, verify=False)
>>> print(response.content)
b"return:\n- eauth: ldap\n  expire: 1667239998.083698\n  perms:\n  - .*\n  - '@jobs'\n  - '@wheel'\n  - '@runners'\n  start: 1666635198.083697\n  token: 6c9b10614a6e7185455aa712e5552919ffb0ba1f\n  user: bla\n"

So i guess this original issue is solved.

Thank you Simon for the feedback and assistance, very appreciated.

Be well

Best,
Dave

Dave Macias

unread,
Oct 24, 2022, 2:24:32 PM10/24/22
to salt-...@googlegroups.com
For future googlers

to send data payload using curl which has special characters, use the `--data-urlencode` option. (i still had to enclose the password in single quotes)

> curl -k -sSi https://salt-master:8000/login -H 'Accept: application/x-yaml' -d username=bla --data-urlencode password='specialcharactershereinpassword' -d eauth=ldap
return:
- eauth: ldap
  expire: 1667240484.3976755
  perms:

  - .*
  - '@jobs'
  - '@wheel'
  - '@runners'
  start: 1666635684.3976748
  token: 195b3eca5604fdcc81edf623e74b9ea3ea6d35e6
  user: bla

Thanks
Reply all
Reply to author
Forward
0 new messages