wazuh-dashboard 500 Error when using Password+TOTP on LDAP

436 views
Skip to first unread message

Leo LemogrA

unread,
Nov 20, 2023, 2:25:05 AM11/20/23
to Wazuh | Mailing List
Greeting everyone,

I have created a Wazuh Cluster following below documentation with 3 servers,3 indexer and a dashboard also using a HaProxy load balancer(which is not included in documentation).

Than i have applied LDAP authentication with FreeIPA which is awesome. I can login with any authorised users in LDAP with password but when i change login method from Password to *Two factor authentication (password + OTP)* wazuh dashboard fails with below 500 error.

{"type":"response","@timestamp":"2023-11-17T10:16:16Z","tags":[],"pid":39364,"method":"post","statusCode":200,"req":{"url":"/auth/login","method":"post","headers":{"x-forwarded-for":"10.10.100.2","host":"10.10.3.72","connection":"upgrade","content-length":"57","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/119.0","accept":"*/*","accept-language":"tr-TR,tr;q=0.8,en-US;q=0.5,en;q=0.3","accept-encoding":"gzip, deflate, br","referer":"https://10.10.3.72/app/login?","content-type":"application/json","osd-version":"2.8.0","osd-xsrf":"osd-fetch","origin":"https://10.10.3.72","sec-fetch-dest":"empty","sec-fetch-mode":"cors","sec-fetch-site":"same-origin","pragma":"no-cache","cache-control":"no-cache"},"remoteAddress":"127.0.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/119.0","referer":"https://10.10.3.72/app/login?"},"res":{"statusCode":200,"responseTime":118,"contentLength":9},"message":"POST /auth/login 200 118ms - 9.0B"}

{"type":"log","@timestamp":"2023-11-17T10:16:17Z","tags":["error","http","server","OpenSearchDashboards"],"pid":39364,"message":"Error: Authentication Exception\n    at SecurityClient.authinfo (/usr/share/wazuh-dashboard/plugins/securityDashboards/server/backend/opensearch_security_client.ts:118:13)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at /usr/share/wazuh-dashboard/plugins/securityDashboards/server/auth/types/authentication_type.ts:208:18\n    at Object.interceptAuth [as authenticate] (/usr/share/wazuh-dashboard/src/core/server/http/lifecycle/auth.js:112:22)\n    at exports.Manager.execute (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at module.exports.internals.Auth._authenticate (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/auth.js:273:30)\n    at Request._lifecycle (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/request.js:281:9)"}

{"type":"error","@timestamp":"2023-11-17T10:16:16Z","tags":[],"pid":39364,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toInternalError (/usr/share/wazuh-dashboard/src/core/server/http/router/response_adapter.js:80:19)\n    at Object.interceptAuth [as authenticate] (/usr/share/wazuh-dashboard/src/core/server/http/lifecycle/auth.js:151:34)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at exports.Manager.execute (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at module.exports.internals.Auth._authenticate (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/auth.js:273:30)\n    at Request._lifecycle (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/request.js:281:9)"},"url":"https://10.10.3.72/","message":"Internal Server Error"}

{"type":"response","@timestamp":"2023-11-17T10:16:16Z","tags":[],"pid":39364,"method":"get","statusCode":500,"req":{"url":"/","method":"get","headers":{"x-forwarded-for":"10.10.100.2","host":"10.10.3.72","connection":"upgrade","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/119.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8","accept-language":"tr-TR,tr;q=0.8,en-US;q=0.5,en;q=0.3","accept-encoding":"gzip, deflate, br","referer":"https://10.10.3.72/app/login?","upgrade-insecure-requests":"1","sec-fetch-dest":"document","sec-fetch-mode":"navigate","sec-fetch-site":"same-origin","sec-fetch-user":"?1","pragma":"no-cache","cache-control":"no-cache"},"remoteAddress":"127.0.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/119.0","referer":"https://10.10.3.72/app/login?"},"res":{"statusCode":500,"responseTime":526,"contentLength":9},"message":"GET / 500 526ms - 9.0B"}

I am %100 sure that my LDAP and password+TOTP is working because i am using same authentication method on any other servers, firewalls and databases.

I know that "Two factor authentication (password + OTP)" is working via LDAP on wazuh indexers because when i try to curl any request to any indexers from wazuh-managers and wazuh-dashboard like below i can get a positive response.

curl -k -u LDAP_user_name https://10.10.3.68:9200
>>>>>>>>>
root@dashboard1~# curl -k -u LDAP_user_name https://10.10.3.68:9200
Enter host password for user 'LDAP_user_name':
{
  "name" : "indexer3",
  "cluster_name" : "wazuh-indexer-cluster",
  "cluster_uuid" : "WuPXxjCnT3ynwTJI85ZOKQ",
  "version" : {
    "number" : "7.10.2",
    "build_type" : "rpm",
    "build_hash" : "db90a415ff2fd428b4f7b3f800a51dc229287cb4",
    "build_date" : "2023-06-03T06:24:25.112415503Z",
    "build_snapshot" : false,
    "lucene_version" : "9.6.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}
root@dashboard1~#
>>>>>>>>>

Just in case i am posting wazuh-dashboard config here /etc/wazuh-dashboard/opensearch_dashboards.yml
>>>>>>>>>
server.host: 0.0.0.0
server.port: 443
opensearch.hosts: ["https://10.10.3.66:9200","https://10.10.3.67:9200","https://10.10.3.68:9200"]
opensearch.ssl.verificationMode: none
opensearch.requestHeadersAllowlist: ["securitytenant","Authorization"]
opensearch_security.multitenancy.enabled: false
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
server.ssl.enabled: true
server.ssl.key: "/etc/wazuh-dashboard/certs/dashboard-key.pem"
server.ssl.certificate: "/etc/wazuh-dashboard/certs/dashboard.pem"
opensearch.ssl.certificateAuthorities: ["/etc/wazuh-dashboard/certs/root-ca.pem"]
uiSettings.overrides.defaultRoute: /app/wazuh
logging.dest: "/var/log/wazuh-dashboard/wazuh-dashboard.log"
>>>>>>>>>

Also posting wazuh-indexer conf here /etc/wazuh-indexer/opensearch-security/config.yml

>>>>>>>>>
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
        internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
    authc:
      kerberos_auth_domain:
        http_enabled: false
        transport_enabled: false
        order: 6
        http_authenticator:
          type: kerberos
          challenge: false
          config:
            krb_debug: false
            strip_realm_from_principal: true
        authentication_backend:
          type: noop
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 4
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      proxy_auth_domain:
        description: "Authenticate via proxy"
        http_enabled: false
        transport_enabled: false
        order: 3
        http_authenticator:
          type: proxy
          challenge: false
          config:
            user_header: "x-proxy-user"
            roles_header: "x-proxy-roles"
        authentication_backend:
          type: noop
      jwt_auth_domain:
        description: "Authenticate via Json Web Token"
        http_enabled: false
        transport_enabled: false
        order: 0
        http_authenticator:
          type: jwt
          challenge: false
          config:
            signing_key: "base64 encoded HMAC key or public RSA/ECDSA pem key"
            jwt_header: "Authorization"
            jwt_url_parameter: null
            jwt_clock_skew_tolerance_seconds: 30
            roles_key: null
            subject_key: null
        authentication_backend:
          type: noop
      clientcert_auth_domain:
        description: "Authenticate via SSL client certificates"
        http_enabled: false
        transport_enabled: false
        order: 2
        http_authenticator:
          type: clientcert
          config:
            username_attribute: cn #optional, if omitted DN becomes username
          challenge: false
        authentication_backend:
          type: noop
      ldap:
        description: "Authenticate via LDAP or Active Directory"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: ldap
          config:
            enable_ssl: false
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: false
            hosts:
            - 10.10.3.19:389
            - 10.10.3.20:389
            bind_dn: uid=ldapadmin,cn=sysaccounts,cn=etc,dc=LDAP_domain_comes_here,dc=net
            password: LDAP_Password_Comes_Here
            userbase: cn=users,cn=accounts,dc=LDAP_domain_comes_here,dc=net
            usersearch: (uid={0})
            #usersearch: (sAMAccountName={0})
            username_attribute: uid
    authz:
      roles_from_myldap:
        description: "Authorize via LDAP or Active Directory"
        http_enabled: true
        transport_enabled: true
        authorization_backend:
          # LDAP authorization backend (gather roles from a LDAP or Active Directory, you have to configure the above LDAP authentication backend settings too)
          type: ldap
          config:
            enable_ssl: false
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: true
            hosts:
            - 10.10.3.19:389
            - 10.10.3.20:389
            bind_dn: uid=ldapadmin,cn=sysaccounts,cn=etc,dc=LDAP_domain_comes_here,dc=net
            password: LDAP_Password_Comes_Here
            rolebase: cn=groups,cn=accounts,dc=LDAP_domain_comes_here,dc=net
            rolesearch_enabled: true
            rolesearch: (member={0})
            userroleattribute: null
            userrolename: none
            rolename: cn
            resolve_nested_roles: true
            userbase: cn=users,cn=accounts,dc=LDAP_domain_comes_here,dc=net
            usersearch: (uid={0})
            skip_users:
              - kibanaserver
              - admin
      roles_from_another_ldap:
        description: "Authorize via another Active Directory"
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          type: ldap
>>>>>>>>>

I'm out of ideas and dont know what to do next, any help will be appreciated.

Leo LemogrA

unread,
Nov 24, 2023, 1:45:30 AM11/24/23
to Wazuh | Mailing List
As still i'am working on this issue i have realized that this is not a wazuh-dashboard issue. It is about wazuh-indexer. When using wazuh-indexer cluster it seems that any of the indexer's tries to authenticate with the LDAP server. Since i use Password+TOTP for authentication, first indexer be able to authenticate but the other follower indexers fails to authenticate.

So basically if i have 3 indexer 3 of them tries authenticate individually. So the first indexer could be able to authenticate with given password + TOTP but the next indexer fails to login since it is expecting another TOTP.  Below logs are taken from my LDAP server. You may notice the conn_id's,  they are making another connection requests.

Failed LDAP authention with Password+TOTP
>>>>>>>>>
{ "date": "[23\/Nov\/2023:17:35:01.843145787 +0300] ", "utc_time": "1700750101.843145787", "event": "BIND_SUCCESS", "dn": "uid=ldapadmin,cn=sysaccounts,cn=etc,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.68", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66727, "op_id": 0, "msg": "" }
{ "date": "[23\/Nov\/2023:17:35:01.896653991 +0300] ", "utc_time": "1700750101.896653991", "event": "BIND_SUCCESS", "dn": "uid=LDAP_user_comes_here,cn=users,cn=accounts,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.68", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66728, "op_id": 0, "msg": "" }
{ "date": "[23\/Nov\/2023:17:35:01.929256968 +0300] ", "utc_time": "1700750101.929256968", "event": "BIND_SUCCESS", "dn": "uid=ldapadmin,cn=sysaccounts,cn=etc,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.68", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66729, "op_id": 0, "msg": "" }
{ "date": "[23\/Nov\/2023:17:35:02.646851958 +0300] ", "utc_time": "1700750102.646851958", "event": "BIND_SUCCESS", "dn": "uid=ldapadmin,cn=sysaccounts,cn=etc,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.67", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66731, "op_id": 0, "msg": "" }
{ "date": "[23\/Nov\/2023:17:35:02.659266590 +0300] ", "utc_time": "1700750102.659266590", "event": "TCP_ERROR", "client_ip": "10.10.3.67", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66732, "msg": "Bad Ber Tag or uncleanly closed connection - B1" }
>>>>>>>>>


Successfully LDAP authention with Only Password
>>>>>>>>>
{ "date": "[23\/Nov\/2023:17:36:42.393713873 +0300] ", "utc_time": "1700750202.393713873", "event": "BIND_SUCCESS", "dn": "uid=ldapadmin,cn=sysaccounts,cn=etc,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.66", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66741, "op_id": 0, "msg": "" }
{ "date": "[23\/Nov\/2023:17:36:42.441067030 +0300] ", "utc_time": "1700750202.441067030", "event": "BIND_SUCCESS", "dn": "uid=LDAP_user_comes_here,cn=users,cn=accounts,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.66", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66742, "op_id": 0, "msg": "" }
{ "date": "[23\/Nov\/2023:17:36:43.085569289 +0300] ", "utc_time": "1700750203.085569289", "event": "BIND_SUCCESS", "dn": "uid=ldapadmin,cn=sysaccounts,cn=etc,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.68", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66743, "op_id": 0, "msg": "" }
{ "date": "[23\/Nov\/2023:17:36:43.126256613 +0300] ", "utc_time": "1700750203.126256613", "event": "BIND_SUCCESS", "dn": "uid=LDAP_user_comes_here,cn=users,cn=accounts,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.68", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66744, "op_id": 0, "msg": "" }
{ "date": "[23\/Nov\/2023:17:36:43.694882083 +0300] ", "utc_time": "1700750203.694882083", "event": "BIND_SUCCESS", "dn": "uid=ldapadmin,cn=sysaccounts,cn=etc,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.67", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66745, "op_id": 0, "msg": "" }
{ "date": "[23\/Nov\/2023:17:36:43.735033843 +0300] ", "utc_time": "1700750203.735033843", "event": "BIND_SUCCESS", "dn": "uid=LDAP_user_comes_here,cn=users,cn=accounts,dc=LDAP_domain_comes_here,dc=net", "bind_method": "SIMPLE", "root_dn": false, "client_ip": "10.10.3.67", "server_ip": "10.10.3.20", "ldap_version": 3, "conn_id": 66746, "op_id": 0, "msg": "" }
>>>>>>>>>

Is there any method / workaround that i can handle this issue?

Kasim Mustapha

unread,
Nov 24, 2023, 8:17:45 AM11/24/23
to Wazuh | Mailing List
Hello Leo,

Apologies for not getting back to you sooner.

So basically, OpenSearch currently lacks support for the mentioned feature. I don't think OpenSearch supports two-factor authentication (2FA) with OTP. 

This absence has been noted as a feature request, which can be found at https://github.com/opensearch-project/security/issues/3288. Feel free to reopen the request if needed.

Let me know if you have further questions.

Regards,
Kasim Mustapha
Reply all
Reply to author
Forward
0 new messages