Kibana Plugin - Search Guard incompatibility?

[Emlyn Stokes]

I'm getting the following error in Kibana and no Wazuh dashboards appear:

Settings: Error when loading Wazuh setup info

This also shows up in the kibana logs at the same time:

{ Authentication Exception :: {"path":"/.wazuh/wazuh-setup/_search","query":{},"statusCode":401,"response":"Unauthorized","wwwAuthenticateDirective":"Basic realm=\"Search Guard\""}
    at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:295:15)
    at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:254:7)
    at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)
    at IncomingMessage.bound (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickDomainCallback (internal/process/next_tick.js:128:9)
  status: 401,
  displayName: 'AuthenticationException',
  message: 'Authentication Exception',
  path: '/.wazuh/wazuh-setup/_search',
  query: {},
  body: 'Unauthorized',
  statusCode: 401,
  response: 'Unauthorized',
  wwwAuthenticateDirective: 'Basic realm="Search Guard"',
  toString: [Function],
  toJSON: [Function],
  isBoom: true,
  isServer: false,
  data: null,
  output:
   { statusCode: 401,
     payload:
      { statusCode: 401,
        error: 'Unauthorized',
        message: 'Authentication Exception' },
     headers: { 'WWW-Authenticate': 'Basic realm="Authorization Required"' } },
  reformat: [Function] }

 

I have Search Guard configured to not allow HTTP or unauthenticated traffic connect to Elasticsearch, I'm wondering if this could be an issue?

Kibana is configured to connect to Elasticsearch with HTTPS and basic auth.

I have just about everything working.

My manager and API is up and running

My agents are registered

Filebeat on the manager instance is sending logs and logs are showing up in `wazuh-alerts-` and `wazuh-monitoring-` indices (I can see them in the discover tab from kibana)

The Kibana Wazuh app is even connected to the Wazuh API, and was able to save the API URL presumably in the `.wazuh` index.

[Manuel Albarral]

Hello Emlyn,

You are right. As the error says, the App is unauthorized to request to Elasticsearch. Try to disable Search Guard and check if it works. If it does, we will develop this feature for future versions.

Thank you,

Manuel Albarral

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e080dc09-793d-447e-86eb-df1a480415d7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Manuel Albarral Siles

Security Engineer

 man...@wazuh.com   https://wazuh.com

 

[Emlyn Stokes]

Would it be possible to get an export of the Wazuh dashboards that I could import manually?

[Marta Gómez]

Hello Emlyn,

It is very simple to import and export dashboards on Kibana. Just go to management menu, click on Saved Objects and there you will see buttons to import and export dashboards, visualizations and searches. All of this will be exported/imported in JSON format.

An exported dashboard will look like this:

Best regards,

Marta

[Emlyn Stokes]

Thanks Marta, I'm asking for an export of all Wazuh dashboards be provided to me, as the plugin is not able to add them itself due to incompatibility with Search Guard. 

The OSSEC-Alerts is a good start if I could get it in plaintext.

I have no panels, and no dashboards, I only see alerts in the discover tabs.

[Marta Gómez]

Hello Emlyn,

You can use "export everything" button to export all Wazuh dashboards but it's not possible to export the internal visualizations that Wazuh uses on its application.

Best regards,
Marta

[0x2a]

Hello,

could you post your sg_config.yml, kibana.yml (without sensitive information), as well as the corresponding sg_roles / sg_role_mapping?

Are the basic auth credentials valid?
$ curl -v -k "https://user:password@localhost:9200/"

The combination of Searchguard + Kibana + Kibana wazuh plugin does work.

[Emlyn Stokes]

Here are some details about my setup: I've replaced some sensitive information with {{ variable }}

Kibana is on a separate instance from my Elasticsearch nodes.

My Elasticsearch nodes sit behind an ELB with hostname: {{ elasticsearch_host }}

From Kibana instance:

curl -XGET -k -v "https://{{ elasticsearch_host }}:9200/"

HTTP/1.1 401 Unauthorized

From Kibana instance:

curl -XGET -k -v -u {{ sg_user }}:{{ sg_kibana_pass }} "https://{{ elasticsearch_host }}:9200/"

*   Trying XXX...

* Connected to {{ elasticsearch_host }} (XXX) port 9200 (#0)

* successfully set certificate verify locations:

*   CAfile: none

  CApath: /etc/ssl/certs

* SSLv3, TLS handshake, Client hello (1):

* SSLv3, TLS handshake, Server hello (2):

* SSLv3, TLS handshake, CERT (11):

* SSLv3, TLS handshake, Server key exchange (12):

* SSLv3, TLS handshake, Request CERT (13):

* SSLv3, TLS handshake, Server finished (14):

* SSLv3, TLS handshake, CERT (11):

* SSLv3, TLS handshake, Client key exchange (16):

* SSLv3, TLS change cipher, Client hello (1):

* SSLv3, TLS handshake, Finished (20):

* SSLv3, TLS change cipher, Client hello (1):

* SSLv3, TLS handshake, Finished (20):

* SSL connection using 

* Server certificate:

* subject: XXX

* start date: 

* expire date: 

* issuer: XXX

* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

* Server auth using Basic with user '{{ sg_kibana_user }}'

> GET / HTTP/1.1

> Authorization: Basic 

> User-Agent: curl

> Host: {{ elasticsearch_host }}:9200

> Accept: */*

>

< HTTP/1.1 200 OK

< content-type: application/json; charset=UTF-8

< content-length: 359

<

{

  "name" : "XXX",

  "cluster_name" : "XXX",

  "cluster_uuid" : "XXX",

  "version" : {

    "number" : "5.5.1",

    "build_hash" : "19c13d0",

    "build_date" : "2017-07-18T20:44:24.823Z",

    "build_snapshot" : false,

    "lucene_version" : "6.6.0"

  },

  "tagline" : "You Know, for Search"

}

* Connection #0 to host {{ elasticsearch_host }} left intact

kibana.yml

server.port: 9000

server.host: "0.0.0.0"

elasticsearch.url: https://{{ elasticsearch_host }}:9200

elasticsearch.username: "{{ sg_kibana_user }}"

elasticsearch.password: "{{  sg_kibana_pass }}"

# Paths to the PEM-format SSL certificate and SSL key files, respectively. These

# files enable SSL for outgoing requests from the Kibana server to the browser.

#server.ssl.cert: /path/to/your/server.crt

#server.ssl.key: /path/to/your/server.key

elasticsearch.ssl.ca: {{ kibana_cacert_path }}

elasticsearch.ssl.verify: false

console.proxyConfig:

  - match:

      protocol: "https"

    ssl:

      verify: false

sg_config.yml

searchguard:

  dynamic:

    http:

      anonymous_auth_enabled: false

      xff:

        enabled: false

    authc:

      kerberos_auth_domain:

        enabled: false

      basic_internal_auth_domain:

        enabled: true

        order: 4

        http_authenticator:

          type: basic

          challenge: true

        authentication_backend:

          type: intern

      kibana_auth_domain:

        enabled: true

        order: 1

        http_authenticator:

          type: basic

          challenge: true

        authentication_backend:

          type: internal

      proxy_auth_domain:

        enabled: false

      host_auth_domain:

        enabled: false

      jwt_auth_domain:

        enabled: false

      clientcert_auth_domain:

        enabled: false

      ldap:

        enabled: false

    authz:

      roles_from_myldap:

        enabled: false

sg_roles.yml

sg_kibana_user:

  cluster:

      - cluster:monitor/nodes/info

      - cluster:monitor/health

      - indices:data/read/mget*

      - indices:data/read/msearch*

  indices:

    '*':

      '*':

        - "indices:*"

    'wazuh*':

      '*':

        - ALL

    '.wazuh*':

      '*':

        - ALL

sg_roles_mapping.yml

sg_kibana_user:

  users:

    - {{ sg_kibana_user }}

sg_internal_users.yml

{{ sg_kibana_user}}:

  hash: {{ sg_kibana_pass_hash }}

  roles:

    - sg_kibana_user

[Emlyn Stokes]

When first loading the Wazuh Overview section in Kibana, I get this error:

log   [21:09:17.671] [error][wazuh][client] {"statusCode":500,"error":9,"message":"Unexpected error. Please, report this error.","html":"Unexpected error. Please, report this error."}

When opening the configuration page in Wazuh I get the following: Settings: Error when loading Wazuh setup info

However, my API/manager settings are saved, and I see alerts from source:/var/ossec/logs/alerts/alerts.json in the Discover tags

[0x2a]

Hello,

could you post the error from the elasticsearch.log?

thanks,

0x2a

[Emlyn Stokes]

I tailed the elasticsearch logs on all my nodes while browsing the Wazuh plugin on Kibana. I didn't see anything come through in the elasticsearch logs, I don't see any logs in elasticsearch related to unauthorized connections.

Here is everything Wazuh related from today:

[2017-08-17T00:17:11,633][INFO ][o.e.c.m.MetaDataCreateIndexService] [elasticsearch-node] [wazuh-alerts-2017.08.17] creating index, cause [auto(bulk api)], templates [wazuh], shards [5]/[1], mappings [agent, wazuh]

[2017-08-17T00:17:11,816][INFO ][o.e.c.m.MetaDataMappingService] [elasticsearch-node] [wazuh-alerts-2017.08.17/6acladk3RGix1TxFDchoyw] update_mapping [wazuh]

[2017-08-17T00:17:12,402][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-node] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[wazuh-alerts-2017.08.17][1], [wazuh-alerts-2017.08.17][4], [wazuh-alerts-2017.08.17][2]] ...]).

[2017-08-17T00:17:44,471][WARN ][c.f.s.c.PrivilegesEvaluator] wazuh-alerts-2017.08.17 does not exist in cluster metadata

[2017-08-17T00:17:44,472][WARN ][c.f.s.c.PrivilegesEvaluator] wazuh-alerts-2017.08.17 does not exist in cluster metadata

[2017-08-17T15:15:35,147][INFO ][o.e.c.m.MetaDataMappingService] [elasticsearch-node] [wazuh-alerts-2017.08.17/6acladk3RGix1TxFDchoyw] update_mapping [wazuh]

[2017-08-17T15:15:35,197][INFO ][o.e.c.m.MetaDataMappingService] [elasticsearch-node] [wazuh-alerts-2017.08.17/6acladk3RGix1TxFDchoyw] update_mapping [wazuh]

[Emlyn Stokes]

To confirm it is an issue specific to search guard incompatibility,

I upgraded to the latest version of search-guard-5:5.5.1-15 which allows you to disable it

Added `searchguard.disabled: true` to elasticsearch.yml

Change'd elasticsearch_host to http in kibana.yml, removed the user/pass in kibana.yml

Restarted everything

Wazuh dashboards got loaded, plugin appeared to have all functionality, looked great.

Turned Search Guard back on, restarted

Wazuh dashboards remain and are functional, but lost all "Panels" again on the Overview and Agents tab.

[Bernie Carolan]

I am also having this issue using Searchguard, which can be mostly fixed but there is still an issue with anonymous calls relating to the wazuh indexes.
First you can create a wazuh role in Searchguard and map this to the kibana user, which will allow the Wazuh Kibana plugin to install all the dashboards and indices.

sg_wazuh:
  cluster:
    - indices:admin/template/get
    - indices:admin/template/put
    - indices:admin/refresh
    - indices:admin/get
    - CLUSTER_MONITOR
    - CLUSTER_COMPOSITE_OPS
  indices:
    'wazuh-*':
      '*':
        - CRUD
        - CREATE_INDEX
        - SEARCH
        - indices:admin/get
    '?wazuh*':
      '*':
        - CRUD
        - CREATE_INDEX
        - SEARCH
        - indices:admin/get

Here is the log from Elasticsearch showing the Wazuh install

[2017-10-05T04:18:04,435][INFO ][o.e.c.m.MetaDataCreateIndexService] [NODE-1] [wazuh-alerts-2017.10.05] creating index, cause [auto(bulk api)], templates [wazuh], shards [5]/[1], mappings [agent, wazuh]
[2017-10-05T04:18:04,460][INFO ][o.e.c.m.MetaDataCreateIndexService] [NODE-1] [.wazuh] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings []
[2017-10-05T04:18:05,080][INFO ][o.e.c.m.MetaDataMappingService] [NODE-1] [.kibana/0prhjGPSRv6NSsqAkUHTEA] create_mapping [index-pattern]
[2017-10-05T04:18:05,184][INFO ][o.e.c.m.MetaDataMappingService] [NODE-1] [wazuh-alerts-2017.10.05/DqSIPSRzTNeWgu6pRV9bGg] update_mapping [wazuh]
[2017-10-05T04:18:05,200][INFO ][o.e.c.m.MetaDataMappingService] [NODE-1] [.wazuh/jvTM0lVjQEeZZS13z3EOqg] create_mapping [wazuh-setup]
[2017-10-05T04:18:05,297][INFO ][o.e.c.m.MetaDataMappingService] [NODE-1] [.kibana/0prhjGPSRv6NSsqAkUHTEA] create_mapping [dashboard]
[2017-10-05T04:18:05,349][INFO ][o.e.c.m.MetaDataMappingService] [NODE-1] [.kibana/0prhjGPSRv6NSsqAkUHTEA] create_mapping [search]
[2017-10-05T04:18:05,353][INFO ][o.e.c.m.MetaDataMappingService] [NODE-1] [.kibana/0prhjGPSRv6NSsqAkUHTEA] create_mapping [visualization]
[2017-10-05T04:18:05,445][INFO ][o.e.c.m.MetaDataMappingService] [NODE-1] [.kibana/0prhjGPSRv6NSsqAkUHTEA] update_mapping [dashboard]
[2017-10-05T04:18:05,482][INFO ][o.e.c.m.MetaDataMappingService] [NODE-1] [.kibana/0prhjGPSRv6NSsqAkUHTEA] create_mapping [config]
[2017-10-05T04:18:07,267][WARN ][c.f.s.a.BackendRegistry  ] Authentication finally failed for null

The last line indicates that a request is being made from Wazuh as an anonymous user, the error in Kibana logs as follows:

{"type":"log","@timestamp":"2017-10-05T04:18:07Z","tags":["\u001b[34mwazuh\u001b[39m","initialize","error"],"pid":22,"level":"error","message":"Authentication Exception","error":{"message":"Authentication Exception","name":"Error","stack":"Authentication Exception :: {\"path\":\"/.kibana/config/5.5.2/_update\",\"query\":{},\"body\":\"{\\\"doc\\\":{\\\"defaultIndex\\\":\\\"wazuh-alerts-*\\\",\\\"timepicker:timeDefaults\\\":\\\"{  \\\\\\\"from\\\\\\\": \\\\\\\"now-24h\\\\\\\",  \\\\\\\"to\\\\\\\": \\\\\\\"now\\\\\\\",  \\\\\\\"mode\\\\\\\": \\\\\\\"quick\\\\\\\"}\\\",\\\"metaFields\\\":[\\\"_source\\\"]}}\",\"statusCode\":401,\"response\":\"Authentication finally failed\"}\n

This can be fixed by setting anonymous_auth_enabled to true, but this is negates the purpose of Searchguard to secure every request.
I believe this also relates to this post https://github.com/wazuh/wazuh-kibana-app/issues/11#issuecomment-287752186

Any help or working configs from Wazuh with Searchguard greatly appreciated.

[Bernie Carolan]

Any updates on this or working configs?

[Javier Castro]

Hi Bernie,

the call you mention is used to set the wazuh-alerts-* index-pattern as default in the Kibana settings.

We use different types of calls to Elasticsearch and some of them use the internal user Kibana has, some of them don't.

I'll take this into consideration for further development.

Thanks.

Regards.

On Mon, Nov 6, 2017 at 11:49 PM, Bernie Carolan <bernie....@gmail.com> wrote:

Any updates on this or working configs?

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/d254def8-1d46-4b3f-8b13-997207267445%40googlegroups.com.

[Kevin Branch]

I also would like to see full Wazuh Kibana app compatibility with Search Guard.  I have a client wanting to install Search Guard on the same Elastic Stack that their Wazuh infrastructure uses, so this is about to become quite relevant to me.  I'd be happy to help with testing and contribute my findings.

Kevin

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/CAP%2BD2rM9cNEhqdOO8%2BRKjhfz0dU9rkoGxoVWzXEasf0GEKNZ7g%40mail.gmail.com.