Windows: establishing SSL communication in ELK using searchguard

[vinod hy]

Hi,

I am new to searchguard. 

My intention is to establish a secure communication in the ELK communication. 

Following are the changes that i have performed initially,

Elasticsearch.yml

searchguard.authcz.admin_dn:                                

  - CN=sgadmin 

searchguard.ssl.transport.keystore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\CN=localhost-keystore.jks

searchguard.ssl.transport.keystore_password: 221749a2add117cf889f 

searchguard.ssl.transport.truststore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\truststore.jks 

searchguard.ssl.transport.truststore_password: 6d6cf1cc017dc874960b

Kibana.yml

elasticsearch.url: "http://localhost:9200"

elasticsearch.username: "admin"

elasticsearch.password: "admin"

Logstash.config

output {

    elasticsearch {

hosts => ["localhost:9200"]

user => admin

        password => admin

}

stdout { codec => rubydebug }

}

With the above changes, i was able to login to kibana and elasticsearch using cerdentials admin/admin . 

Now i want to login to elasticsearch and kibana using https.

following are the changes i performed,

Elasticsearch.yml

searchguard.ssl.http.enabled: true

searchguard.ssl.http.keystore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\CN=localhost-keystore.jks

searchguard.ssl.http.keystore_password: 221749a2add117cf889f

searchguard.ssl.http.truststore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\truststore.jks 

searchguard.ssl.http.truststore_password: 6d6cf1cc017dc874960b 

searchguard.authcz.admin_dn:                                

  - CN=sgadmin 

searchguard.ssl.transport.keystore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\CN=localhost-keystore.jks

searchguard.ssl.transport.keystore_password: 221749a2add117cf889f 

searchguard.ssl.transport.truststore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\truststore.jks 

searchguard.ssl.transport.truststore_password: 6d6cf1cc017dc874960b

With this, i was able to login to elasticsearch using https.

But i want to know what changes i need to do in Kibana.yml and logstash.config .

Please help me in the regard. I am not able to find info online.

[vinod hy]

Hi,

I added the below line in kibana.yml,

elasticsearch.ssl.ca: "D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/root-ca.crt" 

and i am able to login into localhost:5601 using admin/admin. 

But my requirement is to login using https://localhost:5601 ..

Please help me here

[SG]

Kibana.yml:
elasticsearch.url: "https://localhost:9200"
elasticsearch.username: "admin"
elasticsearch.password: "admin"
elasticsearch.ssl.verify: false

Logstash.config:
output {
elasticsearch {
hosts => ["https://localhost:9200"]

user => admin
password => admin

ssl_certificate_verification => false
}
stdout { codec => rubydebug }
}

Just change http:// to https:// (like in your brwoser when you want to reach the ssl version of the webseite)

Docs can be found here:
https://github.com/floragunncom/search-guard-docs/blob/master/kibana.md
https://github.com/floragunncom/search-guard-docs/blob/master/logstash.md
http://floragunncom.github.io/search-guard-docs/


If you get this working than remove enable certificate verification by setting:

Kibana.yml:
elasticsearch.url: "https://localhost:9200"
elasticsearch.username: "admin"
elasticsearch.password: "admin"
elasticsearch.ssl.verify: true
elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/root-ca.pem" ]


Logstash.config:
output {
elasticsearch {
hosts => ["https://localhost:9200"]

user => admin
password => admin

ssl_certificate_verification => true
truststore => "/path/to/truststore.jks"
truststore_password => "changeit"
}
stdout { codec => rubydebug }
}

You can also have a look here:
https://www.elastic.co/guide/en/x-pack/current/logstash.html#ls-http-ssl
https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html
https://www.elastic.co/guide/en/kibana/current/settings.html

> --
> You received this message because you are subscribed to the Google Groups "Search Guard" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/c225afe0-07a0-45bb-8ef6-35ef49fe4e0d%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[vinod hy]

Hi, 

Thanks a lot for your inputs.

I directly tried the second part of your explanation. I got it working but there is one issue which i observed.

In kibana.yml:

If i add the line,

elasticsearch.ssl.verify: true,

i get the below error in kibana logs.

{"type":"log","@timestamp":"2017-07-04T06:10:29Z","tags":["fatal"],"pid":17172,"level":"fatal","message":"child \"elasticsearch\" fails because [child \"ssl\" fails because [\"enabled\" is not allowed]]","error":{"message":"child \"elasticsearch\" fails because [child \"ssl\" fails because [\"enabled\" is not allowed]]","name":"ValidationError","stack":"ValidationError: child \"elasticsearch\" fails because [child \"ssl\" fails because [\"enabled\" is not allowed]]\n    at Object.exports.process (D:\\Softwares\\ELK\\kibana-5.4.0-windows-x86\\kibana-5.4.0-windows-x86\\node_modules\\joi\\lib\\errors.js:140:17)\n    at internals.Any._validateWithOptions (D:\\Softwares\\ELK\\kibana-5.4.0-windows-x86\\kibana-5.4.0-windows-x86\\node_modules\\joi\\lib\\any.js:649:25)\n    at root.validate (D:\\Softwares\\ELK\\kibana-5.4.0-windows-x86\\kibana-5.4.0-windows-x86\\node_modules\\joi\\lib\\index.js:102:23)\n    at Config._commit (D:\\Softwares\\ELK\\kibana-5.4.0-windows-x86\\kibana-5.4.0-windows-x86\\src\\server\\config\\config.js:113:35)\n    at Config.set (D:\\Softwares\\ELK\\kibana-5.4.0-windows-x86\\kibana-5.4.0-windows-x86\\src\\server\\config\\config.js:84:10)\n    at Config.extendSchema (D:\\Softwares\\ELK\\kibana-5.4.0-windows-x86\\kibana-5.4.0-windows-x86\\src\\server\\config\\config.js:57:10)\n    at D:\\Softwares\\ELK\\kibana-5.4.0-windows-x86\\kibana-5.4.0-windows-x86\\src\\server\\plugins\\plugin_collection.js:19:12\n    at next (native)\n    at step (D:\\Softwares\\ELK\\kibana-5.4.0-windows-x86\\kibana-5.4.0-windows-x86\\src\\server\\plugins\\plugin_collection.js:49:191)\n    at D:\\Softwares\\ELK\\kibana-5.4.0-windows-x86\\kibana-5.4.0-windows-x86\\src\\server\\plugins\\plugin_collection.js:49:361"}}

So i used the configuration files as below,

These below configuration worked fine for me.

kibana.yml

elasticsearch.url: "https://localhost:9200"

elasticsearch.username: "admin"

elasticsearch.password: "admin"

elasticsearch.ssl.certificateAuthorities: [ "D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/root-ca.crt" ]

logstash.config

output {

    elasticsearch {

hosts => ["localhost:9200"]

user => admin

    password => admin

ssl => true

        ssl_certificate_verification => true

truststore => "D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/truststore.jks"

    truststore_password => "6d6cf1cc017dc874960b"

}

stdout { codec => rubydebug }

}

My doubts

1. Please let me know why elasticsearch.ssl.verify: true in kibana.yml is throwing error. Is it ok to omit this line.

2. Now i want to use openssl certificates instead of jks. Please help me in using open ssl.

[Jochen Kressin]

What's with the OpenSSL / JKS question? A jks container simply stores certificates in a certain format. Use the keytool command to work with JKS container:

https://docs.oracle.com/javase/6/docs/technotes/tools/solaris/keytool.html

Your OpenSSL certificates are there already, inside the container.

Please read also the docs regarding Kibana and logstash setup:

http://floragunncom.github.io/search-guard-docs/kibana.html

http://floragunncom.github.io/search-guard-docs/logstash.html

There you will find:

You can disable certificate validation in kibana.yml:

Kibana >= 5.3.0

elasticsearch.ssl.verificationMode: none

Kibana < 5.3.0:

elasticsearch.ssl.verify: false




This is a configuration changed introduced by Kibana in 5.3.0.

[vinod hy]

Hi Jochen,

     I really appreciate your time and patience. 

      i will elaborate my situation to make you understand better.

       

       I have attached the self signed certificates which i have generated using openssl command line tool. Please go through the attachment for the same.

     Now, I want to use these certificates instead of using the one which i got generated from searchguard certificate generator link. 

     

      But these certificates doesnt have truststore and keystore password as present in readme document of searchguard certificates.

    May i please know how to handle this. Please let me know if i am missing anything here.

[vinod hy]

Please explain me some some points here on what knowledge i am missing here.

Then i will go through relavant links online and understand better.

My requirement is to use my own certificates instead of using the certificates which i generated using searchguard certificate generator.

I have generated my own self certtificates using openssl command line tool.