Kibana Fails to connect to Elasticsearch after the install of SearchGuard

Neuronring

unread,

Nov 28, 2016, 6:07:15 PM11/28/16

to Search Guard

I am getting the below error when i start the Kibana instance after installing SearchGuard.

I did update the Kibana.yml file with the elasticsearch username and password. Not sure what i am missing here.

Can you help to troubleshoot ?

  log   [22:52:02.901] [warning][elasticsearch] Unable to revive connection: https://xx.yy.zz.yyy:9200/
  log   [22:52:02.902] [warning][elasticsearch] No living connections
  log   [22:52:05.442] [warning][elasticsearch] Unable to revive connection: https://xx.yy.zz.yyy:9200/
  log   [22:52:05.443] [warning][elasticsearch] No living connections

# If your Elasticsearch is protected with basic auth, these are the user credentials
# used by the Kibana server to perform maintenance on the kibana_index at startup. Your Kibana
# users will still need to authenticate with Elasticsearch (which is proxied through
# the Kibana server)
elasticsearch.username: "admin"
elasticsearch.password: "pass"

Jochen Kressin

unread,

Dec 17, 2016, 3:31:47 PM12/17/16

to Search Guard

Have you enabled TLS on the REST layer? Of so, you also need to change the elasticsearch urls in your kibana.yml file to use https, e.g.:

elasticsearch.url: "https://localhost:9200"

(note the https instead of http)

no0ker _

unread,

Mar 4, 2017, 2:53:03 PM3/4/17

to Search Guard

I have the same problem =(

Response by https://127.0.0.1:9200/_searchguard/sslinfo?pretty

1. {
  "principal" : "CN=spock,OU=client,O=client,L=Test,C=DE",
  "peer_certificates" : "2",
  "ssl_protocol" : "TLSv1.2",
  "ssl_cipher" : "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
  "ssl_openssl_available" : false,
  "ssl_openssl_version" : -1,
  "ssl_openssl_version_string" : null,
  "ssl_openssl_non_available_cause" : "java.lang.ClassNotFoundException: org.apache.tomcat.jni.SSL",
  "ssl_openssl_supports_key_manager_factory" : false,
  "ssl_provider_http" : "JDK",
  "ssl_provider_transport_server" : "JDK",
  "ssl_provider_transport_client" : "JDK"
}

Is it ok?

2. when i set elasticsearch.url: "https://localhost:9200", i give this message from log and kibana doesn't work

log [19:52:04.540] [info][status][plugin:kib...@5.2.1] Status changed from uninitialized to green - Ready

log [19:52:04.589] [info][status][plugin:elasti...@5.2.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch

log [19:52:04.614] [info][status][plugin:con...@5.2.1] Status changed from uninitialized to green - Ready

log [19:52:04.627] [error][admin][elasticsearch] Request error, retrying

HEAD https://localhost:9200/ => write EPROTO 0:error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:openssl\ssl\s3_pkt.c:1493:SSL alert number 42

0:error:1409E0E5:SSL routines:ssl3_write_bytes:ssl handshake failure:openssl\ssl\s3_pkt.c:659:

log [19:52:04.858] [warning][admin][elasticsearch] Unable to revive connection: https://localhost:9200/

log [19:52:04.859] [warning][admin][elasticsearch] No living connections

log [19:52:04.861] [info][status][plugin:time...@5.2.1] Status changed from uninitialized to green - Ready

log [19:52:04.863] [error][status][plugin:elasti...@5.2.1] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.

log [19:52:04.867] [info][listening] Server running at http://localhost:5601

log [19:52:04.868] [error][status][ui settings] Status changed from uninitialized to red - Elasticsearch plugin is red

log [19:52:07.421] [warning][admin][elasticsearch] Unable to revive connection: https://localhost:9200/

log [19:52:07.423] [warning][admin][elasticsearch] No living connections

log [19:52:09.973] [warning][admin][elasticsearch] Unable to revive connection: https://localhost:9200/

log [19:52:09.978] [warning][admin][elasticsearch] No living connections

could you please help me?

воскресенье, 18 декабря 2016 г., 1:31:47 UTC+5 пользователь Jochen Kressin написал:

SG

unread,

Mar 4, 2017, 3:22:43 PM3/4/17

to search...@googlegroups.com

Please provide your kibana.yml and elasticsearch.yml

> --
> You received this message because you are subscribed to the Google Groups "Search Guard" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/d2150e65-08d8-4b64-99c7-39024ab6c868%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

no0ker _

unread,

Mar 5, 2017, 2:03:12 PM3/5/17

to Search Guard

elasticsearch.yml

searchguard.ssl.transport.keystore_filepath: node-0-keystore.jks

searchguard.ssl.transport.keystore_password: changeit

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.truststore_password: changeit

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.keystore_filepath: node-0-keystore.jks

searchguard.ssl.http.keystore_password: changeit

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.ssl.http.truststore_password: changeit

searchguard.ssl.http.clientauth_mode: REQUIRE

kibana.yml

elasticsearch.url: "https://localhost:9200"

all other rows in elasticsearch.yml and kibana.yml has been deleted

and there is in elasticsearch log

[2017-03-05T23:59:57,003][WARN ][c.f.s.s.h.n.SearchGuardSSLNettyHttpServerTransport] [6EkH-6l] caught exception while handling client http traffic, closing connection [id: 0x01ce0af2, L:0.0.0.0/0.0.0.0:9200 ! R:/127.0.0.1:53536]

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: null cert chain

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442) ~[netty-codec-4.1.7.Final.jar:4.1.7.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248) ~[netty-codec-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:527) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:481) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.7.Final.jar:4.1.7.Final]

at java.lang.Thread.run(Unknown Source) [?:1.8.0_111]

Caused by: javax.net.ssl.SSLHandshakeException: null cert chain

at sun.security.ssl.Handshaker.checkThrown(Unknown Source) ~[?:?]

at sun.security.ssl.SSLEngineImpl.checkTaskThrown(Unknown Source) ~[?:?]

at sun.security.ssl.SSLEngineImpl.readNetRecord(Unknown Source) ~[?:?]

at sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source) ~[?:?]

at javax.net.ssl.SSLEngine.unwrap(Unknown Source) ~[?:1.8.0_111]

at io.netty.handler.ssl.SslHandler$SslEngineType$2.unwrap(SslHandler.java:218) ~[?:?]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1028) ~[?:?]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:950) ~[?:?]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411) ~[?:?]

... 15 more

Caused by: javax.net.ssl.SSLHandshakeException: null cert chain

at sun.security.ssl.Alerts.getSSLException(Unknown Source) ~[?:?]

at sun.security.ssl.SSLEngineImpl.fatal(Unknown Source) ~[?:?]

at sun.security.ssl.Handshaker.fatalSE(Unknown Source) ~[?:?]

at sun.security.ssl.ServerHandshaker.clientCertificate(Unknown Source) ~[?:?]

at sun.security.ssl.ServerHandshaker.processMessage(Unknown Source) ~[?:?]

at sun.security.ssl.Handshaker.processLoop(Unknown Source) ~[?:?]

at sun.security.ssl.Handshaker$1.run(Unknown Source) ~[?:?]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_111]

at sun.security.ssl.Handshaker$DelegatedTask.run(Unknown Source) ~[?:?]

at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1167) ~[?:?]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1080) ~[?:?]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:950) ~[?:?]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411) ~[?:?]

... 15 more

how sould i change config files?

=(

воскресенье, 5 марта 2017 г., 1:22:43 UTC+5 пользователь Search Guard написал:

Please provide your kibana.yml and elasticsearch.yml

> Am 04.03.2017 um 20:53 schrieb no0ker _ <rustam.s...@gmail.com>:
>
> I have the same problem =(
> Response by https://127.0.0.1:9200/_searchguard/sslinfo?pretty
> 1. {
> "principal" : "CN=spock,OU=client,O=client,L=Test,C=DE",
> "peer_certificates" : "2",
> "ssl_protocol" : "TLSv1.2",
> "ssl_cipher" : "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
> "ssl_openssl_available" : false,
> "ssl_openssl_version" : -1,
> "ssl_openssl_version_string" : null,
> "ssl_openssl_non_available_cause" : "java.lang.ClassNotFoundException: org.apache.tomcat.jni.SSL",
> "ssl_openssl_supports_key_manager_factory" : false,
> "ssl_provider_http" : "JDK",
> "ssl_provider_transport_server" : "JDK",
> "ssl_provider_transport_client" : "JDK"
> }
>
> Is it ok?
>
> 2. when i set elasticsearch.url: "https://localhost:9200", i give this message from log and kibana doesn't work
>

> log [19:52:04.540] [info][status][plugin:kibana@5.2.1] Status changed from uninitialized to green - Ready

> log [19:52:04.589] [info][status][plugin:elasti...@5.2.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch

> log [19:52:04.614] [info][status][plugin:console@5.2.1] Status changed from uninitialized to green - Ready

SG

unread,

Mar 5, 2017, 2:11:29 PM3/5/17

to search...@googlegroups.com

You REQUIRE client auth but you did not send any certificate (from kibana)?

> > log [19:52:04.540] [info][status][plugin:kib...@5.2.1] Status changed from uninitialized to green - Ready

> > log [19:52:04.589] [info][status][plugin:elasti...@5.2.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch

> > log [19:52:04.614] [info][status][plugin:con...@5.2.1] Status changed from uninitialized to green - Ready

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/393d2ecc-2cde-405f-945d-865749a627c1%40googlegroups.com.

no0ker _

unread,

Mar 5, 2017, 11:38:29 PM3/5/17

to Search Guard

How i can send any sertificate from Kibana?

Should i use server.ssl.cert and server.ssl.key? How file shoul i add from example-pki-scripts?

воскресенье, 5 марта 2017 г., 22:11:29 UTC+3 пользователь Search Guard написал:

> > log [19:52:04.540] [info][status][plugin:kibana@5.2.1] Status changed from uninitialized to green - Ready

> > log [19:52:04.589] [info][status][plugin:elasti...@5.2.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch

> > log [19:52:04.614] [info][status][plugin:console@5.2.1] Status changed from uninitialized to green - Ready

no0ker _

unread,

Mar 6, 2017, 2:49:39 AM3/6/17

to Search Guard

with these keys seems to work..

elasticsearch.ssl.cert: kirk.crtfull.pem

elasticsearch.ssl.key: kirk.key.pem

elasticsearch.ssl.verify: false

but without "elasticsearch.ssl.verify: false" it doesn't work... =((

понедельник, 6 марта 2017 г., 7:38:29 UTC+3 пользователь no0ker _ написал:

Reply all

Reply to author

Forward