Searchguard TLS issue with Apache NiFi

[Peter Bíró]

I have a problem with Apache Nifi and Searchguard communication. I am using Apache NiFi for data ingestion and inserting JSON into Elasticsearch.

Apache Nifi has a procesor StandardRestrictedSSLContextService for SSL configuration and procesor PutElasticsearchHttp for Elastic configuration.

Apache NiFi shows handshare failure:

handshake_failure: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Elasticsearch log message from Searchguard: 

SSL Problem Client requested protocol TLSv1 not enabled or not supported

javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1 not enabled or not supported

Apache Nifi is configured for TLSv1.2. Does anybody know what is the problem?

My other configurations:

* SearchGuard v5.6.5 and Elasticsearch v5.6.5

* no enterprise modules

* Java: 1.8.0_121 

* OS: Debian 9

* Search Guard configuration files

 searchguard.ssl.transport.pemcert_filepath: [NODE1].pem

 searchguard.ssl.transport.pemkey_filepath: [NODE1].key

 searchguard.ssl.transport.pemkey_password: [pass]

 searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem

 searchguard.ssl.transport.enforce_hostname_verification: false

 searchguard.ssl.transport.resolve_hostname: false

 searchguard.ssl.http.enabled: true

 searchguard.ssl.http.pemcert_filepath: [NODE1]_http.pem

 searchguard.ssl.http.pemkey_filepath: [NODE1]_http.key

 searchguard.ssl.http.pemkey_password: [pass]

 searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem

 searchguard.ssl.http.clientauth_mode: OPTIONAL

 searchguard.nodes_dn:

 - CN=[NODE1],OU=Elastic

 - CN=[NODE2],OU=Elastic

 - CN=[NODE3],OU=Elastic

 - CN=[NODE4],OU=Elastic

 searchguard.authcz.admin_dn:

 - CN=[username],OU=Elastic

* Elasticsearch log messages on debug level

[2018-05-02T14:47:27,189][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [hdp-nn1] SSL Problem null cert chain

javax.net.ssl.SSLHandshakeException: null cert chain

at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1478) ~[?:?]

at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535) ~[?:?]

at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:813) ~[?:?]

at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:?]

at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_121]

at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:255) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1162) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]

Caused by: javax.net.ssl.SSLHandshakeException: null cert chain

at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:?]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304) ~[?:?]

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:292) ~[?:?]

at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1862) ~[?:?]

at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:233) ~[?:?]

at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026) ~[?:?]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:966) ~[?:?]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:963) ~[?:?]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_121]

at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1416) ~[?:?]

at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1301) ~[?:?]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1214) ~[?:?]

... 18 more

[2018-05-02T14:47:27,196][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [hdp-nn1] SSL Problem Client requested protocol TLSv1 not enabled or not supported

javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1 not enabled or not supported

at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1478) ~[?:?]

at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535) ~[?:?]

at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:813) ~[?:?]

at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:?]

at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_121]

at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:255) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1162) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]

Caused by: javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1 not enabled or not supported

at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:?]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304) ~[?:?]

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:292) ~[?:?]

at sun.security.ssl.ServerHandshaker.clientHello(ServerHandshaker.java:518) ~[?:?]

at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:224) ~[?:?]

at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026) ~[?:?]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:966) ~[?:?]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:963) ~[?:?]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_121]

at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1416) ~[?:?]

at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1301) ~[?:?]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1214) ~[?:?]

... 18 more

* No other ES/Kibana plugins

[Search Guard]

I guess you should ask this better in the Apache Nifi forum/mailinglist because it seems to be a Nifi issue.

You can of course enable TLSv1 support in Search Guard but we do not recommend this because TLSv1 is considered insecure. 

[Jalaja]

On the same note, I want to know if we support these ciphers by default. The problem I am facing is, GO client is unable to communicate with Elastic due to older ciphers? Given below are the ciphers supported by GO.

https://golang.org/src/crypto/tls/cipher_suites.go

Sun JCE Java version:

bash-4.2# java -version

openjdk version "1.8.0_111"

OpenJDK Runtime Environment (build 1.8.0_111-b15)

OpenJDK 64-Bit Server VM (build 25.111-b15, mixed mode)

var cipherSuites = []*cipherSuite{
  	// Ciphersuite order is chosen so that ECDHE comes before plain RSA and
  	// AEADs are the top preference.
  	{TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, 32, 0, 12, ecdheRSAKA, suiteECDHE | suiteTLS12, nil, nil, aeadChaCha20Poly1305},
  	{TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, 32, 0, 12, ecdheECDSAKA, suiteECDHE | suiteECDSA | suiteTLS12, nil, nil, aeadChaCha20Poly1305},
  	{TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, 16, 0, 4, ecdheRSAKA, suiteECDHE | suiteTLS12, nil, nil, aeadAESGCM},
  	{TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, 16, 0, 4, ecdheECDSAKA, suiteECDHE | suiteECDSA | suiteTLS12, nil, nil, aeadAESGCM},
  	{TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, 32, 0, 4, ecdheRSAKA, suiteECDHE | suiteTLS12 | suiteSHA384, nil, nil, aeadAESGCM},
  	{TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, 32, 0, 4, ecdheECDSAKA, suiteECDHE | suiteECDSA | suiteTLS12 | suiteSHA384, nil, nil, aeadAESGCM},
  	{TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, 16, 32, 16, ecdheRSAKA, suiteECDHE | suiteTLS12 | suiteDefaultOff, cipherAES, macSHA256, nil},
  	{TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, 16, 20, 16, ecdheRSAKA, suiteECDHE, cipherAES, macSHA1, nil},
  	{TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, 16, 32, 16, ecdheECDSAKA, suiteECDHE | suiteECDSA | suiteTLS12 | suiteDefaultOff, cipherAES, macSHA256, nil},
  	{TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, 16, 20, 16, ecdheECDSAKA, suiteECDHE | suiteECDSA, cipherAES, macSHA1, nil},
  	{TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, 32, 20, 16, ecdheRSAKA, suiteECDHE, cipherAES, macSHA1, nil},
  	{TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, 32, 20, 16, ecdheECDSAKA, suiteECDHE | suiteECDSA, cipherAES, macSHA1, nil},
  	{TLS_RSA_WITH_AES_128_GCM_SHA256, 16, 0, 4, rsaKA, suiteTLS12, nil, nil, aeadAESGCM},
  	{TLS_RSA_WITH_AES_256_GCM_SHA384, 32, 0, 4, rsaKA, suiteTLS12 | suiteSHA384, nil, nil, aeadAESGCM},
  	{TLS_RSA_WITH_AES_128_CBC_SHA256, 16, 32, 16, rsaKA, suiteTLS12 | suiteDefaultOff, cipherAES, macSHA256, nil},
  	{TLS_RSA_WITH_AES_128_CBC_SHA, 16, 20, 16, rsaKA, 0, cipherAES, macSHA1, nil},
  	{TLS_RSA_WITH_AES_256_CBC_SHA, 32, 20, 16, rsaKA, 0, cipherAES, macSHA1, nil},
  	{TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, 24, 20, 8, ecdheRSAKA, suiteECDHE, cipher3DES, macSHA1, nil},
  	{TLS_RSA_WITH_3DES_EDE_CBC_SHA, 24, 20, 8, rsaKA, 0, cipher3DES, macSHA1, nil},
  
  	// RC4-based cipher suites are disabled by default.
  	{TLS_RSA_WITH_RC4_128_SHA, 16, 20, 0, rsaKA, suiteDefaultOff, cipherRC4, macSHA1, nil},
  	{TLS_ECDHE_RSA_WITH_RC4_128_SHA, 16, 20, 0, ecdheRSAKA, suiteECDHE | suiteDefaultOff, cipherRC4, macSHA1, nil},
  	{TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, 16, 20, 0, ecdheECDSAKA, suiteECDHE | suiteECDSA | suiteDefaultOff, cipherRC4, macSHA1, nil},
  }

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/2532cbb5-0894-4b17-b347-db55c26be497%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[SG]

Without any special configuration we support TLSv1.1 and TLSv1.2 and the following cipher suites


"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
"TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
"TLS_DHE_DSS_WITH_AES_128_GCM_SHA256",
"TLS_DHE_DSS_WITH_AES_256_GCM_SHA384",
"TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA",
"TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
"TLS_DHE_RSA_WITH_AES_128_CBC_SHA",
"TLS_DHE_DSS_WITH_AES_128_CBC_SHA256",
"TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
"TLS_DHE_DSS_WITH_AES_256_CBC_SHA",
"TLS_DHE_RSA_WITH_AES_256_CBC_SHA"


But this can be overriden with the following settings:

WARNING: Expert setting, do only use if you know what you are doing
# If you set wrong values here this this could be a security risk
searchguard.ssl.transport.enabled_ciphers
searchguard.ssl.transport.enabled_protocols
searchguard.ssl.http.enabled_protocols
searchguard.ssl.http.enabled_ciphers

see https://github.com/floragunncom/search-guard-ssl/blob/master/searchguard-ssl-config-template.yml
and https://github.com/floragunncom/search-guard-ssl/blob/master/src/main/java/com/floragunn/searchguard/ssl/util/SSLConfigConstants.java

> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/2532cbb5-0894-4b17-b347-db55c26be497%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

> To post to this group, send email to search...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/CAEfmcNMz7U3SFiNgtnBMhiYNgTU2KEmt9S92Qv7EV6kQW%2B8c4Q%40mail.gmail.com.