Search guard config

Sundeep

unread,

Dec 11, 2018, 7:52:10 PM12/11/18

to Search Guard Community Forum

When asking questions, please provide the following information:

* Search Guard and Elasticsearch version

* Installed and used enterprise modules, if any

* JVM version and operating system version

* Search Guard configuration files

* Elasticsearch log messages on debug level

* Other installed Elasticsearch or Kibana plugins, if any

Search Guard: 6.2.3-23

Elastic: 6.2.3

Java: 1.8.0

We are trying to use search guard with elastic search and would need some guidance/help for the following issues.

-Can we use self-signed certificates for transport layer and if we do that what should be the value of `searchguard.ssl.transport.pemtrustedcas_filepath`, search guard does not start if is unset or set to the same cert, because it is self signed.

Caused by: java.lang.IllegalArgumentException: File does not contain valid private key: Xxxxx.pem

at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:267) ~[?:?]

at io.netty.handler.ssl.SslContextBuilder.forServer(SslContextBuilder.java:90) ~[?:?]

at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.buildSSLServerContext(DefaultSearchGuardKeyStore.java:613) ~[?:?]

at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfig(DefaultSearchGuardKeyStore.java:405) ~[?:?]

at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.<init>(DefaultSearchGuardKeyStore.java:145) ~[?:?]

at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.<init>(SearchGuardSSLPlugin.java:193) ~[?:?]

at com.floragunn.searchguard.SearchGuardPlugin.<init>(SearchGuardPlugin.java:197) ~[?:?]

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]

at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_181-b13]

at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:554) ~[elasticsearch-6.2.3.jar:6.2.3]

at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:505) ~[elasticsearch-6.2.3.jar:6.2.3]

at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:422) ~[elasticsearch-6.2.3.jar:6.2.3]

at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:146) ~[elasticsearch-6.2.3.jar:6.2.3]

at org.elasticsearch.node.Node.<init>(Node.java:303) ~[elasticsearch-6.2.3.jar:6.2.3]

at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.3.jar:6.2.3]

at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.3.jar:6.2.3]

at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.3.jar:6.2.3]

at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.3.jar:6.2.3]

at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.3.jar:6.2.3]

... 6 more

Caused by: java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag = 48)

at sun.security.util.ObjectIdentifier.<init>(ObjectIdentifier.java:257) ~[?:?]

at sun.security.util.DerInputStream.getOID(DerInputStream.java:314) ~[?:?]

at com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267) ~[sunjce_provider.jar:1.8.0_181]

at java.security.AlgorithmParameters.init(AlgorithmParameters.java:293) ~[?:1.8.0_181-b13]

at sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:132) ~[?:?]

at sun.security.x509.AlgorithmId.<init>(AlgorithmId.java:114) ~[?:?]

at sun.security.x509.AlgorithmId.parse(AlgorithmId.java:372) ~[?:?]

at javax.crypto.EncryptedPrivateKeyInfo.<init>(EncryptedPrivateKeyInfo.java:95) ~[?:1.8.0_181]

at io.netty.handler.ssl.SslContext.generateKeySpec(SslContext.java:978) ~[?:?]

-We want to only use internal_users and wanted to know how we can change password after setup is complete. In the sense if we change sg_internal_users.yml with the new password, elastic server restart does not reflect the change, the only way is to change index.

admin:

hash: $2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG

roles:

- admin

attributes:

Is there a way to achieve this without running sgadmin? Does sgadmin need a running instance of elastic server to make this change?

-Fresh install of search guard and I see the following error

Exception in thread "main" java.lang.NoClassDefFoundError: org/elasticsearch/client/transport/NoNodeAvailableException

at java.lang.Class.getDeclaredMethods0(Native Method)

at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)

at java.lang.Class.privateGetMethodRecursive(Class.java:3048)

at java.lang.Class.getMethod0(Class.java:3018)

at java.lang.Class.getMethod(Class.java:1784)

at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)

at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)

Caused by: java.lang.ClassNotFoundException: org.elasticsearch.client.transport.NoNodeAvailableException

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

Thanks

pat...@amatc.com.au

unread,

Dec 11, 2018, 8:07:01 PM12/11/18

to Search Guard Community Forum

The chain including root used to sign the cert, sgadming wont run if node not running.

I just did this myself, you also need to add the root cert to php.ini config so curl will work.

Sundeep

unread,

Dec 11, 2018, 8:16:16 PM12/11/18

to Search Guard Community Forum

What is the root when it is self signed? I just have the cert file and its private key.

pat...@amatc.com.au

unread,

Dec 11, 2018, 8:17:55 PM12/11/18

to Search Guard Community Forum

Did you sign it yourself ?

On Wednesday, December 12, 2018 at 11:52:10 AM UTC+11, Sundeep wrote:

Sundeep

unread,

Dec 11, 2018, 8:23:10 PM12/11/18

to Search Guard Community Forum

Yes.. This is the command we ran

openssl req -new -newkey rsa:2048 -x509

-days 365

-subj /CN=ElasticSearch

-keyout elasticserver.key

-out elasticserver.crt

-config apache\conf\openssl.cnf

-passout keyPass.txt

pat...@amatc.com.au

unread,

Dec 11, 2018, 8:30:26 PM12/11/18

to Search Guard Community Forum

Hi,

Use this put localhost if just one server, works for me dont use pem use keystore config at the bottom, works out of the box.

https://search-guard.com/tls-certificate-generator/

On Wednesday, December 12, 2018 at 11:52:10 AM UTC+11, Sundeep wrote:

Search Guard

unread,

Dec 12, 2018, 12:38:18 AM12/12/18

to Search Guard Community Forum

Can you try https://docs.search-guard.com/latest/offline-tls-tool#tls-tool ?

Sundeep

unread,

Dec 12, 2018, 3:19:12 PM12/12/18

to Search Guard Community Forum

I used the tool to generate only root, node certificates and added the necessary config to elastic search, but when I run sgadmin I get the following error. Does sgadmin needs the new root and node cert at a particular location for it to work?

My elastic search node works fine..

> plugins\search-guard-6\tools\sgadmin.bat -cert config\kirk.pem -cacert config\root-ca.pem -nhnv -icl -key config\kirk-key.pem -cd plugins\search-guard-6\sgconfig -p 8330

Search Guard Admin v6

WARNING: Seems you want connect to the Elasticsearch HTTP port.

sgadmin connects on the transport port which is normally 9300.

Will connect to 127.0.0.1:8330 ... done

Unable to check whether cluster is sane: None of the configured nodes are available: [{#transport#-1}{qGdkaU6ZQ9WpQz4tQYQCEQ}{127.0.0.1}{127.0.0.1:8330}]

12:14:35.765 [elasticsearch[_client_][transport_client_boss][T#1]] ERROR com.floragunn.searchguard.ssl.transport.SearchGuardSSLNettyTransport - SSL Problem General SSLEngine problem

javax.net.ssl.SSLHandshakeException: General SSLEngine problem

at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521) ~[?:1.8.0_181-b13]

at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528) ~[?:1.8.0_181-b13]

at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:802) ~[?:1.8.0_181-b13]

at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766) ~[?:1.8.0_181-b13]

at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_181-b13]

at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:281) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1215) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1127) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1162) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.16.Final.jar:4.1.16.Final]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181-b13]

Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem

at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:1.8.0_181-b13]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709) ~[?:1.8.0_181-b13]

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318) ~[?:1.8.0_181-b13]

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310) ~[?:1.8.0_181-b13]

at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1620) ~[?:1.8.0_181-b13]

at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) ~[?:1.8.0_181-b13]

at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) ~[?:1.8.0_181-b13]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:970) ~[?:1.8.0_181-b13]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:967) ~[?:1.8.0_181-b13]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_181-b13]

at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459) ~[?:1.8.0_181-b13]

at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1364) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1272) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

... 19 more

Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) ~[?:1.8.0_181-b13]

at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_181-b13]

at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_181-b13]

at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) ~[?:1.8.0_181-b13]

at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281) ~[?:1.8.0_181-b13]

at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136) ~[?:1.8.0_181-b13]

at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1607) ~[?:1.8.0_181-b13]