sgadmin initial "initializing" not working

[Benedikt Haug]

Dear Searchguard Developers,

ES version: 6.3.0
Searchguard version: 6.3.0-22.3 (com.floragunn:search-guard-6:6.3.0-22.3)
JVM: 1.8.0_121-b13
Nr. of nodes: 2

Deployed via puppet: https://forge.puppet.com/elastic/elasticsearch

Description of the bug:

sgadmin.sh does not work as expected. Instead of initially "Initializing" the cluster it crashes:

```

usr/share/elasticsearch/plugins/search-guard-6# /usr/share/elasticsearch/plugins/search-guard-6/tools/sgadmin.sh --diagnose -cd /etc/elasticsearch/mes_any_log1/ -cacert /etc/ssl/team/certs/searchguard/admin-elasticsearchanylog-portal-wfe-bs.qa.server.lan.pem -cert /etc/ssl/team/certs/searchguard/admin-elasticsearchanylog-portal-wfe-bs.qa.server.lan.pem  -h mes-any-log-qa002.qa.server.lan -cn mes_any_log
WARNING: JAVA_HOME not set, will use /usr/bin/java
Search Guard Admin v6
Will connect to mes-any-log-qa002.qa.server.lan:9300 ... done
ERR: An unexpected IllegalStateException occured: failed to load plugin class [com.floragunn.searchguard.SearchGuardPlugin]
Trace:
java.lang.IllegalStateException: failed to load plugin class [com.floragunn.searchguard.SearchGuardPlugin]
    at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:701)
    at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:114)
    at org.elasticsearch.client.transport.TransportClient.newPluginService(TransportClient.java:107)
    at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:132)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:269)
    at com.floragunn.searchguard.tools.SearchGuardAdmin$TransportClientImpl.<init>(SearchGuardAdmin.java:887)
    at com.floragunn.searchguard.tools.SearchGuardAdmin.main0(SearchGuardAdmin.java:442)
    at com.floragunn.searchguard.tools.SearchGuardAdmin.main(SearchGuardAdmin.java:124)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:692)
    ... 7 more
Caused by: ElasticsearchException[Empty file path for searchguard.ssl.transport.pemkey_filepath]
    at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.checkPath(DefaultSearchGuardKeyStore.java:701)
    at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.resolve(DefaultSearchGuardKeyStore.java:193)
    at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfig(DefaultSearchGuardKeyStore.java:282)
    at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.<init>(DefaultSearchGuardKeyStore.java:145)
    at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.<init>(SearchGuardSSLPlugin.java:193)
    at com.floragunn.searchguard.SearchGuardPlugin.<init>(SearchGuardPlugin.java:189)
    ... 12 more

```

This certificate is present and elasticsearch starts up as expected.
```/etc/elasticsearch/mes_any_log1# realpath elasticsearchanylog-portal-wfe-bs.qa.server.lan.pem
/etc/elasticsearch/mes_any_log1/elasticsearchanylog-portal-wfe-bs.qa.server.lan.pem
```
The file is declared relatively in the config but declaring it absolutely (referencing the whole path does not change this behaviour)
```# grep -r searchguard.ssl.transport.pemkey_filepath /etc/elasticsearch/mes_any_log1/
/etc/elasticsearch/mes_any_log1/elasticsearch.yml:searchguard.ssl.transport.pemkey_filepath: elasticsearchanylog-portal-wfe-bs.qa.server.lan.pem
```

It does not write any diagnose files and the documention lists no more things to try.

[Benedikt Haug]

Also tried placing the certificates in different paths and changing the directory the script runs in. The result does not change.

[Jochen Kressin]

Your sgadmin call is just lacking the private key for your certificate:

Empty file path for searchguard.ssl.transport.pemkey_filepath

You need to add:

-key <file>  Path to the key of admin certificate
-keypass <password> Password of the key of admin certificate (optional)