Improving installation & configuration

Jochen Kressin

unread,

Jul 4, 2016, 6:08:10 PM7/4/16

to Search Guard

Hi folks,

Jochen from floragunn here. As you probably know, we have released Search Guard two weeks ago, and received a lot of feedback since then. Thanks all for your input!

After reading a lot of mails, posts on Google Groups etc., we're wondering: What can we do to improve the overall experience with Search Guard? It seems that installing and configuring Search Guard still is a little bit cumbersome. So, the question is: With which part of the installation and configuration process did you struggle most? Is the documentation unclear or misleading? And what do you miss in order to get started with Search Guard quickly?

At the moment we're thinking about:

Adding scripts to quickly generate and also install all required TLS certificates. Of course, the certificates would be self-signed-demo-certs only, but it would safe you from copying files around manually

something like install_demo_certificates.sh on the command line

Implementing a more verbose configuration check. At the moment, some configuration errors (sometimes as simple as typos) are not properly reported by SG

something similar to Apaches configtest

GUI for configuring users, roles and permissions

however, this will require some time to implement. Contributions are always welcome ;)

Improving docs on how to integrate SG with other ES plugins and products (Marvel, Watcher, Elastalert etc.)
What else?

As always, any feedback is highly appreciated. You can answer here, or drop me an email: jkre...@floragunn.com

And while I'm at it, please also check out our Search Guard bundle: An ES installation with SG pre-installed and -configured, to help setting up a PoC quickly:

https://github.com/floragunncom/search-guard/wiki/Search-Guard-Bundle

Thanks,

Jochen

Ajay B

unread,

Jul 18, 2016, 10:05:46 AM7/18/16

to Search Guard

I would like setup and configuration process be documented with following details (based on my attempts so far to get it going first time):

1. Sequence and order of installation steps:
a. Install base essential pre-requisites : Elasticsearch, Openssl
b. Install steps for SG plugins

2. Creating different types of SSL certificates. Not all Elasticsearch admins may be managing SSLs themselves, therefore for POC purposes, script should generate all types of certificates including admin certs to be used from ES nodes and remote nodes and specific files and location where they should be copied and there ownership requirements root or elasticsearch user.

3.. Configuration process order:
a. When to add SG related configuration to elasticsearch.yml ? i.e. before running sgadmin.sh or after sgadmin run. Whether all required configuration be added at same time or gradually after certain step has been completed successfully?
b. To run sgadmin first time, how to create admin certs, where to copy cert files, file ownership and naming rules/conventions .
c. Examples of creating admin cert , running sgadmin from one of Elasticsearch node and running sgadmin from a Non-Elasticsearch node with access to ES cluster.
Example should highlight the items that need to be customized for individual installation.

4. Configuration required for Logstash and Kibana. Do they need separate SSL certificates ?

5. Authentication examples using other means like LDAP, Kerberos etc,

6. Examples scripts provided are not same when downloaded from in the Bundle and from GIT. This makes difficult to figure out the issue is it the script differences or some additional info required to be passed on to run the scripts. It is creating confusion what is correct script to use and what are the reasons for differences.

For example: lines from bundled example.sh:
./gen_root_ca.sh "$ORG_NAME" "ca pass" "changeit"
./gen_node_cert.sh "$ORG_NAME" "CN=node-0.example.com,OU=SSL,O=Test,L=Test,C=DE" localhost changeit "ca pass" \
&& ./gen_node_cert.sh "$ORG_NAME" "CN=node-1.example.com,OU=SSL,O=Test,L=Test,C=DE" localhost changeit "ca pass" \
&& ./gen_node_cert.sh "$ORG_NAME" "CN=node-2.example.com,OU=SSL,O=Test,L=Test,C=DE" localhost changeit "ca pass"
./gen_client_node_cert.sh "$ORG_NAME" "CN=spock,OU=client,O=client,L=Test,C=DE" changeit "ca pass"
./gen_client_node_cert.sh "$ORG_NAME" "CN=kirk,OU=client,O=client,L=Test,C=DE" changeit "ca pass"

Lines from GIT repo master example.sh:
./gen_root_ca.sh capass changeit
./gen_node_cert.sh 0 changeit capass && ./gen_node_cert.sh 1 changeit capass && ./gen_node_cert.sh 2 changeit capass
./gen_client_node_cert.sh spock changeit capass
./gen_client_node_cert.sh kirk changeit capas

I am willing to volunteer to create and share bash scripts from detailed individual steps to deploy in stand alone ES 2.3.3 cluster which can be managed locally from ES nodes themselves or from a remote admin node.

Thanks
Ajay

Markus Korn

unread,

Jul 26, 2016, 9:27:10 AM7/26/16

to Search Guard

Hi Jochen,

Hi floragunn team,

first of all, thank you very much for search-guard and search-guard-ssl - for me, both tools are a great and welcome addition to the ELK stacks I'm maintaining, and they also add the layer of security on top of elasticsearch I was looking for. Plus it gives me an alternative to shield, and alternatives are always great ;)

For me the installation process in general is really okay, not a big deal, and close to perfect. I dont need a GUI, I dont need tools, there are well defined and documented set of configuration options for both products, which are working really well in my scenarios.

There is one big BUT, unfortunately: all my stacks are living in docker world, I've an image of elasticsearch with searchguard, and search-guard-ssl (which I would love to publish once everything is working perfectly) The thing is, the requirement to run a script (`sgadmin.sh`), which requires a running elasticsearch instance does not really fit into the concept of docker. there should never be a running ES instance saying "SG is not initialized", auth should be a first class thingy in ES, so if sg is not initialized ES should not start at all.

So for me it would be really great if there was a way to initialize sg without having a run elasticsearch. Even if that means that auth would in this case become a per instance feature (rather than a feature of the cluster (by storing the config in an index) it would be perfectly fine for me. Same for instant config updates: my config can be considered as static, and if there is a config change restarting ES is fine for me as well. IMHO auth does not really belong to the cluster anyways, it is a feature of any ES instance itself.

Again, thanks for your great work. Looking forward to see if search-guard can be improved in a way that it works even better in docker environments.

Have a nice day,

Markus

Message has been deleted

rocky

unread,

Jul 27, 2016, 4:29:34 PM7/27/16

to Search Guard

Hi Jochen,

I followed same steps as per link
https://github.com/floragunncom/search-guard/wiki/Search-Guard-Bundle.
When I try to start elasticsearch I get below error message
"service elasticsearch start
Starting elasticsearch: Exception in thread "main" ElasticsearchException[Failed to load plugin class [com.floragunn.searchguard.SearchGuardPlugin]]; nested: InvocationTargetException; nested: IllegalStateException[SearchGuardSSLPlugin must be be installed];
Likely root cause: java.lang.IllegalStateException: SearchGuardSSLPlugin must be be installed
at com.floragunn.searchguard.SearchGuardPlugin.checkSSLPluginAvailable(SearchGuardPlugin.java:135)
at com.floragunn.searchguard.SearchGuardPlugin.(SearchGuardPlugin.java:53)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:472)
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:432)
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:129)
at org.elasticsearch.node.Node.(Node.java:158)
at org.elasticsearch.node.Node.(Node.java:140)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details."
When checked in logs I see below
[2016-07-27 14:27:49,501][WARN ][bootstrap ] unable to install syscall filter: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed
[2016-07-27 14:27:49,594][INFO ][node ] [node-1] version[2.3.3], pid[28517], build[218bdf1/2016-05-17T15:40:04Z]
[2016-07-27 14:27:49,594][INFO ][node ] [node-1] initializing ...
[2016-07-27 14:27:49,920][ERROR][bootstrap ] Exception
ElasticsearchException[Failed to load plugin class [com.floragunn.searchguard.SearchGuardPlugin]]; nested: InvocationTargetException; nested: IllegalStateException[SearchGuardSSLPlugin must be be installed];
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:483)
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:432)
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:129)
at org.elasticsearch.node.Node.(Node.java:158)
at org.elasticsearch.node.Node.(Node.java:140)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:472)
... 8 more
Caused by: java.lang.IllegalStateException: SearchGuardSSLPlugin must be be installed
at com.floragunn.searchguard.SearchGuardPlugin.checkSSLPluginAvailable(SearchGuardPlugin.java:135)
at com.floragunn.searchguard.SearchGuardPlugin.(SearchGuardPlugin.java:53)
... 13 more
Can anyone help me in fixing this issue?

ES Yaml file is as below Should I add any open ssl certficate of server?

searchguard.ssl.transport.enabled: true

searchguard.ssl.transport.keystore_filepath: CN=localhost-keystore.jks

searchguard.ssl.transport.keystore_password: XXXX

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.truststore_password: XXX

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.keystore_filepath: CN=localhost-keystore.jks

searchguard.ssl.http.keystore_password: XXX

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.ssl.http.truststore_password: XXX

searchguard.kerberos.krb5_filepath: /Users/temp/kerberos_ldap_environment/krb5.conf

searchguard.kerberos.acceptor_keytab_filepath: http_srv.keytab

searchguard.audit.type: internal_elasticsearch

searchguard.authcz.admin_dn:

- CN=sgadmin

SG

unread,

Jul 27, 2016, 5:18:34 PM7/27/16

to search...@googlegroups.com

You must not execute "service elasticsearch start".

Instead execute "./elasticsearch-2.3.3-localhost/bin/elasticsearch" as describe here https://github.com/floragunncom/search-guard/wiki/Search-Guard-Bundle

> --
> You received this message because you are subscribed to the Google Groups "Search Guard" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/25f28f49-1bb5-4a7b-9147-90540b4df7c3%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

rocky

unread,

Jul 28, 2016, 1:48:32 AM7/28/16

to Search Guard

Tried it and I get below error. Initially I tried as root it failed and later I tried with different user and I still get this error.

Can you please help me in getting this fixed?

log4j:ERROR setFile(null,true) call failed.

java.io.FileNotFoundException: /usr/share/elasticsearch/bin/elasticsearch-2.3.3-localhost/logs/elasticsearch.log (Permission denied)

at java.io.FileOutputStream.open0(Native Method)

at java.io.FileOutputStream.open(FileOutputStream.java:270)

at java.io.FileOutputStream.<init>(FileOutputStream.java:213)

at java.io.FileOutputStream.<init>(FileOutputStream.java:133)

at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)

at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)

at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)

at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)

at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)

at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)

at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)

at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)

at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)

at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)

at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)

at org.elasticsearch.common.logging.log4j.LogConfigurator.configure(LogConfigurator.java:128)

at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:243)

at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)

log4j:ERROR Either File or DatePattern options are not set for appender [file].

log4j:ERROR setFile(null,true) call failed.

java.io.FileNotFoundException: /usr/share/elasticsearch/bin/elasticsearch-2.3.3-localhost/logs/elasticsearch_deprecation.log (Permission denied)