Empty file path for searchguard.ssl.transport.truststore_filepath

[iv...@uchicago.edu]

Hello,

I have just started playing with Search Guard and got stuck configuring LDAP.

First I installed the demo. It worked OK except I do not see "Tenants" tab on the left as the document says. See the screenshot. Is it a problem? I do want each user to be able to store his Kibana dashboards both privately and publicly.

Next I tried to enable LDAP. When I try to connect to ES from a web browser using userid/password from LDAP, I am getting in ES logs:

====

Unable to connect to ldapserver 128.135.112.111:389 due  
to ElasticsearchException[Empty file path for searchguard.ssl.transport.truststore_filepath]

====

How to fix it?

Do I have to use truststore? Currently I just have whatever certificates were generated by demo.

ldapsearch -x -v -h 128.135.112.111 -p 389 -b "dc=rcc,dc=uchicago,dc=edu"

from the host I am running ES on works fine.

Once I'll manage to configure LDAP, besides Kibana, I want to be able to use ES from python.

What options do I need to specify to

===

from elasticsearch import Elasticsearch
es = Elasticsearch()
===

to connect to the protected ES when running on the same host where Kibana and ES are running from inside user account known to LDAP?

Thank you,

Igor

=====================

When asking questions, please provide the following information:

* Search Guard and Elasticsearch version

6.2.1 

* Installed and used enterprise modules, if any

Elasticsearch, Kibana, Search Guard

* JVM version and operating system version

1.8.0_121

CentOS 7.2

* Search Guard configuration files

Attached

* Elasticsearch log messages on debug level

Attached

* Other installed Elasticsearch or Kibana plugins, if any

None

=======================

[Jochen Kressin]

From the configs and screeshots and LDAP config you sent I see that you enable TLS and also want to verify hostnames:

enable_ssl: true

verify_hostnames: true

This means that the connection to LDAP will be TLS encrypted and that Search Guard needs to verify the TLS certificate that the LDAP server sends against a Root CA / trusted CA. So in order to use TLS, you need to configure the trusted CAs in the LDAP module. This would be the root and/or signing certificate of the certificate you installed on the LDAP server.

However, in the LDAP config you specify port 389 for the LDAP connection, which is usually used for plain / unencrypted connections. The port for TLS would be 636. Are you sure your LDAP server listens in port 389 for TLS connections? What happens if you configure:

enable_ssl: false

verify_hostnames: false

[iv...@uchicago.edu]

Hi Jochen,

After following your recommendations I am now getting a different kind of error when trying to use my LDAP credentials to log into ES:

=========

No password given for bind_dn cn=admin,dc=rcc,dc=uchica
go,dc=edu. Will try to authenticate anonymously to ldap
[2018-02-19T11:20:13,156][WARN ][c.f.s.a.BackendRegistry  ] Authentication finally failed for ivy2
=========

Does one normally have to use something like a password or certificate to talk to LDAP? What bind_dn is usually given?

Thank you,

Igor

[Jochen Kressin]

Yes, usually you need a user (in LDAP terminology: bind_dn) and password to talk to an LDAP server. But this depends on how your LDAP is configured actually: If it allows anonymous (usually read-only) access then you can skip the bind_dn and password section if the sg_config LDAP configuration. Search Guard will then try to access the server without bind_dn and password. But if your LDAP server requires authentication, then you need to add the bind_dn and password. This depends on what server you user and how it is configured.

[Igor Yakushin]

On Mon, Feb 19, 2018 at 2:44 PM, Jochen Kressin <jkre...@floragunn.com> wrote:

Yes, usually you need a user (in LDAP terminology: bind_dn) and password to talk to an LDAP server. But this depends on how your LDAP is configured actually: If it allows anonymous (usually read-only) access then you can skip the bind_dn and password section if the sg_config LDAP configuration. Search Guard will then try to access the server without bind_dn and password. But if your LDAP server requires authentication, then you need to add the bind_dn and password. This depends on what server you user and how it is configured.

Is there a way to figure out how my server is configured by using for example 'ldapsearch' or some other tool?

I looked into a working java program that creates a web page where users can type their credentials and get access via LDAP to our system and at first glance I do not see that it is using any password to query LDAP server (except for the credentials of the user that tries to login):

==========

            environment.put(javax.naming.Context.PROVIDER_URL, "ldaps://ldap.uchicago.edu");

            environment.put(javax.naming.Context.SECURITY_AUTHENTICATION, "simple");

            environment.put(javax.naming.Context.SECURITY_PRINCIPAL, usernameFormatted);

            environment.put(javax.naming.Context.SECURITY_CREDENTIALS, password);

            DirContext ctx = new InitialDirContext(environment);

            SearchControls controls = new SearchControls();

            controls.setSearchScope(SearchControls.SUBTREE_SCOPE);

            NamingEnumeration<SearchResult> results = ctx.search("ou=people,dc=uchicago,dc=edu", "uid="+username, controls);

==========

Here is also a working php code

==========

<?php
       $ldap_dn = "uid=".$_POST["username"].",dc=uchicago,dc=edu";
       $ldap_password = $_POST["password"];
       $ldap_con = ldap_connect("ldap.uchicago.edu","636");
       ldap_set_option($ldap_con, LDAP_OPT_PROTOCOL_VERSION, 3);

       if(@ldap_bind($ldap_con,$ldap_dn,$ldap_password))
               echo "Authenticated";
       else
               echo "Invalid Credential";
?>

==========

Apparently I should be using ldap.uchicago.edu:636. That probably means that I should use 

enable_ssl_client_auth: true
?

Can you tell me from above what bind_dn, userbase I should use?

I tried the following settings for both authc and authz but still no luck:

==========

                    enable_ssl_client_auth: true

          hosts:
             - ldap.uchicago.edu:636
           bind_dn: 'dc=uchicago,dc=edu'
           #password: password
           userbase: 'ou=people,dc=uchicago,dc=edu'

==========

LDAP response read timed out, timeout used:5000ms., providerException=javax.naming.
NamingException: LDAP response read timed out, timeout used:5000ms.]. Try next.
[2018-02-19T15:57:52,005][WARN ][c.f.s.a.BackendRegistry  ] Authentication finally failed for ivy2

==========

--
You received this message because you are subscribed to a topic in the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/jVjrVCh0yLY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/8d481577-a50a-4343-aaa2-9af7c39f4ed8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Igor Yakushin, Ph.D.

Computational Scientist

Kavli Institute for Cosmological Physics, ERC #413

Research Computing Center, room #2

The University of Chicago

773-834-5313

https://rcc.uchicago.edu

[SG]

you can just omit the bind_dn and password, then a LDAP anonymous bind will be performed which seems the way to go in your setup

If this is still not working than pls enable debug logging and post (or mail) the complete logfile. Pls refer to http://docs.search-guard.com/latest/troubleshooting-tls#setting-the-log-level-to-debug

> To unsubscribe from this group and all its topics, send an email to search-guard...@googlegroups.com.

> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/8d481577-a50a-4343-aaa2-9af7c39f4ed8%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Igor Yakushin, Ph.D.
> Computational Scientist
> Kavli Institute for Cosmological Physics, ERC #413
> Research Computing Center, room #2
> The University of Chicago
> 773-834-5313
> https://rcc.uchicago.edu
>
>

> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

> To post to this group, send email to search...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/CALQ0CdWtMVG1cC328x3Fdu3NHtvKBb2DQRLxgh3jjrkUSXSrsA%40mail.gmail.com.

[iv...@uchicago.edu]

On Tuesday, February 27, 2018 at 11:49:09 AM UTC-6, Search Guard wrote:

you can just omit the bind_dn and password, then a LDAP anonymous bind will be performed which seems the way to go in your setup

If this is still not working than pls enable debug logging and post (or mail) the complete logfile. Pls refer to http://docs.search-guard.com/latest/troubleshooting-tls#setting-the-log-level-to-debug

Hello,

I am still stuck with LDAP. The log file is attached.

Thank you,

Igor

[SG]

pls post your complete sg_config.yml (beware of sensitive information like passwords!)

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/4ebf3dfe-f450-4a0e-8bc4-959c4e367301%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

> <log.txt>

[iv...@uchicago.edu]

On Tuesday, March 13, 2018 at 3:56:07 PM UTC-5, Search Guard wrote:

pls post your complete sg_config.yml (beware of sensitive information like passwords!)

Attached.

[SG]

enable_ssl_client_auth: true --> but no client certificates denoted! So i think you should set this to false. This has nothing to do with port 636 or 389 !!

And if you use enable_start_tls than normally the port is 389, so i think you can remove the 636 entries (but configuration maybe different for your servers)
On the other hand you can set enable_start_tls to false and set enable_ssl to true and omit all the 389 entries

From the example code you mentioned earlier i suggest you try:

enable_ssl: true
enable_start_tls: false
enable_ssl_client_auth: false
verify_hostnames: false
hosts:
- ldap.rcc.uchicago.edu:636
- ldap.uchicago.edu:636

If that is working then try setting verify_hostnames to true (because false is insecure)

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/6f62d946-09ff-4512-867d-930551947e39%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

> <elasticsearch.yml><sg_config.yml>

[iv...@uchicago.edu]

Still the same problems.

The logs and config files are attached.

How does one usually authenticate to LDAP server?

[SG]

pls add this to the sg_config.yml (same indent level as enable_ssl)

pemtrustedcas_filepath: HadoopCertificates/incommon.crt

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/14d553e7-4c21-47da-9a4c-eb30fb41eb61%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

> <log.txt><sg_config.yml>

[Igor Yakushin]

On Tue, Mar 13, 2018 at 5:54 PM, SG <in...@search-guard.com> wrote:

pls add this to the sg_config.yml (same indent level as enable_ssl)

pemtrustedcas_filepath: HadoopCertificates/incommon.crt

The error changed but still no luck. The files are attached.

Is the user from which account Elastic Search is running supposed to be known to LDAP? I created a local elasticsearch account that LDAP does not know about. Does it matter?

If I can run ldapsearch from this account successfully:

ldapsearch -x -v -h ldap.rcc.uchicago.edu -p 389 -b "uid=ivy2,ou=People,dc=rcc,dc=uchicago,dc=edu"

does it tell anything about SearchGuard's ability to use LDAP or this is not related?

[SG]

If ldapsearch -x -v -h ldap.rcc.uchicago.edu -p 389 -b "uid=ivy2,ou=People,dc=rcc,dc=uchicago,dc=edu" works then you can also try


enable_ssl: false

enable_start_tls: false
enable_ssl_client_auth: false
verify_hostnames: false
hosts:

- ldap.rcc.uchicago.edu:389
- ldap.uchicago.edu:389

> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/CALQ0CdW9ha_0AWe4ikAfunbO_eYCZv26D7J_2bbZai%3DmJD_PRA%40mail.gmail.com.

[iv...@uchicago.edu]

On Wednesday, March 14, 2018 at 6:44:38 AM UTC-5, Search Guard wrote:

If ldapsearch -x -v -h ldap.rcc.uchicago.edu -p 389 -b "uid=ivy2,ou=People,dc=rcc,dc=uchicago,dc=edu" works then you can also try


enable_ssl: false
enable_start_tls: false
enable_ssl_client_auth: false
verify_hostnames: false
hosts:
  - ldap.rcc.uchicago.edu:389
  - ldap.uchicago.edu:389

It still does not work. The logs and the configuration file is attached.

I have root access to ldap.rcc.uchicago.edu. Is there some log that would tell if my request reaches LDAP server and if it does, why the user is not found?

At first glance, I did not find such a log.

[Search Guard]

nothing attached

On Wednesday, 14 March 2018 16:58:53 UTC+1

[iv...@uchicago.edu]

On Wednesday, March 14, 2018 at 11:02:43 AM UTC-5, Search Guard wrote:

nothing attached

Oh, sorry, fixed.

How important are permissions on certificate files? Originally there were complaints about insecure permissions. I fixed that.

Also, to double check how changes from sg_config.yml propagate into Elastic Search: once I change this file, I run:

===

/home_local/elasticsearch/elasticsearch-6.2.1/plugins/search-guard-6/tools/sgadmin.sh -cd /home_local/elasticsearch/ela
sticsearch-6.2.1/plugins/search-guard-6/sgconfig -icl -key /home_local/elasticsearch/elasticsearch-6.2.1/config/HadoopC
ertificates/hadoop.rcc.uchicago.edu-key.pem -cert /home_local/elasticsearch/elasticsearch-6.2.1/config/HadoopCertificat
es/hadoop.rcc.uchicago.edu-chain.crt -cacert /home_local/elasticsearch/elasticsearch-6.2.1/config/HadoopCertificates/in
common.crt -nhnv

===

from tools directory and get messages that the changes were successful (provided that elastic search is running).

As far as I understand, elasticsearch does not have to be restarted after such changes?

[Search Guard]

connection seems ok but the user is not found.

you need to set these props correctly:

userbase:

username_attribute:

usersearch:

see https://docs.search-guard.com/latest/active-directory-ldap-authentication#active-directory-and-ldap-authentication

On Wednesday, 14 March 2018 17:10:50 UTC+1,

[iv...@uchicago.edu]

On Wednesday, March 14, 2018 at 11:34:06 AM UTC-5, Search Guard wrote:

connection seems ok but the user is not found.

you need to set these props correctly:

userbase:

username_attribute:

usersearch:

see https://docs.search-guard.com/latest/active-directory-ldap-authentication#active-directory-and-ldap-authentication

It does not really explain to me how to find out those strings.

Based on the fact that ldapsearch works:

ldapsearch -x -v -h ldap.rcc.uchicago.edu -p 389 -b "uid=ivy2,ou=People,dc=rcc,dc

=uchicago,dc=edu"

I tried the following:

authentication_backend:

bind_dn: 'ou=People,dc=rcc,dc=uchicago,dc=edu'

userbase: 'ou=People,dc=rcc,dc=uchicago,dc=edu'

usersearch: '(sAMAccountName={0})'

username_attribute: uid

authz:

bind_dn: 'ou=People,dc=rcc,dc=uchicago,dc=edu'

rolebase: 'ou=People,dc=rcc,dc=uchicago,dc=edu'

rolesearch: '(member={0})'

userbase: 'ou=People,dc=rcc,dc=uchicago,dc=edu'

usersearch: '(uid={0})'

[iv...@uchicago.edu]

I think, I am getting somewhere after changing usersearch.

Now in the web browser I am getting an error:

=====================

{"error":{"root_cause":[{"type":"security_exception","reason":"no permissions for [cluster:monitor/main] and User [name=ivy2, roles=[], requestedTenant=null]"}],"type":"security_exception","reason":"no permissions for [cluster:monitor/main] and User [name=ivy2, roles=[], requestedTenant=null]"},"status":403}

=====================

Does that mean that I finally got through LDAP but I need to configure the role somehow?

The logs and configuration are attached.

[SG]

yes, authentication succeeded! now its about the roles ...

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/81c285c6-6d66-4217-9752-7516288aa10d%40googlegroups.com.

[iv...@uchicago.edu]

On Wednesday, March 14, 2018 at 4:00:04 PM UTC-5, Search Guard wrote:

yes, authentication succeeded! now its about the roles ...

The roles have nothing to do with LDAP but are internal to SearchGuard as far as I understand?

[Igor Yakushin]

On Tue, Mar 13, 2018 at 4:42 PM, SG <in...@search-guard.com> wrote:

If that is working then try setting verify_hostnames to true (because false is insecure)

If use multiple nodes in Elastic Search cluster and do

verify_hostnames: true

does it mean that I need different certificates on different hosts or can I reuse the same certificates?

The complication is that the rest of the nodes are on internal network and are not visible to the outside world and our university IT people would only generate certificates for hosts with public ip addresses as far as I know.

Another question: now that I have LDAP working, how can I use pyelasticsearch to connect to the cluster without hardcoding my password?

One can obviously provide a password in a plain text in the code but that is not a good idea:

from pyelasticsearch import ElasticSearch

es = ElasticSearch('http://localhost:9200/', username='', password='')

 

> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/6f62d946-09ff-4512-867d-930551947e39%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <elasticsearch.yml><sg_config.yml>

--
You received this message because you are subscribed to a topic in the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/jVjrVCh0yLY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/CB47BA77-DD0A-418A-9E01-8D259D9EB908%40search-guard.com.

For more options, visit https://groups.google.com/d/optout.

[iv...@uchicago.edu]

And another question: how to secure Kibana?

In the demo kibanaserver user is created that uses password given in plain text in Kibana configuration file. I changed that password both on elasticsearch and kibana sides but it is still quite insecure. Do I understand correctly that I can instead use the same certificates as I used for LDAP to get rid of password authentication (or at least not to have it in a plain text format) and enable SSL?

I found a way in elasticsearch-py to specify username and password in plain text when connecting to elasticsearch server. Is there a way to avoid hardcoding passwords in python program?

[iv...@uchicago.edu]

On Thursday, March 15, 2018 at 1:31:37 PM UTC-5, iv...@uchicago.edu wrote:

And another question: how to secure Kibana?

In the demo kibanaserver user is created that uses password given in plain text in Kibana configuration file. I changed that password both on elasticsearch and kibana sides but it is still quite insecure. Do I understand correctly that I can instead use the same certificates as I used for LDAP to get rid of password authentication (or at least not to have it in a plain text format) and enable SSL?

I enabled SSL and got rid of kibanaserver password in the configuration file. 

At first glance Kibana is working. However, SearchGuard tab disappeared when I log as admin. How to get it back? Is admin now treated for some reason as a regular user account?

The kibana configuration and log files are attached. 

In the log there were complaints about cookie password. I set it but what is it for? Is it necessary to specify it?