sgadmin fails while enable shards allocation

[Vijaya Krishna]

I have installed Elasticsearch 6.5.4 on 2 nodes.

Search Guard-6 installed on both nodes.

Generated certificates using SG offline TLS tool and copied certs to both the nodes.

Error:

sgadmin.sh --enable-shard-allocation -key /etc/elasticsearch/ssl/admin.key -cert /etc/elasticsearch/ssl/admin.pem -cacert /etc/elasticsearch/ssl/root-ca.pem -icl -nhnv -h elastic70.example.net

WARNING: JAVA_HOME not set, will use /bin/java

Search Guard Admin v6

Will connect to elastic70.example.net:9300 ... done

Unable to check whether cluster is sane: None of the configured nodes are available: [{#transport#-1}{FFTFfLosSFqu9lkrx0uqUA}{elastic70.example.net}{10.10.10.10:9300}]

ERR: Cannot connect to Elasticsearch. Please refer to elasticsearch logfile for more information

Trace:

NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{FFTFfLosSFqu9lkrx0uqUA}{elastic70.example.net}{10.10.10.10:9300}]]

at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:349)

at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247)

at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)

at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:382)

at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:395)

at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:384)

at com.floragunn.searchguard.tools.SearchGuardAdmin.main0(SearchGuardAdmin.java:454)

at com.floragunn.searchguard.tools.SearchGuardAdmin.main(SearchGuardAdmin.java:123)

/etc/elasticsearch/elasticsearch.yml

#action.destructive_requires_name: true

# BEGIN ANSIBLE MANAGED BLOCK

cluster.name: escluster-elastictest

network.host: 0.0.0.0

#node.master: true

#node.data: false

transport.tcp.port: 9300

http.port: 9200

network.bind_host: 0.0.0.0

xpack.security.enabled: false

searchguard.disabled: true

# END ANSIBLE MANAGED BLOCK

discovery.zen.ping.unicast.hosts: ["10.10.10.10","10.10.10.11"]

node.name: elastic70

searchguard.ssl.transport.pemcert_filepath: ssl/elastic70.pem

searchguard.ssl.transport.pemkey_filepath: ssl/elastic70.key

searchguard.ssl.transport.pemtrustedcas_filepath: ssl/root-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.resolve_hostname: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemcert_filepath: ssl/elastic70_http.pem

searchguard.ssl.http.pemkey_filepath: ssl/elastic70_http.key

searchguard.ssl.http.pemtrustedcas_filepath: ssl/root-ca.pem

searchguard.nodes_dn:

- CN=elastic70.example.net,OU=Ops,O=example EX\, Ltd.,DC=example,DC=net

- CN=elastic71.exmaple.net,OU=Ops,O=example EX\, Ltd.,DC=example,DC=net

searchguard.authcz.admin_dn:

- CN=root.exmaple.net,OU=Ops,O=example Com\, Inc.,DC=example,DC=net

/etc/elasticsearch/ssl

drwxr-s---. 4 root elasticsearch 4096 Mar  5 19:39 ..

-rw-r-----. 1 root elasticsearch 1196 Mar  5 19:39 elastic70.csr

-rw-r-----. 1 root elasticsearch 3334 Mar  5 19:39 elastic70.pem

-rw-r-----. 1 root elasticsearch 1704 Mar  5 19:39 elastic70.key

-rw-r-----. 1 root elasticsearch 3334 Mar  5 19:39 elastic70_http.pem

-rw-r-----. 1 root elasticsearch 1704 Mar  5 19:39 elastic70_http.key

-rw-r-----. 1 root elasticsearch 1184 Mar  5 19:39 elastic70_http.csr

-rw-r-----. 1 root elasticsearch 1246 Mar  5 19:39 elastic70_elasticsearch_config_snippet.yml

-rw-r-----. 1 root elasticsearch 1403 Mar  5 19:40 root-ca.pem

-rw-r-----. 1 root elasticsearch 1704 Mar  5 19:43 admin.key

-rw-r-----. 1 root elasticsearch 1110 Mar  5 19:43 admin.csr

-rw-r-----. 1 root elasticsearch 3249 Mar  5 19:43 admin.pem

Elasticsearch cluster health

curl -X GET "elastic70.example.net:9200/_cluster/health"?pretty

{

  "cluster_name" : "escluster-elastictest",

  "status" : "green",

  "timed_out" : false,

  "number_of_nodes" : 2,

  "number_of_data_nodes" : 2,

  "active_primary_shards" : 0,

  "active_shards" : 0,

  "relocating_shards" : 0,

  "initializing_shards" : 0,

  "unassigned_shards" : 0,

  "delayed_unassigned_shards" : 0,

  "number_of_pending_tasks" : 0,

  "number_of_in_flight_fetch" : 0,

  "task_max_waiting_in_queue_millis" : 0,

  "active_shards_percent_as_number" : 100.0

curl -X GET "10.10.10.10:9200/_cluster/health"?pretty

{

  "cluster_name" : "escluster-elastictest",

  "status" : "green",

  "timed_out" : false,

  "number_of_nodes" : 2,

  "number_of_data_nodes" : 2,

  "active_primary_shards" : 0,

  "active_shards" : 0,

  "relocating_shards" : 0,

  "initializing_shards" : 0,

  "unassigned_shards" : 0,

  "delayed_unassigned_shards" : 0,

  "number_of_pending_tasks" : 0,

  "number_of_in_flight_fetch" : 0,

  "task_max_waiting_in_queue_millis" : 0,

  "active_shards_percent_as_number" : 100.0

SG TLS tool config file to generate certs

search-guard-tlstool-1.6/config/es_cluster.yml

###

### Self-generated certificate authority

### 

# 

# If you want to create a new certificate authority, you must specify its parameters here. 

# You can skip this section if you only want to create CSRs

#

ca:

   root:

      # The distinguished name of this CA. You must specify a distinguished name.   

      dn: CN=root.ca.example.net,OU=CA,O=example.EX\, Ltd.,DC=example,DC=net

 

      # The size of the generated key in bits

      keysize: 2048

 

      # The validity of the generated certificate in days from now

      validityDays: 3650

      

      # Password for private key

      #   Possible values: 

      #   - auto: automatically generated password, returned in config output; 

      #   - none: unencrypted private key; 

      #   - other values: other values are used directly as password   

      pkPassword: none

      

      # The name of the generated files can be changed here

      file: root-ca.pem

      

   # If you want to use an intermediate certificate as signing certificate,

   # please specify its parameters here. This is optional. If you remove this section,

   # the root certificate will be used for signing.         

   intermediate:

      # The distinguished name of this CA. You must specify a distinguished name.

      dn: CN=signing.ca.example.net,OU=CA,O=example.EX\, Ltd.,DC=example,DC=net

   

      # The size of the generated key in bits   

      keysize: 2048

      

      # The validity of the generated certificate in days from now      

      validityDays: 3650

  

      pkPassword: none

            

      # If you have a certificate revocation list, you can specify its distribution points here      

      crlDistributionPoints: URI:https://raw.githubusercontent.com/floragunncom/unittest-assets/master/revoked.crl

 

### 

### Default values and global settings

###

defaults:

 

      # The validity of the generated certificate in days from now

      validityDays: 3650 

      

      # Password for private key

      #   Possible values: 

      #   - auto: automatically generated password, returned in config output; 

      #   - none: unencrypted private key; 

      #   - other values: other values are used directly as password   

      pkPassword: none

      

      # Specifies to recognize legitimate nodes by the distinguished names

      # of the certificates. This can be a list of DNs, which can contain wildcards.

      # Furthermore, it is possible to specify regular expressions by

      # enclosing the DN in //. 

      # Specification of this is optional. The tool will always include

      # the DNs of the nodes specified in the nodes section.            

      #nodesDn:

      #- "CN=*.example.com,OU=Ops,O=Example Com\\, Inc.,DC=example,DC=net"

      # - 'CN=node.other.com,OU=SSL,O=Test,L=Test,C=DE'

      # - 'CN=*.example.com,OU=SSL,O=Test,L=Test,C=DE'

      # - 'CN=elk-devcluster*'

      # - '/CN=.*regex/' 

 

      # If you want to use OIDs to mark legitimate node certificates, 

      # the OID can be included in the certificates by specifying the following

      # attribute

      

      # nodeOid: "1.2.3.4.5.5"

 

      # The length of auto generated passwords            

      generatedPasswordLength: 12

      

      # Set this to true in order to generate config and certificates for 

      # the HTTP interface of nodes

      httpsEnabled: true

      

      # Set this to true in order to re-use the node transport certificates

      # for the HTTP interfaces. Only recognized if httpsEnabled is true

      

      # reuseTransportCertificatesForHttp: false

      

      # Set this to true to enable hostname verification

      #verifyHostnames: false

      

      # Set this to true to resolve hostnames

      #resolveHostnames: false

      

      

###

### Nodes

###

#

# Specify the nodes of your ES cluster here

#      

nodes:

  - name: elastic70

    dn: CN=elastic70.example.net,OU=Ops,O=example EX\, Ltd.,DC=example,DC=net

    dns: 

      - elastic70.example.net

    ip: 

      - 10.10.10.10

 

  - name: elastic71

    dn: CN=elastic71.example.net,OU=Ops,O=example EX\, Ltd.,DC=example,DC=net

    dns: 

      - elastic71.example.net

    ip: 

      - 10.10.10.11

 

###

### Clients

###

#

# Specify the clients that shall access your ES cluster with certificate authentication here

#

# At least one client must be an admin user (i.e., a super-user). Admin users can

# be specified with the attribute admin: true    

#        

clients:

  - name: admin

    dn: CN=root.example.net,OU=Ops,O=example Com\, Inc.,DC=example,DC=net

    admin: true

someone please help me whats wrong with my configuration, thanks.

[Vijaya Krishna]

it looks working but still has an issue saying search guard not initialized

bash sgadmin.sh --enable-shard-allocation -key /etc/elasticsearch/ssl/admin.key -cert /etc/elasticsearch/ssl/admin.pem -cacert /etc/elasticsearch/ssl/root-ca.pem -icl -nhnv -h elastic70.example.net

WARNING: JAVA_HOME not set, will use /bin/java

Search Guard Admin v6

Will connect to elastic70.example.net:9300 ... done

Elasticsearch Version: 6.5.4

Search Guard Version: 6.5.4-24.1

Connected as CN=root.example.net,OU=Ops,O=example Com\, Inc.,DC=example,DC=net

Persistent and transient shard allocation enabled

issue:

curl -X GET "https://10.10.10.10:9200/_cluster/health"?pretty

Search Guard not initialized (SG11). See http://docs.search-guard.com/v6/sgadmin

[SG]

run something like

sgadmin.sh -cd ???? -key /etc/elasticsearch/ssl/admin.key -cert /etc/elasticsearch/ssl/admin.pem -cacert /etc/elasticsearch/ssl/root-ca.pem -icl -nhnv -h elastic70.example.net

where ???? point to the directory with the sg_*.yml files in there

See also https://docs.search-guard.com/latest/search-guard-installation#initializing-search-guard

> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/380b02c9-5c7b-40dd-98c7-f2bb02f26252%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Vijaya Krishna]

Thanks, that works.