a question about a high availability of Elasticsearch cluster

[jih...@braincolla.com]

Hello Pat,

May I have a question about Elasticsearch cluster in PIO and UR.

I've implemented some Elasticsearch cluster consisted of 3 nodes on below options.

<elasticsearch.yml>

cluster.name: my-search-cluster

network.host: 0.0.0.0

discovery.zen.ping.unicast.hosts: [“node 1”, “node 2", “node 3”]

And I writed PIO options below.

<pio-env.sh>

...

# Elasticsearch Example

PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch

PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http

PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/usr/local/elasticsearch

# The next line should match the ES cluster.name in ES config

PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=my-search-cluster

# For clustered Elasticsearch (use one host/port if not clustered)

PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=node1,node2,node3

PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200,9200,9200

...

My questions are below.

1. I killed the Elasticsearch process in node 2 or node 3. PIO is well working. But when the Elasticsearch process in node 1 is killed, PIO is not working. Is it right?

2. I've changed PIO options below. I killed the Elasticsearch process in node 1 or node 3. PIO is well working. But when the Elasticsearch in node 2 is killed, PIO is not working. Is it right?

PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=node2,node1,node3

3. In my opinion, if first node configurd at PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS is killed, PIO is not working. Is it right? If yes, please let me know why it happened.

Thank you.

[Pat Ferrel]

This should work with any node down. Elasticsearch should elect a new master.

What version of PIO are you using? PIO and the UR changed the client from the transport client to the RET client in 0.12.0, which is why you are using port 9200.

Do all PIO functions work correctly like:

• pio app list
• pio app new

with all the configs and missing nodes you describe? What I’m trying to find out is if the problem is only with queries, which do use ES is a different way.

What is the es.nodes setting in the engine.json’s sparkConf?

--
You received this message because you are subscribed to the Google Groups "actionml-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to actionml-use...@googlegroups.com.
To post to this group, send email to action...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/254aed9f-c975-4726-8b90-2ea80d6a2a34%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.