Can't update crafter delivery content because of elasticsearch

[Benediktus Ghara]

So i make an update for a template in my crafter authoring studio and published it. Why did when i open the template on crafter delivery the content is not updated in my application. But when i trace into the crafter authoring and delivery VM on path repos, it's already updated (i open the .xml file on repos). From my crafter delivery log i got this error:

Caused by: org.craftercms.search.elasticsearch.exception.ElasticsearchException: [******] Error indexing document /***/***/***/***.xml
        at org.craftercms.search.elasticsearch.impl.ElasticsearchServiceImpl.doIndex(ElasticsearchServiceImpl.java:208)
        at org.craftercms.search.elasticsearch.impl.ElasticsearchServiceImpl.index(ElasticsearchServiceImpl.java:195)
        at org.craftercms.search.elasticsearch.impl.ElasticsearchServiceImpl.index(ElasticsearchServiceImpl.java:228)
        at org.craftercms.search.elasticsearch.batch.ElasticsearchIndexingUtils.doUpdate(ElasticsearchIndexingUtils.java:66)
        ... 15 more
Caused by: org.craftercms.search.elasticsearch.exception.ElasticsearchException: [*****] Error deleting document /***/***/***/***.xml
        at org.craftercms.search.elasticsearch.impl.ElasticsearchServiceImpl.doDelete(ElasticsearchServiceImpl.java:279)
        at org.craftercms.search.elasticsearch.impl.ElasticsearchServiceImpl.doIndex(ElasticsearchServiceImpl.java:204)
        ... 18 more
Caused by: ElasticsearchStatusException[Elasticsearch exception [type=cluster_block_exception, reason=blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]]
        at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177)
        at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:2050)
        at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:2026)
        at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1775)
        at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1732)
        at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1694)
        at org.elasticsearch.client.RestHighLevelClient.delete(RestHighLevelClient.java:1042)
        at org.craftercms.search.elasticsearch.impl.ElasticsearchServiceImpl.doDelete(ElasticsearchServiceImpl.java:277)
        ... 19 more
        Suppressed: org.elasticsearch.client.ResponseException: method [DELETE], host [http://localhost:9202], URI [/***/***/***/***?timeout=1m], status line [HTTP/1.1 403 Forbidden]
{"error":{"root_cause":[{"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"}],"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"},"status":403}
                at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:929)
                at org.elasticsearch.client.RestClient.performRequest(RestClient.java:229)
                at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1762)
                ... 23 more
        Caused by: org.elasticsearch.client.ResponseException: method [DELETE], host [http://localhost:9202], URI [/***/***/***/***?timeout=1m], status line [HTTP/1.1 403 Forbidden]
{"error":{"root_cause":[{"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"}],"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"},"status":403}
                at org.elasticsearch.client.RestClient$1.completed(RestClient.java:548)
                at org.elasticsearch.client.RestClient$1.completed(RestClient.java:533)
                at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
                at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181)
                at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
                at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
                at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
                at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
                at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
                at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
                at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
                at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
                at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
                at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
                at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
                at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
                ... 1 more

I've looked, this may be caused by low disk space or elastic search configuration and index settings, but I'm only using 10% space and read-only mode is already not set for the entire cluster. What should i do?

[Sumer Jabri]

Hi Benedictus,

What version of CrafterCMS are you using? Also, is it the Enterprise Edition or the Community Edition?

At a high level, CrafterCMS cannot _write_ to the Elasticsearch index. Consider investigating the Elasticsearch configuration/setup to understand better why it's doing that.

If you'd like a more direct exchange of support/ideas, consider joining us in Slack at https://craftercms.com/slack

--sumer

[Benediktus Ghara]

We're using community edition and version 3.1.14. I didn't make any change in elasticsearch configuration, so the configuration still a default. But here's my configuration on /crafter-delivery/bin/elasticsearch/config/elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: ${ES_INDEXES_DIR}
#
# Path to log files:
#
path.logs: ${ES_LOGS_DIR}
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
http.port: ${ES_PORT}
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#

# Disable cluster discovery
discovery.type: single-node

#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes:
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true