How to configure the index in the cloud

12 views
Skip to first unread message

Xander Steinmann

unread,
May 4, 2021, 10:09:40 AMMay 4
to dotCMS User Group
Hi all,

We have a dotCMS 5.3.8.4 with its ES index in AWS. To make it work we set the following settings in dotcms-config-cluster-ext.properties:

ES_PORT=443
ES_AUTH_TYPE=NONE
ES_TLS_ENABLED=false
DIST_INDEXATION_ENABLED=false
AUTOWIRE_CLUSTER_TRANSPORT=false

Since the AWS server is in our own managed environment we don't need authentication or TLS. We can access the https URL on our server.

This all works fine until we restart dotCMS and/or redeploy the plugins. Then dotCMS can't find the index any more, or any index really, and we need to reindex to make everything work again. When looking in the database I do see the current index which exists in AWS. For some reason dotCMS can't find the existing indexes in AWS... It works fine locally with Docker or a local ES. Maybe we misconfigured the ES settings or it requires security? Any ideas?

Kind regards,

Xander

Will Ezell

unread,
May 4, 2021, 11:53:40 AMMay 4
to dot...@googlegroups.com
All dotCMS elasticsearch indexes are prefixed with the DOTCMS_CLUSTER_ID for the dotCMS cluster to which they belong - that way multiple dotCMS environments can share a single elasticsearch cluster without stepping on each other.  The DOTCMS_CLUSTER_ID is generated at startup and is stored in the dot_cluster table or it can be passed in as an system or environmental variable.  You should make sure that you are not deleting the DOTCMS_CLUSTER_ID and that DOTCMS_CLUSTER_ID persists across restarts.

If that's not the case, then yeah, maybe permissions on ES?  Fire up Kibana and make sure the indexes are there.

Xander Steinmann

unread,
May 5, 2021, 4:24:28 AMMay 5
to dotCMS User Group
Hi Will,

Yes, you were correct. I also needed to set the DOTCMS_CLUSTER_ID and DOTCMS_CLUSTER_SALT to make it work.
One remark: if there is already something in the dot_cluster table other than the DOTCMS_CLUSTER_ID then dotCMS will keep on trying to add the DOTCMS_CLUSTER_ID and fail because of a duplicate key. After clearing the dot_cluster table I was able to stop/start dotCMS without issues. Maybe you would want to fix this in the ClusterFactory .writeClusterData() method?

Thanks!

Xander
Op dinsdag 4 mei 2021 om 17:53:40 UTC+2 schreef Will Ezell:
Reply all
Reply to author
Forward
0 new messages