high disk watermark [90%] exceeded

758 views
Skip to first unread message

Bartłomiej

unread,
Nov 14, 2016, 12:44:39 PM11/14/16
to Nexus Users
Hi,

I've run into the following issue running nexus in docker (3.1) which has its '/nexus-data' on AWS EFS share.

2016-11-14 17:19:31,960+0000 WARN  [elasticsearch[84D7013A-6E76DF96-CD56BA97-EDE2864B-01E6A584][management][T#2]] *SYSTEM org.elasticsearch.cluster.routing.allocation.decider - [84D7013A-6E76DF96-CD56BA97-EDE2864B-01E6A584] high disk watermark [90%] exceeded on [Xe5R7PlYSd-RVoZ3GiBsPg][84D7013A-6E76DF96-CD56BA97-EDE2864B-01E6A584][/nexus-data/elasticsearch/nexus/nodes/0] free: 8191.9pb[-99.9%], shards will be relocated away from this node
2016-11-14 17:19:31,962+0000 INFO  [elasticsearch[84D7013A-6E76DF96-CD56BA97-EDE2864B-01E6A584][management][T#2]] *SYSTEM org.elasticsearch.cluster.routing.allocation.decider - [84D7013A-6E76DF96-CD56BA97-EDE2864B-01E6A584] rerouting shards: [high disk watermark exceeded on one or more nodes]

EFS shows up as 8 Exabytes to a client system, it seems like elasticsearch 2.2 can't handle it properly. I've had to start nexus with:
-Des.cluster.routing.allocation.disk.threshold_enabled=false
to work around the issue.

Since elasticsearch is running in one node configuration perhaps it should be a default setting anyway (es node won't be able to relocate any shards ever).

Peter Lynch

unread,
Nov 14, 2016, 3:27:47 PM11/14/16
to Bartłomiej, Nexus Users
Hello,

Elasticsearch compatibility with AWS is a known issue:


-Peter

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/62396b49-4b9b-4843-9ca8-d79ac0cdf82e%40glists.sonatype.com.
For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

Reply all
Reply to author
Forward
0 new messages