Cluster setup by the Kubernetes Operator failed

[Björn Þór Björnsson]

Hi

One database server node in a default configuration using the Kubernetes Operator suddenly failed by getting stuck in the Pending state, with the event log message:

MountVolume.NewMounter initialization failed for volume "arangodb-local-storage-3d86b0-sm3rxsop3kpft8p2" : path "/var/lib/arango-storage/sm3rxsop3kpft8p2" does not exist FailedMount ...

Trying to remove the pod/deployment, it immediately got recreated, but just to the same pending state.

I was unable to see further evidence of what went wrong and can only guess that the storage got exhausted, as storage size wasn't being explicitly set, so each dbserver just had 8 gigabytes of storage (which should have been enough though, as the backup size is just 150 megabytes?).

Do you have any idea what may have been failing?  How could I have better inspected what was the issue?  How can the available space of the Persistent Volume Claims be monitored / inspected?

/Björn