2/3 of the nodes refuse to restart

[David Melamed]

Hi everyone,

I am a BigCouch user with a very basic 3-nodes cluster.

I had no particular issue installing it on EC2 until recently, where the whole cluster crashed due to a lot of conflicts apparently.

My problem is that I didn't manage to restart all the nodes. 

The process is either crashing on startup with the following message on console: Kernel pid terminated (application_controller) ({application_terminated,mem3,shutdown}) - looks like it doesn't write any additional crash log - OR the daemon and beam processes seem to launch put listen to port 5986 only (which return returns an empty response upon curl call); no port 5984 listening.

The only working node seems to work just fine.

My questions:

1) Are all the data present on the surviving node or the data are split in the 3 nodes?

2) What should I do to overcome my current problem? Try to reinstall bigcouch in the 2 nodes? Move the data to another server? Recreate a cluster based on the data of node2?

Any help would be highly appreciated.

Thanks,

David