Hi Austin
My questions is how to make sure primary stays as primary with read,write access when secondary node is down/unreachable(or any other problems).
Note: replica set only contains 2 nodes, primary and secondary.
A MongoDB replica set was designed to provide high availability, and it was designed to be operated with three nodes as minimum. That is, a three-node replica set can tolerate the loss of one node, and still be operational. These failure scenarios are well understood and the official drivers are expected to continue working without needing manual interference.
If two nodes are lost (or the primary cannot communicate with the other two nodes), then the remaining node will become a secondary (if it’s currently the primary).
This is to prevent a “split-brain” situation, since you cannot know if the two nodes are actually down, or is just unreachable due to network partition.
If you’re trying to force the primary to stay primary, then you’re working against the high-availability design.
In your case, there are two possible solutions that will stay within then designed replica set high availability:
An arbiter node will carry no data and allow you to have a 3-node replica set, although actually you only have 2 data-bearing node. Having said that, it is strongly recommended to have 3 data bearing nodes instead of using an arbiter.
I would recommend you to check out the Replication manual page for more details into MongoDB’s replica set.
Please note that there are some free courses in MongoDB University that may be of interest to you.
Best regards
Kevin