Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Leadership lost while committing log

31 views
Skip to first unread message

AJ

unread,
Nov 17, 2024, 4:19:06 PM11/17/24
to raft-dev
Hi, 

I have initialized a raft node and added 2 peers one-by-one. While the first peer is added, the second peer addition fails with an error "leadership lost while committing log.". 
 
It seems like after adding the first peer, the original leader (cluster-aus) is losing the leadership due to re-election. Not sure if this is normal behavior. What is the correct way to add the two peers to form the cluster? 
 
The exact same example in a unit test code or example with the same workflow works fine. However, when testing with a set of VM nodes, it faces the below issues. Not sure if some timer values are influencing the protocol behavior.

The leader node: cluster-aus
Peer 1 node: cluster-nyc
Peer 2 node: cluster-sjc

Below is the chronology of the events. 
  • time="2024-11-17 20:48:03.898223" level=info msg="#### raft node state transition ####" current_leader=cluster-aus new_state=Leader node_id=cluster-aus previousS_state=UnKnown 
  • time="2024-11-17 20:48:03.898277" level=info msg="adding peer node to the cluster" current_leader=cluster-aus peer_node_addr=192.168.133.170 peer_node_id=cluster-nyc peer_node_port=30653 
  • time="2024-11-17 20:48:03.898879" level=info msg="the node joined into the cluster" leader_node_id=cluster-aus node_id=cluster-aus peer_node_id=cluster-nyc 
<<< Perhaps not required, but added a 1 sec delay for the first peer to join the cluster >>>
  • time="2024-11-17 20:48:04.899394" level=info msg="adding peer node to the cluster" current_leader=cluster-aus peer_node_addr=192.168.133.178 peer_node_id=cluster-sjc peer_node_port=30653 
  • time="2024-11-17 20:48:08.900554" level=error msg="failed to join the node into the cluster" current_leader= error="failed to add the node to the raft cluster via leader cluster-aus, error leadership lost while committing log" node_id=cluster-aus peer_node_id=cluster-sjc 

AJ. 


Reply all
Reply to author
Forward
0 new messages