Raft : best way to handle a scenario, Ageing of data (resources which dont change after some time)

[RajBha]

Hello,

Consider I have 5 servers (A,B,C,D and E) . I have a client which writes to server A,B,C,D and E in round robin manner.

Lets consider : 

client  writes first to server A . Server A , creates a new file and a new  Raft Group , elects itself the leader and all others become followers to the data which is written to A

client  then writes  to server B . Server B creates a new file and  creates a new  Raft Group , elects itself the leader and all others become followers to the data which is written to B.

and this process goes on for a time.

After some time, server A decides to closes the previous existing file. (say it reached a particular size or for a particular set interval) , creates a new file  and a new  Raft Group  and elects it self the leader and the same process is repeated. Lets call the initia raft group as Raft_A1 and 2nd one as Raft_A2

The reason for closing file is that we want to age the data and remove it. The removal doesnt happen immediately. So even after closing we need to keep the data. We need to ensure minimum copies are available. If two servers goes down and two new servers are added  (the data is replicated to the these 2 new servers).

Once the data needs to be deleted, how to handle a deletion of raft group.

Another question : is it ok to have large number of raft groups say 10000 or there is a better mechanism to handle it 

[Oren Eini (Ayende Rahien)]

Why not handle the group deletion as a raft decision?

So When A1 transition to A2, it transfer ownership on the file.

And A2, in order to delete, decide at some point to do so, and send a raft command whose commit will do so.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "raft-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raft-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.