About the replication algorithm

[Burak Sezer]

Hello,

I wonder that what replication algorithm has been used in Seaweedfs? I also know that Raft is used to replicate logs between cluster(probably master's log replication for a disaster) members but I think that this doesn't explain my question. 

If I remove one of file chunks(or a data loss case) on a volume server, how the master decides to create a new copy of that chunk to keep the file safe? inotify may be used to do this job but there is no clue in the code for this. Traversing over the files periodically? It's not feasible. 

I hope that I could ask my question clearly. 

Thanks is advance.

[Chris Lu]

If a volume has n copies, and one copy is lost, the other n-1 copies will become read only. When the lost copy goes back online, the n copies will become writable.

--
You received this message because you are subscribed to the Google Groups "Seaweed File System" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Burak Sezer]

Thank you but I think that this is not a answer to my question. I still wonder that if a chunk is disappeared(faulty harddrive, maybe), the system is how to decide to create one more copy for it.

To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+unsubscribe@googlegroups.com.

[Burak Sezer]

In another words, how to detect a disappeared file in weedfs before creating one more copy for it?

[Chris Lu]

The system does not decide to create one more copy. 

On Tuesday, November 17, 2015, Burak Sezer <bbura...@gmail.com> wrote:

Thank you but I think that this is not a answer to my question. I still wonder that if a chunk is disappeared(faulty harddrive, maybe), the system is how to decide to create one more copy for it.

On Tuesday, November 17, 2015 at 5:11:57 PM UTC+2, ChrisLu wrote:

If a volume has n copies, and one copy is lost, the other n-1 copies will become read only. When the lost copy goes back online, the n copies will become writable.

On Tuesday, November 17, 2015, Burak Sezer <bbura...@gmail.com> wrote:

Hello,

I wonder that what replication algorithm has been used in Seaweedfs? I also know that Raft is used to replicate logs between cluster(probably master's log replication for a disaster) members but I think that this doesn't explain my question. 

If I remove one of file chunks(or a data loss case) on a volume server, how the master decides to create a new copy of that chunk to keep the file safe? inotify may be used to do this job but there is no clue in the code for this. Traversing over the files periodically? It's not feasible. 

I hope that I could ask my question clearly. 

Thanks is advance.

--
You received this message because you are subscribed to the Google Groups "Seaweed File System" group.

To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Seaweed File System" group.

To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+...@googlegroups.com.

[Jan Kunzmann]

Hi Chris,

so far I understand from this thread: if a volume server breaks because of a physical damage, all volumes which have one replica on that server turn read-only. There is currently no automated way to provision a new machine with a volume server on it and let it replicate the missing volumes.

However, I would assume that it's possible to use 'weed backup' to clone the degraded volumes. But how can I start a new volume server afterwards, make the cloned volumes known to the master and then have a full writeable service again?

Jan

On Tuesday, November 17, 2015 at 4:51:50 PM UTC+1, ChrisLu wrote:

The system does not decide to create one more copy. 

On Tuesday, November 17, 2015, Burak Sezer <bbura...@gmail.com> wrote:

Thank you but I think that this is not a answer to my question. I still wonder that if a chunk is disappeared(faulty harddrive, maybe), the system is how to decide to create one more copy for it.

On Tuesday, November 17, 2015 at 5:11:57 PM UTC+2, ChrisLu wrote:

If a volume has n copies, and one copy is lost, the other n-1 copies will become read only. When the lost copy goes back online, the n copies will become writable.

On Tuesday, November 17, 2015, Burak Sezer <bbura...@gmail.com> wrote:

Hello,

I wonder that what replication algorithm has been used in Seaweedfs? I also know that Raft is used to replicate logs between cluster(probably master's log replication for a disaster) members but I think that this doesn't explain my question. 

If I remove one of file chunks(or a data loss case) on a volume server, how the master decides to create a new copy of that chunk to keep the file safe? inotify may be used to do this job but there is no clue in the code for this. Traversing over the files periodically? It's not feasible. 

I hope that I could ask my question clearly. 

Thanks is advance.

--
You received this message because you are subscribed to the Google Groups "Seaweed File System" group.

To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Seaweed File System" group.

To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+unsubscribe@googlegroups.com.

[Chris Lu]

Hi, Jan,

To make a volume server known to the master, you just need to start it with -mserver option.

Chris

On Thursday, November 19, 2015, Jan Kunzmann <jan.ku...@erasys.de> wrote:

Hi Chris,

so far I understand from this thread: if a volume server breaks because of a physical damage, all volumes which have one replica on that server turn read-only. There is currently no automated way to provision a new machine with a volume server on it and let it replicate the missing volumes.

However, I would assume that it's possible to use 'weed backup' to clone the degraded volumes. But how can I start a new volume server afterwards, make the cloned volumes known to the master and then have a full writeable service again?

Jan

On Tuesday, November 17, 2015 at 4:51:50 PM UTC+1, ChrisLu wrote:

The system does not decide to create one more copy. 

On Tuesday, November 17, 2015, Burak Sezer <bbura...@gmail.com> wrote:

Thank you but I think that this is not a answer to my question. I still wonder that if a chunk is disappeared(faulty harddrive, maybe), the system is how to decide to create one more copy for it.

On Tuesday, November 17, 2015 at 5:11:57 PM UTC+2, ChrisLu wrote:

If a volume has n copies, and one copy is lost, the other n-1 copies will become read only. When the lost copy goes back online, the n copies will become writable.

On Tuesday, November 17, 2015, Burak Sezer <bbura...@gmail.com> wrote:

Hello,

I wonder that what replication algorithm has been used in Seaweedfs? I also know that Raft is used to replicate logs between cluster(probably master's log replication for a disaster) members but I think that this doesn't explain my question. 

If I remove one of file chunks(or a data loss case) on a volume server, how the master decides to create a new copy of that chunk to keep the file safe? inotify may be used to do this job but there is no clue in the code for this. Traversing over the files periodically? It's not feasible. 

I hope that I could ask my question clearly. 

Thanks is advance.

--
You received this message because you are subscribed to the Google Groups "Seaweed File System" group.

To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Seaweed File System" group.

To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Seaweed File System" group.

To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+...@googlegroups.com.