question about leofs cluster change

[许若辰]

Hi friends, I have some question about leofs and need your help.

1.How can I move all data from one cluster to another?

Maybe I need a command to list all my files and then I can move them one by one. I can use 'ls' in nfs mode, any other way?

2.How to understand update-managers command?

Can I update both manager in the same time?

What will happen when I update just one manager?

Thanks.

[mocchira]

Hi,

Sorry for the long delay.

> 1.How can I move all data from one cluster to another?

> Maybe I need a command to list all my files and then I can move them one by one. I can use 'ls' in nfs mode, any other way?

Other than ls in nfs mode, there are two ways.

## One is listing all files by leofs-adm diagnosis-start. details are described on http://leo-project.net/leofs/docs/admin/system_operations/data/#diagnosis

Compared to listing in nfs mode,

- Pros

  - the speed of listing is very fast so if you already have much objects in the source cluster then this choice would be suitable in most cases.

- Cons

  - need to run the command the number of storage nodes times like below (but it shouldn't be big deal)

    - leofs-adm diagnosis-start storage_0@a

    - leofs-adm diagnosis-start storage_1@b

    - leofs-adm diagnosis-start storage_2@c

  - Since the list generated by diagnosis-start can contain the same file multiple times, you may have to dedupe before putting files into the destination cluster (or just ignoring duplicated files also should work although the number of requests sent to the destination will be increased)

  -  Since the list generated by diagnosis-start can contain files that already got deleted in the past, you may have to ignore such files when sending files into the destination. (or doing compaction on all nodes and prohibiting to request deletes during the moving process grantees there is no files already deleted in the list)

IMO, Although this option would force you to have more knowledge than listing in nfs however I'd recommend you this one as listing in nfs may take too much time and make a significant load against the source cluster during listing.

## The other one is setting up multi data center replication (MDCR). details are described on http://leo-project.net/leofs/docs/admin/system_operations/multi_dc_replication/

A brief procedure is 

- setting up MDCR properly

- invoke leofs-adm recover-cluster then data copy into the destination cluster will be started

Compared to listing in nfs mode,

- Pros

  - the speed of transfer is very fast so if you already have much objects in the source cluster then this choice would be suitable in most cases.

- Cons

  - need to tweak MDCR settings

  - since MDCR use the specific tcp ports, you may have to configure firewall rules and so on depending on your network settings. 

IMO, diagnose-start VS MDCR

the winner is diagnose-start as there has been much experiences than MDCR to move (transfer) data to the another cluster.

> 2.How to understand update-managers command?

> Can I update both manager in the same time?

Yes both at once.

Best,

Kanno.

2017年12月5日火曜日 18時32分44秒 UTC+9 许若辰:

[许若辰]

Thanks, your reply are detailed. But I have more questions here, maybe explaining more principles in doc would be a excellent thing.

1.How to understand manager update?

Would the storage nodes send their info about data to new manager pair? Or the data would lost right?

So anytime I want to just change manager pair, I can setup two new manager and do update?

2.How to understand cluster replication?

For example, now I setup cluster B as a replication of A. In both A and B the replication number is 3. But the number from A to B is 1.(They are two different config option right?)

Now I put an object into A, finally how many copies will be in B?(I think the answer is 1)

Should I put objects into B or only use B as a replication cluster?(I think is answer is only replication)

If I want to destroy A and use B as our main cluster, is it possible? How to make B storing 3 copies?

Thanks again.

在 2017年12月7日星期四 UTC+8下午4:30:36，mocchira写道：

[mocchira]

Hi,

> maybe explaining more principles in doc would be a excellent thing.

Agreed as there are still some missing parts in our docs now. we will add more principals into our docs.

> 1.How to understand manager update?

> Would the storage nodes send their info about data to new manager pair? Or the data would lost right?

If `data` is meant to be files stored in LeoFS then data lost never happen as manager(s) are not involved with actual data (files) but only take care cluster members.

also just in case let me explain what update-manager will do is sending new manager pair to all members in the cluster. after that, each member send heartbeat to the new manager pair.  

> So anytime I want to just change manager pair, I can setup two new manager and do update?

Yes.

> For example, now I setup cluster B as a replication of A. In both A and B the replication number is 3. But the number from A to B is 1.(They are two different config option right?)

Right.

> Now I put an object into A, finally how many copies will be in B?(I think the answer is 1)

You can configure the number of copies through mdc_replication.num_of_replicas_a_dc at leo_manager.conf in A.

So the actual answer is mdc_replication.num_of_replicas_a_dc.

> Should I put objects into B or only use B as a replication cluster?(I think is answer is only replication)

You can put/get/delete objects directly into/out of B as MDCR adopts active-active style replication.

> If I want to destroy A and use B as our main cluster, is it possible? How to make B storing 3 copies?

Possible if you follow the settings below

set mdc_replication.num_of_replicas_a_dc at leo_manager.conf in A to 3 and

set consistency.num_of_replicas at leo_manager.conf in B to 3.

Best,

Kanno.

2017年12月8日金曜日 10時13分25秒 UTC+9 许若辰:

[许若辰]

More

1.What would happen if I restart all node after manager update? Would all nodes send heartbeat to old manager pair?

Should I modify config files of all nodes for updating manager pair forever?

2.Actually I mean my example cross all problems.

That is if I set mdc_replication.num_of_replicas_a_dc at leo_manager.conf in A to 1 and consistency.num_of_replicas in both A and B to 3.

What will happen?

Thanks a lot.

在 2017年12月8日星期五 UTC+8上午10:09:44，mocchira写道：

[yoshiyuki kanno]

> 1.What would happen if I restart all node after manager update? Would all nodes send heartbeat to old manager pair?
> Should I modify config files of all nodes for updating manager pair forever?

Yes you should modify config files to make it permanent change.

> That is if I set mdc_replication.num_of_replicas_a_dc at leo_manager.conf in A to 1 and consistency.num_of_replicas in both A and B to 3.
> What will happen?

Objects put into A will be replicated to B with 1 replica.
Objects put into B will be replicated across the local cluster with 3 replica.

Best,
Kanno.

> --
> You received this message because you are subscribed to the Google Groups
> "LeoProject.LeoFS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to leoproject_leo...@googlegroups.com.
> To post to this group, send email to leoproje...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/leoproject_leofs/e739b8fd-abd4-43ee-b894-93fa40673b3c%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Yoshiyuki Kanno
LeoFS Committer(http://leo-project.net/leofs/index.html)

[许若辰]

For the 2nd question.

Now I want to destroy A and use B as main cluster.

What should I do to make every object that copied from A have 3 copies?

在 2017年12月8日星期五 UTC+8下午12:37:17，mocchira写道：

[yoshiyuki kanno]

>For the 2nd question.
>Now I want to destroy A and use B as main cluster.
>What should I do to make every object that copied from A have 3 copies?

Invoke
leofs-adm recover-node storage_node_d...@example.com
on each storage in B then every object get to have 3 copies.

Best,
Kanno.

> https://groups.google.com/d/msgid/leoproject_leofs/d8bce1e7-40ca-4006-878c-8a96712f61b0%40googlegroups.com.