Leofs and racks - data protection

[Martin]

Hello friends, I have some question about storage.racks implementation with leofs.

Is possible implement this feature to leofs like elasticsearch node.racks feature or cassandra?

Sincerely,

Martin

[mocchira]

Hi Martin,

Sorry for the long delay.

> Is possible implement this feature to leofs like elasticsearch node.racks feature or cassandra?

Yes it's possible.

However now it might be difficult to find the information related to rack awareness support of LeoFS because of the lack of information on our official document. We will publish the detailed explanation about rack awareness like https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-awareness.html on our doc ASAP.

So for now I'd like to explain about it below. I hope you find it helpful on how to use rack awareness on LeoFS. Please let me know if you have any questions/troubles.

- leo_storage.conf

  - replication.rack_awareness.rack_id which enables you to specify which rack the leo_storage belongs

- leo_manager.conf (on master)

  - consistency.rack_aware_replicas which enables you to specify how many replicas should spread across the rack

For example, let's say that we try to create a cluster with

- 6 storage nodes (node[1-6])

- 2 physical racks (rack[1-2])

- 2 replicas and each replica should belong to a different rack

then set the configurations as following

- replication.rack_awareness.rack_id on each leo_storage.conf

  - node1: replication.rack_awareness.rack_id = rack1

  - node2: replication.rack_awareness.rack_id = rack1

  - node3: replication.rack_awareness.rack_id = rack1

  - node4: replication.rack_awareness.rack_id = rack2

  - node5: replication.rack_awareness.rack_id = rack2

  - node6: replication.rack_awareness.rack_id = rack2

- consistency.rack_aware_replicas = 2

Best,

Kanno.

2018年5月18日金曜日 17時34分34秒 UTC+9 Martin:

[Martin]

Great!

I will try it.

Thank you!

Martin

Dňa utorok, 22. mája 2018 10:54:02 UTC+2 mocchira napísal(-a):

[yoshiyuki kanno]

Hi Martin,

After I posted the reply to you, I confirmed this rack awareness works
as expected, just in case. Unfortunately it didn't for some reason (It
seems some degradation occurred at some point) so I filed the issue at
https://github.com/leo-project/leofs/issues/1046 to solve this
problem.

That being said, I'd like to ask you to try rack awareness of LeoFS on
the next stable release 1.4.2 which will come out around the end of
June.

Sorry for the inconvenient.

Best,
Kanno.

> --
> You received this message because you are subscribed to the Google Groups
> "LeoProject.LeoFS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to leoproject_leo...@googlegroups.com.
> To post to this group, send email to leoproje...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/leoproject_leofs/c889f68e-95f3-4a3e-b263-5e528ea8e46d%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Yoshiyuki Kanno
LeoFS Committer(http://leo-project.net/leofs/index.html)

[mocchira]

Hi Martin,

As you might know, 

We have released LeoFS 1.4.2 yesterday which includes the fix for rack-awareness replication so now you can try it out.

and also you can find the doc for rack-awareness on https://mocchira.github.io/leofs/admin/settings/cluster/#rack-awareness (this is not the official doc but the same content will be published on the official doc later).

Please ask me if you have any questions or face any difficulties.

Best,

Kanno.

2018年5月24日木曜日 15時35分25秒 UTC+9 mocchira:

> email to leoproject_leofs+unsubscribe@googlegroups.com.
> To post to this group, send email to leoproject_leofs@googlegroups.com.

[Martin]

Great!

I will try it.

And is there any doc for AWS Signature v4 support or configuration with LeoFS?

Best,

Martin.

Dňa piatok, 20. júla 2018 6:13:41 UTC+2 mocchira napísal(-a):

2018年5月24日木曜日 15時35分25秒 UTC+9 mocchira:

> email to leoproject_leo...@googlegroups.com.
> To post to this group, send email to leoproje...@googlegroups.com.

[mocchira]

Hi Martin,

LeoFS can speak both AWS Signature v4 and v2 (deprecated) by default.

You can refer to https://github.com/leo-project/leofs_client_tests repository (which is one of the integration test tools to verify each AWS S3 Client working with LeoFS) as a doc to connect to LeoFS via v4 with the S3 client you are supposed to use.

I hope you find it useful.

Best,

Kanno.

2018年7月27日金曜日 22時48分05秒 UTC+9 Martin: