Backrunner questions: metadata and buckets
36 views
Skip to first unread message
Anatoly Popov
Mar 21, 2016, 11:31:50 AM3/21/16
to reverbrain
Hello.
1. How much space should I reserve for metadata per file? Are there any estimates?
2. Can I update buckets metadata without losing data? E.g. for adding new groups to buckets? Will backrunner replicate existing data to new buckets or I should initiate `recovery on read` manually?
Best regards,
Anatoly Popov.
1. How much space should I reserve for metadata per file? Are there any estimates?
2. Can I update buckets metadata without losing data? E.g. for adding new groups to buckets? Will backrunner replicate existing data to new buckets or I should initiate `recovery on read` manually?
Best regards,
Anatoly Popov.
Evgeniy Polyakov
Mar 21, 2016, 11:38:06 AM3/21/16
to Anatoly Popov, reverbrain
Hi Anatoly
21.03.2016, 18:32, "Anatoly Popov" <anatol...@gmail.com>:
maintained by elliptics, but it is heavily internal structures which you will likely never see in your production deployment,
it is close to 150 bytes per file.
You will need 2 * blob size for eblob to run defragmentation though, but you can run defragmentation on the separate device.
> 2. Can I update buckets metadata without losing data? E.g. for adding new groups to buckets? Will backrunner replicate existing data to new buckets or I should initiate `recovery on read` manually?
Do you mean per-bucket metadata? I.e. bucket name, list of replica groups, acl and so on?
When you add new groups into bucket, data will not be recovered into that new replica until either read-recovery fires up (i.e. some client reads data),
or you run 'dnet_recovery' tool where you will have to specify newly added group.
21.03.2016, 18:32, "Anatoly Popov" <anatol...@gmail.com>:
> 1. How much space should I reserve for metadata per file? Are there any estimates?
What do you mean by 'metadata per file'? Neither elliptics nor backrunner stores per-file metadata, there are index headers
maintained by elliptics, but it is heavily internal structures which you will likely never see in your production deployment,
it is close to 150 bytes per file.
You will need 2 * blob size for eblob to run defragmentation though, but you can run defragmentation on the separate device.
> 2. Can I update buckets metadata without losing data? E.g. for adding new groups to buckets? Will backrunner replicate existing data to new buckets or I should initiate `recovery on read` manually?
When you add new groups into bucket, data will not be recovered into that new replica until either read-recovery fires up (i.e. some client reads data),
or you run 'dnet_recovery' tool where you will have to specify newly added group.
Anatoly Popov
Mar 21, 2016, 11:51:12 AM3/21/16
to reverbrain, anatol...@gmail.com, z...@ioremap.net
Hi, Evgeniy.
1. I thought backrunner stores some metadata-per-file too in groups, specified for metadata. It seems, that I'm wrong.
2. Yep, I mean per-bucket metadata.
Thank you for answers.
Best regards,
Anatoly Popov.
Evgeniy Polyakov
Mar 21, 2016, 11:53:22 AM3/21/16
to Anatoly Popov, reverbrain
21.03.2016, 18:51, "Anatoly Popov" <anatol...@gmail.com>:
> 1. I thought backrunner stores some metadata-per-file too in groups, specified for metadata. It seems, that I'm wrong.
Backrunner only stores per-bucket metadata in metadata groups (that's why metadata groups are needed), there is no per-file metadata.
Reply all
Reply to author
Forward
0 new messages