future proof storage

[Marcin Ciesielski]

Hi everyone,

I am thinking of using s3backer to store my photos. We are talking almost 8TB at the moment and it will grow.

Would it be possible to create two 4TB volumes on S3backer and combine them on LVM?

then if needed I could add the third volume and add to the same pool.

another option is to create future-proof volume. Something like 50TB or maybe 100 and pray it will always be enough.

Any thoughts?

[Marcin Ciesielski]

It looks like I missed the possibility to simply grow s3backer by forcing new --size parameter.

I understand I have to fsck the s3backer file after that.

Would this download the whole content of s3backer? I would obviously try to avoid it.

Marcin

[Archie Cobbs]

On Tuesday, July 10, 2018 at 4:30:54 AM UTC-5, Marcin Ciesielski wrote:

It looks like I missed the possibility to simply grow s3backer by forcing new --size parameter.

That's step one...

Back to the original question, it's worth asking what is the downside of configuring a humongous size in the first place, more than you'll ever need?

The answer depends entirely on the upper layer filesystem you're using...

• All filesystems have a maximum size - is it big enough?
• How big is the filesystem meta-data overhead when the size is very large?
  

I understand I have to fsck the s3backer file after that.

Actually you would need to resize the upper filesystem first. E.g., for ext4 see resize2fs(8).

 

Would this download the whole content of s3backer? I would obviously try to avoid it.

I believe resizing does not do a whole lot of I/O but that of course depends on the filesystem and resize tool

Running fsck probably does a lot more I/O, but again how much is dependent on the filesystem, and things like the average file size (fsck reads filesystem meta-data but usually not actual file data).

One of the nice features of s3backer is the simplicity of the service it performs. Unfortunately, that means most practical questions must pass through to the upper filesystem to be answered :)

-Archie

[Marcin Ciesielski]

Thank you Archie

loads to learn