Increasing storage space
164 views
Skip to first unread message
James Royal-Lawson
Sep 16, 2016, 5:20:49 PM9/16/16
to s3backer-devel
Hi everyone,
I set up s3backer on my ubuntu server a few years ago and created a 250gb partition for storing backups on. I've filled that up now and I've been trying to find out how I can make it bigger. I've not found any instructions or information that I feel confident in following.
I wrote a blog post back then detailing what I did (mainly as a way of noting/remembering what I did and why!) which you can see here
I start s3backer with the following...
s3backer --blockSize=1024k --size=250g --listBlocks --vhost --ssl --timeout=250 --blockCacheThreads=3 --maxUploadSpeed=35k --accessFile=/etc/passwd-s3backer --prefix=s3 buggles /s3
I'm running an ext4 file system on top of that mounted as follows
mount -t ext4 -o loop /s3/file /backups
As far as I understand, I need to increase the size of the s3backer file and then increase the size of the ext4 partition. Is that correct? How do I do that without losing data?
/James
Archie Cobbs
Sep 16, 2016, 5:32:16 PM9/16/16
to s3backe...@googlegroups.com
At the s3backer level, increasing the total size of the s3backer "file" should be easy. Just restart s3backer with a new --size parameter (but the same --blockSize of course).
I've never actually done this. In any case you can easily test that this first step works: just add the --readOnly flag to s3backer, and then mount your filesystem read-only on top of that. Then verify you can still see all your files OK, etc.Now this "should" all work but I'm only theorizing. You should test it all out on a throwaway filesystem first.
Report back and let us know how it works.
-ArchieReport back and let us know how it works.
--
You received this message because you are subscribed to the Google Groups "s3backer-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to s3backer-devel+unsubscribe@googlegroups.com.
To post to this group, send email to s3backer-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/s3backer-devel.
For more options, visit https://groups.google.com/d/optout.
--
Archie L. Cobbs
James Royal-Lawson
Apr 22, 2017, 8:47:50 AM4/22/17
to s3backer-devel
I ran s3backer with --size=500g (instead of the usual --size=250g). I also included the --readOnly flag (I tried it later without, but with the same result)
Even though i'm specifying the size, s3backer is autodetecting the size and complaining that the filesystem size and s3backer file size don't match and exits... so I can't run resize2fs as s3backer isn't running.
root@srvr:/var/log# s3backer --readOnly --blockSize=1024k --size=500g --listBlocks --vhost --ssl --timeout=250 --maxUploadSpeed=100k --blockCacheThreads=6 --blockCacheFile=/tmp/s3backerCache --accessFile=/etc/xxx --prefix=s3 srvr /s3
s3backer: auto-detecting block size and total file size...
s3backer: auto-detected block size=1m and total size=250g
s3backer: error: configured file size 500g != filesystem file size 250g
Then I did a bit of checking and spotted in s3b_config.c that s3backer will return a warning rather than an error if I include the --force flag.
So I tried again with
s3backer --force --blockSize=1024k --size=500g --listBlocks --vhost --ssl --timeout=250 --maxUploadSpeed=100k --blockCacheThreads=6 --blockCacheFile=/tmp/s3backerCache --accessFile=/etc/xxx --prefix=s3 srvr /s3
And I got further!
But before I could extend the filesystem with resize2fs I had to run:
e2fsck -f /s3/file
It took about 20 mins or so for it to complete recoverig the journal - I was getting worried! But after that it progressed with the passes quite nicely.
Once that was complete I could do
resize2fs -p /s3/file
and then remount the enlarged filesystem.
/dev/loop0 493G 226G 242G 49% /backups
/James
If that works, then unmount the top level filesystem, restart s3backer without --readOnly, and then you should be able to run resize2fs to enlarge the ext4 filesystem.At the s3backer level, increasing the total size of the s3backer "file" should be easy. Just restart s3backer with a new --size parameter (but the same --blockSize of course).I've never actually done this. In any case you can easily test that this first step works: just add the --readOnly flag to s3backer, and then mount your filesystem read-only on top of that. Then verify you can still see all your files OK, etc.Now this "should" all work but I'm only theorizing. You should test it all out on a throwaway filesystem first.-Archie
Report back and let us know how it works.
On Wed, Sep 14, 2016 at 2:20 PM, James Royal-Lawson <ja...@ccl4.org> wrote:
Hi everyone,I set up s3backer on my ubuntu server a few years ago and created a 250gb partition for storing backups on. I've filled that up now and I've been trying to find out how I can make it bigger. I've not found any instructions or information that I feel confident in following.I wrote a blog post back then detailing what I did (mainly as a way of noting/remembering what I did and why!) which you can see hereI start s3backer with the following...s3backer --blockSize=1024k --size=250g --listBlocks --vhost --ssl --timeout=250 --blockCacheThreads=3 --maxUploadSpeed=35k --accessFile=/etc/passwd-s3backer --prefix=s3 buggles /s3I'm running an ext4 file system on top of that mounted as followsmount -t ext4 -o loop /s3/file /backupsAs far as I understand, I need to increase the size of the s3backer file and then increase the size of the ext4 partition. Is that correct? How do I do that without losing data?/James
--
You received this message because you are subscribed to the Google Groups "s3backer-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to s3backer-deve...@googlegroups.com.
To post to this group, send email to s3backe...@googlegroups.com.
Visit this group at https://groups.google.com/group/s3backer-devel.
For more options, visit https://groups.google.com/d/optout.
--Archie L. Cobbs
Reply all
Reply to author
Forward
0 new messages