service providers who support bup backups

[Jonathan Polak]

Gentlemen 

Can you share what service provider you're using for offsite storage of your BUPs? I've looked at the usual suspects (rsync.net / crashplan / tarsnap ) but none offer BUP. My VPS provider just went out of business - before I go and spin up another I would be grateful  to know what the community has been using.

Jon

[Sean Bartell]

Jonathan Polak on 2015-06-07:
>*Can you share what service provider you're using for offsite storage of
>your BUPs? *I've looked at the usual suspects (rsync.net / crashplan /

>tarsnap ) but none offer BUP. My VPS provider just went out of business -
>before I go and spin up another I would be grateful to know what the
>community has been using.

If you have a local bup repository, all you have to do is backup the files in
it to a remote service. You can exclude *.idx, *.midx, and *.bloom files. The
big *.pack files are never modified or deleted (except by the experimental bup
gc), so you don't even need incremental update.

My bup-bundle patch[0] is intended to make this even easier--you get a new
incremental bundle file every time you run it, which you just have to upload to
a service. Don't use the version I posted there, though, because it has a bug
and stores lots of unnecessary data in the bundles. I'm using a WIP fixed
version to create bundles, which I encrypt and send to Amazon S3, a cheap
dedicated server, and some DVD-Rs.

If you don't want to have a local bup repository, it *might* work to use
s3fs-fuse for Amazon S3, or another remote filesystem that works with a web
service.

[0] http://thread.gmane.org/gmane.comp.sysutils.backup.bup/5018

[Jonathan Polak]

Hello - thank you for your reply. 

Yes, I was pushing the backup directly offsite by mounting sshfs -- (which I've determined is not a good idea unless you are on the same LAN with the remote server, or have some other way of ensuring the connection is low latency) 

[Ben Kelly]

I've been using Google Drive via the unofficial command line client[1]. It's a bit clunky but it gets the job done; my bup cron job runs 'drive push' after each backup.

I had a look around for backup providers that are bup-compatible, but they all insist you use their own software, there's no "unlimited backup solution" that's just "you give us money, we give you an sftpd to connect to".

[1] https://github.com/odeke-em/drive

[Jonathan Polak]

Ben

What is the latency and bandwidth of your upstream connection to Gdrive? The documentation seems to imply that there is little IO scheduling - the client supports resumes in uploads? 

Running one's own VPS (on S3 / azure / etc) is the singular 'non clunky' solution. However - the problem with one's own VPS is that, things go wrong, and a VPS does not provide the same availability, data integrity assurance, security, longevity, etc.. as a real offsite backup provider  (crashplan / tarsnap / rsync / etc) 

There is a gap for offsite-cloud-backup that supports sftp, sshfs, etc, with the SLA of established providers?

--
You received this message because you are subscribed to a topic in the Google Groups "bup-list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/bup-list/pBQLHwIwyu0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to bup-list+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ben Kelly]

Latency ~8ms, upstream bandwidth 10Mbps.

The client will resume interrupted pushes (i.e. if you run it again it'll only push the files that didn't make it last time), I don't know if it will resume partially uploaded files or just nuke them and start over. My net connection is slow but reliable so I've never had to test that.

[Jonathan Polak]

Here is a potential substitute to the Gdrive hack. 

https://serversub.com/data-security.php

The prices are very reasonable -- servers are in Quebec.  

[Matthieu Rakotojaona]

Excerpts from Ben Kelly's message of 2015-06-08 06:49:44 -0700:

> there's no "unlimited backup solution" that's just "you give us money,
> we give you an sftpd to connect to".

rsync.net fits the description here, it does:

"rsync / sftp / scp / git-annex / duplicity / rdiff-backup / unison"

[Gabriel Filion]

ooh. I didn't know they supported sftp/scp. that could be pretty
interesting for bup repos.
however since bup doesn't support storage encryption yet, one would have
to "manually" encrypt before sending out to them (because ... don't
trust any 3rd party with your data)

--
Gabriel Filion

[Ben Kelly]

$30/mo for 500GB doesn't sound very reasonable when the competition is $5/mo for infinity GB. :(

rsync.net works out to $62/mo for my current backup set and will only get worse as I back up more stuff, although once rm/gc are working I could perhaps prune my old backups some.

[Jonathan Polak]

Humm... seems obvious but 

http://en.wikipedia.org/wiki/Comparison_of_online_backup_services

here's a pullout

Provider	Win.	Linux	Mac	Android	BlackBerry	iOS	Encrypted storage	Zero knowledge	Secure Key Management
AltDrive	Yes	Yes	Yes				Yes	Optional[2]	?
ASUS WebStorage	Yes	Yes (only 32 bit)	Yes	Yes		Yes	No	No	?
Barracuda Backup Service	Yes	Yes	Yes				Yes[4]	No	?
Bitcasa	Yes[5]	Yes	Yes (Only 64-bit)	Yes		Yes	Yes	Bitcasa cannot decrypt data, but uses convergent encryptionref_152ref_15215 [6]	?
CloudMe	Yes	Yes	Yes	Yes	Yes, via Storage Made Easy client	Yes	No, but 3rd party apps available[citation needed]		?
Copy	Yes	Yes	Yes	Yes	No	Yes	Yes	No	?
CrashPlan	Yes	Yes	Yes	Yes		Yes	Yes[27]	Yes[28]	Yes
Diino	Yes	Yes	Yes				No[36]	No	?
Dropbox	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No,[37] but 3rd party apps available[38]	?
Dropmysite	Yes	Yes	Yes				Yes	No	?
Druva Insync	Yes	Yes	Yes	Yes	Yes	Yes	Yes[39]	Optional[39]	?
Egnyte	Yes	Yes	Yes				Optional[40]	Optional[40]	?
ElephantDrive	Yes	Yes[42]	Yes	Yes[43]			Yes	Optional (off by default) [44]	?
EVault	Yes	Yes	Yes					Yes[citation needed]	?
Handy Backup	Yes	Yes	No	Yes			Yes	Yes	?
IASO Backup	Yes	Yes	Yes			Yes[59]	Yes[59]	Yes	?
iCloud	Yes	Yes, via Storage Made Easy client	Yes	Yes, via Storage Made Easy client	Yes, via Storage Made Easy client	Yes	Yes	No	?
Infinit	Yes	Yes	Yes				Yes[60]	Optional[60]	?
Infrascale	Yes	Yes	Yes	Yes	No	Yes	Yes	Optional	Optional
Jungle Disk	Yes	Yes	Yes		No		Yes[65]	Yes[65]	?
MEGA	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	?
Memopal	Yes	Yes	Yes	Yes	Yes	Yes	Yes[71]	No[72]	?
Mozy	Yes	Yes	Yes	Yes	No	Yes	Yes[74]	Optional[74]	?
OwnDrive(ownCloud)	Yes	Yes	Yes	Yes		Yes	Yes	Yes	?
pCloud	Yes[82]	Yes[82]	Yes[82]	Yes		Yes	Yes[83]	Yes[83]
SpiderOak	Yes	Yes	Yes	No, the android client is only capable of accessing spideroak cloud data	Yes, HTML5 client in Q4 2012, Android version from Amazon store for Blackberry 10	Yes	Yes[84]	Yes[84]	?
Tarsnap	Using Cygwin	Yes	Yes	No	No	No		Yes[91]	?
TeamDrive	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	?
Unitrends Vault2Cloud	Yes	Yes	Yes				Yes[97]	No	?
Wuala	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes[101]	?
Yandex.Disk	Yes	Yes[108]	Yes	Yes		Yes	No[109]	No	?
Zetta.net	Yes[111]	Yes	Yes				Yes	No	?

I filtered out all providers without linux support.

[Jpo...@1datapoint.com]

Mattheiu,

Thank you for the Rsync tip. They do support sshFS, SFTP, SCP. I just mounted 50GB in sshFS -- it's FAST. I emailed them and they have a special rate for code-contributors etc. It's very substantial.

basically it makes Rsync the most cost-effective as well. Happy to elaborate by email.

Jon