Where is everyone going to migrate to?

[Michael Berman]

I know John has given us a year (thank you John!), but I want to start
looking. My backups are far from perfect now (nothing to do with DSU
- issues with my scripts, etc.), so I figure it'll take me a while to
get something that is actually working correctly with another service
if it's anything more than changing the rsync destination. I've
checked out a few already, but I'm either finding Window's only
clients or rates that are so high (i.e. rsync.net) that I could get a
dedicated server myself and still come out ahead.

I have a pretty decent pipe (25/25Mb and that'll likely to go up) and
I don't have excessively large files that frequently change, so I
could probably make a ftp based solution work if I had to (but it
wouldn't be my first choice).

I need some space. To do it right between pictures, video, etc., I'd
really need around a TB and room to grow beyond that.

So, has anyone found any great replacements?

Michael Berman

[Rusty Nejdl]

I went with DSU exactly because I couldn't find anyone else that offered anything like this.  I would be willing to trade backup services with someone else if interested.  I am in the even worse spot of being FreeBSD which is like the bastard stepchild of linux.  Ping me offline of you are interested.

Rusty Nejdl

[niknah]

I have been using idrive on windows, which has a similar price to dsu.  But I don't like the software, it struggles with mapped network drives and some times gets stuck and doesn't warn you.

Here's something similar with shell access for $102 per month for 100gig
www.microtronix-tech.com/cart.php?a=confproduct&i=0

Google storage and Amazon S3 are about $144 per year for 100gig storage, that's not including the network costs.
But Amazon S3's uploads are free.

A 2tb drive is around $120 these days, it'd be good if someone can start an alternative service.  I only have adsl so I can't do that.

[Michael Berman]

The whole idea of DSU is offsite backups. I have a safe deposit box
at a bank (got it for free because of a loan) with the theory being to
keep backups there. In the span of several years, I've taken data
there once. It has to be easy and at least near automated for it to
work for me. I'm guessing that is true of others as well. I'm pretty
sure we are mostly trying to backup personal data. It has to be easy
and reasonably cheap.

I've looked at Amazon before. Just not viable for any more than 50GB
of data. Just way too expensive. All the extras (bandwidth, server
charges, dealing with the S3 interface, etc.) add up. For Google,
unless things have changed, the interface just doesn't work well for
this type of purpose. You can dump some tarballs there, but it sucks
for incremental backups, etc. (I guess one could do incremental
tarballs?).

For $50 o $60 a month, there are numerous places where one can get a
dedicated server with a 160GB (or better) hard drive. Honestly, I
need more space than that and have less of a budget.

I have a friend that is about 10 miles from me that would probably let
me keep something at this house, but it would have to be very low
power, I'd still have to maintain it, and I was really liking the idea
of something that was further away from me.

It won't help me, but I'm possibly contemplating the idea of trying to
setup some sort of service. No promises. I can get unmetered 150
down/35Mb up service where I am for about $250 a month (about $130 for
the lowest plan with a static IP). It's something I'm thinking
about....

John

How much bandwidth have we been burning through? Can you give some
high level anonymized info on the subscriber base? i.e. 50
subscribers on the 100GB plan, 20 at 400GB, etc. Any comments about
hardware or what it has taken to run this service?

Michael Berman

[John - DataStorageUnit - Owner]

This may work for some of you guys:  http://www.crashplan.com/

I'll dig around some more and see what I can find.

[knnniggett]

I've got about ~550GB of data and most sites would charge me way too much. Consequently, I'm kicking around a couple of ideas (crashplan looks interesting), but at the moment I am leaning towards a peer-to-peer backup with a nearby friend.

On Tuesday, July 3, 2012 4:15:31 PM UTC-5, phildobbin wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/07/2012 21:52, John - DataStorageUnit - Owner wrote:

> This may work for some of you guys:  http://www.crashplan.com/
>
> I'll dig around some more and see what I can find.

Hi, John.

Congratulations on the promotion. Hope everything works out.

Thanks for the sterling service. You'll be missed.

All the best,

  Phil...

- --
But masters, remember that I am an ass.
Though it be not written down,
yet forget not that I am an ass.

        Wm. Shakespeare - Much Ado About Nothing


-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: §auto-key-locate cert pka ldap hkp://keys.gnupg.net
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJP82DsAAoJEKpMeDHWT5ADq2QIAIIICeIyeZpYgmuG7le6yLIo
JT+7IKWVNXHlkUF3bYjphZP5QsqLluxZrI2stgY9pW9TL3TUGMBTs5QtGTXQCwKa
zL8NoS8x6aBjkxlSxl1nIYa4L2RUJZNH8i2GzAiUd5enCuIWEPL0vuTNLDP+FHit
/7QbkiAIYNhSz0iZiXXeEPJpJKTFSQF5ed65z7n2X57P0uaBuHR1eta9iVc1ai2R
iAeQSq9DH1aaXwk6BIl1aDlHWjgRV5PhQxWhh8dQpR9IFE8dr1NmW9Q77UW26Z6l
MCODZzT4uHPeXICFVjFmCmZTzsLRdT4fx6iV8YZEKKhNW9pCZIyW9R6nRd7uPYg=
=Qa8F
-----END PGP SIGNATURE-----

[John - DataStorageUnit - Owner]

Some folks were asking questions about taking over this service. 

Bandwidt Requirements:

We were at about 136 active subscribers at the busiest point.  With that number, I never saw more than 20 folks connected at a given time.  That being the case...20 folks connected simultaneously did often bring things to a crawl.  My server was hosted in a nearby call-center's DataCenter.  It was fed by a 20 Mbps circuit.  20 up and 20 down.

Server Hardware was definitely nothing high-end.  I had a pair of identical 3U storage servers...(supermicro brand).  They each held 15 3TB SATA drives.  Hitachi 7200rpm. 
Each server has 8 GB's of Ram & dual Xeon Processors.  Not sure what clock speed they are.  They're single core as well....so they're obviously old.

I'd highly recommend Raid6.  My hardware Raid wasn't capable of Raid6, so the next best choice for me was Raid 50....which was basically 3 Raid 5's striped across each other.  5 drives in each Raid 5.  A lot of folks are pro software Raid, but I'm not. 

I started this service out of my home.  Initially I was running it on a 20meg cable connection.  I don't think I could have kept it going very long on just that.  You really need a dediacted connection if you plan on supporting more than 20-25 clients.  Can you start out at home?  Yes, but it's a big hassle to uproot and start over in the middle of aquiring customers

If I had it to do over, or had the time/money to do some things different...what would I have changed:
    * I would have never offered unlimited storage.  That put me in a bind right from the start.
    * Do not spend the money you make.  (LOL, I knew this when I started...but it's hard to avoid).  You got to be disciplined here.  You'll need to re-invest in the business over time.
             -  You'll hit a brick wall eventually.  Probably in about 5 years.
    * Don't let folks pressure you into installing packages that don't benefit the majority. 
       Particularly on production servers.  If you have a "day job"...you'll end up in a situation where you'll be supporting a multitude of client types. 
       Keep it simple.  Unless of course...you have unlimited time & a lot of help.   :)

    * I think my largest client bought about 500 GB's of storage.  That being the case...a pair of 30 TB storage servers may not be a good idea. 
        You may be much better served with a couple 1U servers 4 Drives in Raid 6.  You can get 4 TB sata drives now.
          That'd give you about 3.5 TB's of usuable storage in Raid 6. 
            @ 100 GB's per client...you could have 30 + clients on that hardware.
               You'd give up the ability to have 200 clients on a single server...but you forgoe the difficulty associated with managing such large Data LUN's.
                    This also eliminates any issues sticking with a common and widely supported Files System...like EXT4.  I had to use XFS...which is great, dont' get me wrong.
                       but I ran into some quirky issues regarding quotas with XFS.                     
                  I don't know...this concept is debatable.  I'm thinking in terms of starting small and keeping cost down here.

    *  You're definitely going to want lights out management if you can't just up and run to the server when there's an issue.
         A single port KVM would be the least I think you could live with.  Ideally...something like HP's ILO would be preferred.
  

   * Don't run off of port 22...LOL, I knew better than that.  But once I had it rolling, I didn't want to disrupt you guys. 
       I have a package that bans you after failed logons...but that's not a total solution.

    *  Distribution choice....  I went with Debian intitially...then Ubuntu.  Not a lot of difference in those.  I went with those because it's what I'm familiar with.
          I sometimes think they're both a little bloated and I felt it on older hardware...I considered Arch toward the end.  But I'm not comfortable enough with it just yet.
             Debian/Ubuntu both have tons of support docs online.  Easy to google and find answers to issues you run into along the way.
              Ultimately I'd lean heavily toward what you know best.

...if there's any other questions that you guys have, let me know.  I could go on and on here I think.

[John - DataStorageUnit - Owner]

Another option you guys might find appealing... "Google Drive"  .  No official Linux Client yet...but I believe this project called "grive" should be very promising.  You can read about it here:  http://www.lbreda.com/grive/  .  I've tested it on Arch Linux & Ubuntu 12.04.  Development seems to be very active and improvements are coming pretty often.  It's actually the solution I've went to now.  I also tested Crashplan...and it worked well.  The backup client for Crashplan is really good in linux.  If you like "Traditional" backup...one in which you just send your files offsite and you don't retrieve them unless a disaster hits...you'll probably like it.  But my 'wants' were more in line with cloud accessible storage, that you can sync your local files off-site to, and are universally accessible by various methods...(windows client, web, smartphone).  Google drive suits me better for that reason. 

Why Google Drive over Dropbox?  Well, it's a lot cheaper...especially if you have bought storage in the past.  I got grandfathered into having 80 GB's annually for $20.  I think the starting rate today is $4.99 a month for 100 GB's.

There's another Google Drive client called "Insync".  I haven't tested it, some say it's better.

On Monday, July 2, 2012 10:14:11 PM UTC-4, Michael Berman wrote:

[Peter Heitman]

I ended up switching to crashplan+. I get unlimited space to backup up to 10 computers. I can use their mobile apps to check on the backup status and can download any file to my mobile device. So far I have been quite satisfied.

[Phil Dobbin]

Peter Heitman wrote:

> I ended up switching to crashplan+. I get unlimited space to backup up
> to 10 computers. I can use their mobile apps to check on the backup
> status and can download any file to my mobile device. So far I have been
> quite satisfied.
>
> On Sep 6, 2012 4:37 PM, "John - DataStorageUnit - Owner"

> > wrote:
>
> Another option you guys might find appealing... "Google Drive" .
> No official Linux Client yet...but I believe this project called
> "grive" should be very promising. You can read about it
> here: http://www.lbreda.com/grive/ . I've tested it on Arch Linux
> & Ubuntu 12.04. Development seems to be very active and
> improvements are coming pretty often. It's actually the solution
> I've went to now. I also tested Crashplan...and it worked well.
> The backup client for Crashplan is really good in linux. If you
> like "Traditional" backup...one in which you just send your files
> offsite and you don't retrieve them unless a disaster hits...you'll
> probably like it. But my 'wants' were more in line with cloud
> accessible storage, that you can sync your local files off-site to,
> and are universally accessible by various methods...(windows client,
> web, smartphone). Google drive suits me better for that reason.

[snip]

I ended up going with Duplicity & S3 & also Google Drive (thanks for the
heads up on Insync, John).

I'm using five different distros so my backups, local & remote, use a
mixture of software; apart from the two mentioned above, I also use
mondoarchive, fwbackups, deja-dup, luckybackup & carbon copy cloner.
Took a while to set up but now they're all run by cron so I can just
forget about them (well, the nightlies at least)

Cheers,

Phil...

[Martin Larsen]

I have looked at Grive and Insync and I have a couple of questions:

1. Can they be used for backing up anything or just "documents (.doc, .xls, .ppt, etc)? In other words, any file, any size?

2. Do they use a rsync like mechanism (delta) or will they upload the whole file for even minor changes?

[John - DataStorageUnit - Owner]

On Monday, September 17, 2012 6:13:26 AM UTC-4, marlar wrote:

I have looked at Grive and Insync and I have a couple of questions:

1. Can they be used for backing up anything or just "documents (.doc, .xls, .ppt, etc)? In other words, any file, any size?

                   - yes, anything I believe.  Individual file size limit is 10 GB's

2. Do they use a rsync like mechanism (delta) or will they upload the whole file for even minor changes?

                  - Grive sync's at a block level with the latest version to the best of my knowledge. 


Overall It can be buggy.  I ran into a lot of errors over the course of about a week to get my 90GB's sync'd up...but now that it's sync'd...it's pretty sound.

[Phil Dobbin]

John - DataStorageUnit - Owner wrote:
>
>

I had a lot of problems with Insync on Ubuntu 12.04 & ended up not using it.

I'm hoping that Amazon finally release an API for Cloud Drive as that's
by far the cheapest plan & is basically S3. But unfortunately, it looks
unlikely.

I've also stored all my music, photos, that kind of thing on Amazon's
Glacier which is perfect if you don't need regular access to them (i.e.
as a stored, long term backup). It's intended to be a replacement for
large companies tape drives & it's also amazingly cheap.

[Martin Larsen]

Hi,

Now a few months have passed and I wonder about your conclusions
finding an alternative online backup service.

What are you using, if any?

I am still on DSU but looking more intensively for an alternative.

Regards,
Martin

[Phil Dobbin]

I'm using Amazon S3 for daily backups, Amazon Glacier for vault-like,
long term stuff (MP3s, photos, etc) & a mixture of Mondoarchive &
Clonezilla for bare metal recovery.

I use Deja-Dup & duplicity as clients for the Amazon S3 stuff. No
complaints so far after six months.

Amazon's prices have dropped substantially recently too.

Cheers,

Phil...

--
currently (ab)using
CentOS 5.8 & 6.3, Debian Squeeze & Wheezy, Fedora Beefy & Spherical,
Lubuntu 12.10, OS X Snow Leopard & Ubuntu Precise & Quantal

[Michael Berman]

I'm looking at 3 different options right now:

1.  Cheap server.  I found a couple of places that sell relatively cheap storage servers.  Looking at http://www.ovh.ie/dedicated_servers/kimsufi.xml right now.  Most are unmirrored dedicated hardware.  A few have VPS on RAID, but are really too much money.  Honestly, the cheap dedicated box is pushing my budget for backups.

2.  I have a couple of friends that I might swap space with.  This is really the direction I'm hoping to go, but more logistics to work out, etc.  Also, the primary friend that I could do this swap with is only about 10 miles away.  I was hoping for further.

3.  Sneaker net.  I have a free safety deposit box through my bank.  Thinking about picking up a few hard drives and a couple of docking stations to cycle through drives.  It has the disadvantage of being a much bigger pain (as I rarely go to the bank).

I was looking at hosting a box at home, but I just don't think it is going to work out.  Have been too busy with the kids, real job, etc.

Michael

--
You received this message because you are subscribed to the Google Groups "DataStorageUnit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datastorageun...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[knnniggett]

I am using Crashplan+ Unlimited.
http://www.crashplan.com/

Runs natively in Linux as a daemon
only $60 a year
has unlimited storage (they say they won't ever change that)
Allows other Crashplan clients to backup to you (this feature is free). 
No file size restrictions

[Michael Berman]

I really need to play with crashplan.  No way I can touch the price and it looks like it would be a lot less maintenance that maintaining yet another server, etc.

Michael

--

[Phil Dobbin]

On 02/03/2013 08:35 PM, Michael Berman wrote:

> I really need to play with crashplan. No way I can touch the price and
> it looks like it would be a lot less maintenance that maintaining yet
> another server, etc.

This year, I'm going to invest in a couple of HP ProLiant N40Ls: 8 TB of
storage (SATA), 16GB's of RAM, Raid 1 & runs Linux no problem.

You can get one with a 250GB drive & 2 GB RAM for £159 at Amazon UK.
It'll be a bit of a layout to get them up & running but they've a pretty
solid reputation, use very little power & are very quiet.

I'll just carry on using Amazon for the offsite stuff but with two of
the HPs running Raid 1 I'll be pretty safe anyway.

[Peter Heitman]

I have been happy with Crashplan. I got the multi computer unlimited plan. I am backing up 3 computers, two windows 7 and one Linux file server. Pretty cool.

[Fish Kungfu]

I also use CrashPlan.  Another I'm considering trying is Cyphertite, https://www.cyphertite.com/.

...Fish

2013/2/3 Peter Heitman <pe...@heitman.us>

[Martin Larsen]

Crashplan seems to be a popular choice. Does it utilize a rsync like
transfer mechanism, ie. with delta packages? Also, can it run on a
headless Linux server?

Ideally I would prefer a backup service that allowed me to use rsync
(just like DSU) as I find it very effective and versatile. For
example, with rsync I can maintain several backup snapshots using
hardlinks so that only changed files are in fact stored. This allows
me to go back in time and get a previous version of a file if needed.

But in reality I could use two backup services, one for my main data
(documents, source code etc) which will never add up to more than a
few GB and would probably be within the free limit of many services,
and then something like Crashplan for my digital images and videos
which adds up to 100+ GB. It is only the main data I want to keep
different versions of, it is not necessary for my media files. For
those I only need to mirror my local storage.

[Martin Larsen]

I have found this one that looks promising:
http://www.onlinestoragesolution.com/signup.html

They have unlimited backup storage, you can use rsync as well as FTP,
Webdav, Rsync, CIFS etc. meaning that you can attach your storage
space as a network drive.

Their pricing is very affordable indeed (comparable to DSU), $35 for
year, $45 for two years.

It actually looks a little to good to be true!

Does anyone know this service?

[Martin Larsen]

> It actually looks a little to good to be true!

Well, it probably is. I just googled their name and found loads of
negative comments. This look like something to stay far away from.

[knnniggett]

Running headless is an unsupported feature which I have not tried myself:
http://support.crashplan.com/doku.php/how_to/configure_a_headless_client

By default, Crashplan saves every version of every file you ever upload to them. You can change this if you want. Additionally, Crashplan supports backup sets. You could try creating two. One set would remember previous file versions and the other set would not.

[niknah]

I'm using crashplan headless.  You can also run the gui remotely, login with ssh -X  from a computer with X windows and start the gui and it'll appear on your desktop.

The software is free, you don't need to pay if you don't use their storage.

If you're not in the USA, try logging in via a proxy in the USA and get the USD prices.  It was cheaper in USD for me.

Yes, it only sends the changes of the file.  They have continuous back up, unlike rsync it backs up the file a few minutes after you change it.

http://support.crashplan.com/doku.php/articles/how_backup_works

[Jim Lynch]

I went the most expensive way. I bought a condo at the beach,
installed a cable modem and a computer and now I have my own dedicated
server. Recurring costs at $50 a month and a big investment
initially.

:)

Jim.

On Sun, Feb 3, 2013 at 5:20 AM, Martin Larsen <mar...@larsen.dk> wrote:

[Martin Larsen]

On 5 February 2013 00:14, niknah <nik...@gmail.com> wrote:

I'm using crashplan headless.  You can also run the gui remotely, login with ssh -X  from a computer with X windows and start the gui and it'll appear on your desktop.

Does crashplan preserve Linux permissions and ownership? 

 

[niknah]

Yes.  Permissions, owners.  But not facl information if you use that.

Saves last modified time too.  But not access time or change time.

[Martin Larsen]

On 3 February 2013 12:56, Phil Dobbin <phild...@gmail.com> wrote:

I'm using Amazon S3 for daily backups, Amazon Glacier for vault-like,
long term stuff (MP3s, photos, etc) & a mixture of Mondoarchive &
Clonezilla for bare metal recovery.

I use Deja-Dup & duplicity as clients for the Amazon S3 stuff. No
complaints so far after six months.

Amazon Glacier is incredibly cheap. But how do you backup to Glacier? As far I as can see, duplicity does not support it.

Thanks,

Martin

[Phil Dobbin]

On 12/02/2013 15:31, Martin Larsen wrote:

> On 3 February 2013 12:56, Phil Dobbin <phild...@gmail.com

The way I do it is from my Macbook Pro. I basically just backup my Music
& Pictures directories transferred from a Samba share. They're large &
have been collected over many, many years & so are priceless to me.

I use a GUI app called Arq <http://www.haystacksoftware.com/arq/> & it's
Mac only but there is an API to roll your own with Glacier. I'm just
being lazy & using the tools I have at my disposal.

Cheers,

Phil...

[pedorro]

I'm using three cloud services (that are actually all free) that are filling the DSU gap and actually provide more that DSU did.  I'm now using Picasa/G+ for my photos.  Photos smaller than 2048 on the longest edge and videos shorter than 15min don't count towards the free account quota.  So all my pictures & video are backed up and available online.  I'm using Google Music to back up & stream my music collection.  The free account includes 20,000 audio files and I doubt I'll ever hit that.  And I'm using Ubuntu One + encfs for everything else.  The free 5GB of cloud storage is plenty when I don't include my photos, videos, or music.

That's my 2c :)

[niknah]

Thanks for the tips!  Google are so generous!  But they won't let me use google music cause I'm not in the USA.

Here's another freebie I found recently, they give you 50gb per email address...

https://www.box.com/signup/o/dell_50gb_give_get

Here's how to mount it and do rync...

http://linuxfordummies.org/mount-your-box-com-account-in-linux/

[Martin Larsen]

On 13 February 2013 22:48, niknah <nik...@gmail.com> wrote:

Thanks for the tips!  Google are so generous!  But they won't let me use google music cause I'm not in the USA.

No problem. You just sign up via a US proxy (eg. you can get a free trial of Unblock US). All subsequent access works normally without a proxy etc. so you don't need Unblock US more than once. Unless you need other services, of course.

I have used Google Music half a year now. Recently they added a music recognition feature so don't actually have to upload your music. If Google recognizes the music files, they are instantly added to your account.

[Martin Larsen]

Hi,

Just something funny I wanted to share with you.

Because of the ways cookies track our browsing the internet, ads with Crashplan and other backup solutions popup everywhere I go on the net these days while searching for the perfect alternative to DSU.

So, while reading about the russion meteor, I saw this:

I think that is quite funny -- Crashplan :-)

Martin