Reminder: Meeting tonight on Backups

[Seth House]

Hey, all. Sorry for the late reminder.

Tonight I'll present on backups; what to backup and how to backup, and
how to recreate Apple’s Time Machine with a one line command. The
presentation will be about 30-45 minutes and we’ll socialize after —
stay as long as you want to — if you have any computer questions,
bring ‘em!

See you tonight.

- Seth

Weber County Main Library, Board Room
2464 Jefferson Ave.
Ogden, UT 84401

http://maps.google.com/maps?q=Weber+County+Library,2464+Jefferson+Ave.,+84401

[edf...@gmail.com]

Sorry. I forgot what time the meeting is at.

Sent via BlackBerry by AT&T

[Seth House]

On Jun 30, 9:22 am, edf...@gmail.com wrote:
> Sorry. I forgot what time the meeting is at.

Oh, that would’ve been a good thing to include in the first
message! :)

From 7:30 to 8:45 PM tonight.

[Michael Heath]

On Tue, Jun 30, 2009 at 8:59 AM, Seth House<whit...@gmail.com> wrote:
>
> Tonight I'll present on backups; what to backup and how to backup, and
> how to recreate Apple’s Time Machine with a one line command. The
> presentation will be about 30-45 minutes and we’ll socialize after —
> stay as long as you want to — if you have any computer questions,
> bring ‘em!

For those of us who couldn't attend, what is the magic one line
command? And are there any slides or materials from the presentation
available anywhere?

[Seth House]

On Jul 1, 3:53 pm, Michael Heath <mike.thomas.he...@gmail.com> wrote:
> For those of us who couldn't attend, what is the magic one line
> command?

Run this command:

rsync -a $HOME/ /var/timemachine/$USER/`date '+%FT%R'`

Then put this in your crontab (all on one line):

1 */2 * * * BACKUPDIR=/var/timemachine/$USER; rsync -a
--link-dest=$BACKUPDIR/`/bin/ls -t $BACKUPDIR | head -1` $HOME/
$BACKUPDIR/`date '+%FT%R'`

> And are there any slides or materials from the presentation available
> anywhere?

The slides are available in the group Files section. Be sure to read
through the handout notes (print the HTML file) as they’re pretty
conversational and will give you a good feel for what was said at the
meeting.

http://groups.google.com/group/oalug/web/oalug_2009-06-30_backups.tar.bz2

[Roly C.]

How often does that cron job run?  I finally started creating backups when I accidentally recursively changed the permissions of my friend's home to 000.  (it took me about 1 hour to get everything back to the way it was...)

[Seth House]

On Jul 3, 6:49 am, "Roly C." <rol...@gmail.com> wrote:
> How often does that cron job run?

1 */2 * * * somecommand

``somecommand`` runs every two hours, one minute after the top of the
hour. The crontab manpage has a bunch of easy to understand examples.
(It's actually a typo, the one in my crontab runs at the top of the
hour.)

[Michael Heath]

I'm playing with an idea similar to this for incrimental backups. I
read an article recently that discussed unionfs for backing up like
this. Basically, to snapshot a folder, you create a union ontop of the
directory, and then create another directory, like
/.snapshots/home-since-20090702. You add this new directory to the
union, and set the original RO and the new directory to RW. You can
then add any further snapshots by simply creating a new directory,
adding it to the union, and setting the older dirs read only.

I've been working on some scripts for this, and I'm going to work on a
python plugin for nautilus that adds right click functionality like
"revert to version as of DATE". If anyone is interested, I'll share
the finished products here.

(Please pardon any bizarre formatting in this message. I'm out of town
and messaging from my phone.)

[brandon]

What would be really interesting is to benchmark it after each step and
plot out the performance degradation--it has to exist because you are
keeping snapshots live, the question is how much of an impact does it have.

FYI, for incrementals, I just use gnu tar with --newer-mtime (infact I
have attached an old script I made in solaris to make it behave like the
old unix 'dump').

I recommend monthly fulls, weekly differential and daily incremental
(for this I use a different script, which is part of a bigger
infrastructure, but uses the same --newer-mtime method). On one site I
do this to a DataDomain de-dup device / nfs share. It has 40TB of
"real" storage but currently we have 300TB of actual data stored only in
about 20TB of real disk (it is a cool device... I want an open source
version :)

-Brandon

[Michael Heath]

I don't have the original source (the site has died) but on the first
site I read about the unionfs snapshot method, the author said that hr
benchmarked filesystem performance at 80% of original with 4 snapshots
(original + 2 frozen + live).

This article was about 5 years old, and from what I understand,
unionfs has been vastly improved performance wise.

I'll run some benchmarks later this week.

[Michael Heath]

I'm going to post the full stats later, but I ran some benchmarks
using bonnie++. I actually used aufs rather than unionfs.

The big surprise came from new file creation. The speed of that was
only a small fraction of the original performance, though still high -
bonnie++ still measured several thousand files per second. I guess the
lesson here is that while this may be suitable for desktops, on many
server systems with many dynamic files it may not work.

Other performance indicators like read and write speeds were a touch
slower but very close to original.

The snapshot tools I'm working on will be another week or so at least
- it's not complex, but I had a change of heart and switched from bash
to python.

[Stuart Jansen]

Have you seen the LWN review of union filesystems on Linux? If not, I
highly recommend it. The lesson I took away was: Don't use a union
filesystem unless you absolutely have to.

--
"XML is like violence: if it doesn't solve your problem, you aren't
using enough of it." - Chris Maden

[Aaron Toponce]

Stuart Jansen wrote:
> Have you seen the LWN review of union filesystems on Linux? If not, I
> highly recommend it. The lesson I took away was: Don't use a union
> filesystem unless you absolutely have to.

And those in the BSD camp? FFS for NetBSD seems to be remarkably stable,
support a vast array of options, and even hold it's own with I/O benchmarks.

--
. O . O . O . . O O . . . O .
. . O . O O O . O . O O . . O
O O O . O . . O O O O . O O O