How to Backup OSX
Gerry Scott-Moore
a dsk drawer. Since I got the G4 and the FW CDRW I simply make copies
of directories and get as many as I can on a CD.
Now, with OSX, and all it's myriad invisible files, can I simply drag a
directory to Toas (actually operative under os 9.2.2) in order to back
up my osx partition's directories?
I can't figure out in any other way to back up and with recent Safari
scares, I need to make my first back up and soon!
Any help appreciated...
--
///--- Nazodesu no more.
Tim Lance
Scott-Moore <222...@adelphia.net.invalid> wrote:
What you need to be most concerned with is correctly backing up your
boot volume - all the stuff that passes for your old "System Folder."
In X you can't just do a copy. The term used is to "clone" a volume.
There are now several apps that do backups: Retrospect, Synchronize X,
Personal Backup, Tri-Backup come to mind. They all have some sort of
cloning feature.
For a pittance you can get Carbon Copy Cloner that clones the boot
volume. All other files you can just copy to wherever.
Tim
--
Howard S Shubs
Tim Lance <NotAbitOFspam*tla...@austin.rr.com> wrote:
> The term used is to "clone" a volume.
Uh, no. If he uses "clone", most people won't have a clue what he's
talking about. Anyone who told you otherwise is either ignorant or
selling something.
--
Today, on Paper-view: The World Origami Championship
Howard S Shubs
Gerry Scott-Moore <222...@adelphia.net.invalid> wrote:
> Any help appreciated...
Could you please choose a non-Japanese font?
Gerry Scott-Moore
Shubs <how...@shubs.net> wrote:
> In article <110120031325534317%222...@adelphia.net.invalid>,
> Gerry Scott-Moore <222...@adelphia.net.invalid> wrote:
>
> > Any help appreciated...
>
> Could you please choose a non-Japanese font?
Sure. Is that better?
Gerry Scott-Moore
Shubs <how...@shubs.net> wrote:
> In article <110120031804305822%NotAbitOFspam*tla...@austin.rr.com>,
> Tim Lance <NotAbitOFspam*tla...@austin.rr.com> wrote:
>
> > The term used is to "clone" a volume.
>
> Uh, no. If he uses "clone", most people won't have a clue what he's
> talking about. Anyone who told you otherwise is either ignorant or
> selling something.
Okay then. How *do* I back up the drive?
Matthew David Hills
Personally, I'd recommend concentrating your backups on your user directory.
(I also keep a folder that contains any files that I download and install,
such as updaters or downloaded programs). Restoration would then consist of
formatting, re-installing OSX+patches, then installing my applications from
original CD's, then restoring my user directory from backup, then installing
supplementary programs and all patches from that backed-up "Downloads" folder.
Now, what I actually do:
- firewire hard drives
- carbon copy cloner
(backing up multiple gigs to CDs is pretty annoying, so I get a HD that
can hold my entire disk, and just back it up in its entirety. Backing up
important data files to CD is not a bad idea either, by the way)
Matt
Gerry Scott-Moore
<hi...@Stanford.EDU> wrote:
> Gerry Scott-Moore <222...@adelphia.net.invalid> wrote:
> >Sadly I only have a firewire CD burner and retrospect gathering dust in
> >a dsk drawer. Since I got the G4 and the FW CDRW I simply make copies
> >of directories and get as many as I can on a CD.
> >
> >Now, with OSX, and all it's myriad invisible files, can I simply drag a
> >directory to Toas (actually operative under os 9.2.2) in order to back
> >up my osx partition's directories?
> >
> >I can't figure out in any other way to back up and with recent Safari
> >scares, I need to make my first back up and soon!
>
> Personally, I'd recommend concentrating your backups on your user directory.
> (I also keep a folder that contains any files that I download and install,
> such as updaters or downloaded programs).
Okay then, my user directory, is it not loaded with many invisible
files. And will they be included when I drag that folder to toast and
burn a CD?
> Restoration would then consist of
> formatting, re-installing OSX+patches, then installing my applications from
> original CD's, then restoring my user directory from backup, then installing
> supplementary programs and all patches from that backed-up "Downloads" folder.
Would I not be able to do the same thing with my applications directory
too. That is, are the invisible files in question not included in a
drag-directory-to-Toast situation?
> Now, what I actually do:
> - firewire hard drives
> - carbon copy cloner
> (backing up multiple gigs to CDs is pretty annoying, so I get a HD that
> can hold my entire disk, and just back it up in its entirety. Backing up
> important data files to CD is not a bad idea either, by the way)
Hmm. There's a thought. The hard drive I have 9.22 on has only 5 of
20 gig used, while my os x drive has all but 36 of 40 gig used. Most
of it I don't need to do a back up on anyway. I suppose I could dust
off and re-install Retrospect (shelved for the past year) and backup
the critical directories off the os drive. Sound feasible?
Matthew David Hills
>Okay then, my user directory, is it not loaded with many invisible
>files. And will they be included when I drag that folder to toast and
>burn a CD?
Copying a directory should include all constituent invisible files
(similar to copying in OS9).
>Would I not be able to do the same thing with my applications directory
>too. That is, are the invisible files in question not included in a
>drag-directory-to-Toast situation?
Maybe, although it can get a bit trickier with maintaining file permissions,
and also picking up the tidbits that get into /Library, and near impossible if
the program installs things into one of the standard unix directories. (eg,
into the /etc directory...)
Personally, unless you want to just mirror your entire drive, I'd still
recommend just backing up the installers and updaters for any downloaded
applications that you will want to restore. I expect that a single CD could
back up a large number of these...
Matt
Gerry Scott-Moore
<hi...@Stanford.EDU> wrote:
I see the difficulties. As part of my new non-os9 life I've started
discarding installers for shareware and even purchased software if I
did not get it on a CD, since I'm always having to hunt down the newest
incarnation anyway.
My G4 did not come with a RW or superdrive, but I'm pondering getting
one and installing it to make use of Apple's .mac download/backup
softwre. Any experience there? It seems a very native backup scheme
using a very native device.
Since--dammit!--having a firewire drive doesn't seem to buy me anything
vis-a-vis backups in OSX world. I can't begin to fathom that...
David C.
I've been using Retrospect Desktop for my backups. It's not terribly
expensive, it supports tape drives, and will back up all of your OS X
system. If something completely fails, I can re-install the OS and
Retrospect, and then restore my backup. (I'm not sure if I can just
boot a CD and restore the whole system from there.)
I currently backup the entire drive to DVD-RW media in my SuperDrive.
When my data gets too big for that to be convenient, I'll probably
invest in a high capacity FireWire tape drive (probably the Exabyte
VXA-1, but there may be something better available by then.)
-- David
David C.
>
> Personally, unless you want to just mirror your entire drive, I'd
> still recommend just backing up the installers and updaters for any
> downloaded applications that you will want to restore. I expect
> that a single CD could back up a large number of these...
Personally, I prefer using a backup program that is capable of
getting all the drirectories and using it to make periodic full
backups. I usually make mine monthly. If I'm working on something
critical, I'll make more frequent backups of those specific files to
a Zip disk.
-- David
Daryl Forrest
wrote:
The price of Retrospect just dropped, too! Substantially, I believe. You
can check it out at <http:www.dantz.com>.
- Daryl Forrest
David C.
>
> My G4 did not come with a RW or superdrive, but I'm pondering
> getting one and installing it to make use of Apple's .mac
> download/backup softwre. Any experience there? It seems a very
> native backup scheme using a very native device.
I would avoid this program. If you don't have a paid-up .mac
account, it won't work at all. I've heard that it has issues backing
up to CD media as well.
I'd recommend Retrospect. The latest version 5.0.238 is Jaguar
compatible. Previous versions of 5.0 are compatible with 10.1. The
Express version isn't very expensive and can back up to all kinds of
disk media (file, hard drive, zip, CD, DVD).
The Desktop version costs a little more, but also supports tape
drives. The Destkop version allso allows to to make remote backups of
other machines on your LAN (it includes 2 extra licenses, so a home
LAN of up to 3 Macs can be backed up from one package.)
Just make sure your hardware (backup device and SCSI card, if
applicable) are compatible with it. Dantz's web site has a hardware
compatibility list:
http://www.dantz.com/index.php3?SCREEN=compatibility_list
> Since--dammit!--having a firewire drive doesn't seem to buy me
> anything vis-a-vis backups in OSX world. I can't begin to fathom
> that...
Sure it does. Just not with the software you have right now.
-- David
Matthew David Hills
>
>Since--dammit!--having a firewire drive doesn't seem to buy me anything
>vis-a-vis backups in OSX world. I can't begin to fathom that...
Why not?
Just buy CD-R's in bulk and treat them like single-write floppies.
They should be pretty good for everything but movie editing.
(haven't got a great solution for that yet--just using hard drives
for now, but I think sending back out to tape is probably going to
be the long-term solution)
CD-R's should be pretty cheap. If you can remember to mail in your rebate,
you can often end up with 25 CD-R's for the price of a postage stamp and
sales tax.
Matt
Matthew David Hills
>When my data gets too big for that to be convenient, I'll probably
>invest in a high capacity FireWire tape drive (probably the Exabyte
>VXA-1, but there may be something better available by then.)
I took a very brief look at some of the tape solutions, but
it didn't seem like a great solution:
- Exabyte VXA-1 Drive $1k (msrp)
- VXA-1 33GB tape $67
It seems that $/GB is about even between an IDE hard drive
and the tapes. The drive cost is on top of that.
Is the stability of the media substantially better?
My impression is that tape backups don't really hit their stride until
you're looking into an autoloader system and fully automated backups.
But still seems kinda expensive.
Matt
T. Kelley Boylan
<hi...@Stanford.EDU> wrote:
Agreed. I'm working without a backup device at the moment (and am not
too happy about it), but a tape drive large enough to handle the load
simply costs too much. A cheap IDE drive, while it doesn't give me any
kind of archiving, can mirror all the data, and cheaply.
-K-
Gerry Scott-Moore
wrote:
> > Since--dammit!--having a firewire drive doesn't seem to buy me
> > anything vis-a-vis backups in OSX world. I can't begin to fathom
> > that...
>
> Sure it does. Just not with the software you have right now.
That's good to know. The Retrospect I have that didn't support
firewire was 4.1.
Can anyone verify that Retrospect for OSX supports a yamaha firewire
8824? I've been unable to ensure *specific* support that at their
site. Backwhen I first got the drive and the software, everyone here
seemed assured it would work or would work RSN. It didn't for over a
year. I'm surprised it really does now. I might just try to call
Dantz, if anyone there has a phone...
Gerry Scott-Moore
Forrest <Dar...@yahoo.com> wrote:
> The price of Retrospect just dropped, too! Substantially, I believe. You
> can check it out at <http:www.dantz.com>.
Well I spent my quarterly 15 minutes at the Dantz website. I find no
verification of their use with Yamaha Firewire drives, or a reference
to my 8824 in this regard. They support yamaha. That's nice. They
have another message addressing why it takes so long to get support for
some drives. This too is under the heading "Yamaha".
Does anyone know?
James L. Ryan
using Retrospect Express under OS-X:
a) When I tried backing up onto CDs using my Que-Fire burner things took
literally forever. I read, somewhere--I now forget where, that there was a
problem with the firmware for the Que-Fire that prevented Retrospect Express
from backing up at an optimal speed.
b) I purchased a Maxtor 250 GB firewire drive (but I suppose any firewire
drive would do just as well) and now have Retrospect Express configured to do
two backups in the middle of the night, a "duplicate," and a "backup." The
"duplicate" creates folder-by-folder replication of my internal hard drive
allowing me to do copy and paste restoration of the most recently backed up
version of a file, and the backup creates an incrementally augmented file
which allows me to step back in time if I need to recover an earlier version
of a file.
So far, this arrangement seems to be working as anticipated--although I often
wonder why the number of files backed up to the two different backups
sometimes, but not alwaysk, differ by a small number of files, say a dozen.
I'll admit to not giving this arrangement the "acid test" which would be to
completely replace the contents of my internal drive with one of the backups.
In the near future I will be purchasing a second Maxtor drive and will be
backing up to both drives, hopefully giving myself a high degree of
protection against the failure of one of the Maxtor drives.
-- James L. Ryan -- TaliesinSoft
Bux
wrote:
> Gerry Scott-Moore <222...@adelphia.net.invalid> writes:
> >
> > My G4 did not come with a RW or superdrive, but I'm pondering
> > getting one and installing it to make use of Apple's .mac
> > download/backup softwre. Any experience there? It seems a very
> > native backup scheme using a very native device.
>
> I would avoid this program. If you don't have a paid-up .mac
> account, it won't work at all. I've heard that it has issues backing
> up to CD media as well.
>
> I'd recommend Retrospect. The latest version 5.0.238 is Jaguar
> compatible. Previous versions of 5.0 are compatible with 10.1. The
> Express version isn't very expensive and can back up to all kinds of
> disk media (file, hard drive, zip, CD, DVD).
Maybe to all kinds of media but via all makes and models of CD writers.
For instance Retrospect can't back up via the CD-RW drive installed in
my PB-G4. This is standard Apple issued CD-RW. I assume a lot of people
have this problem. As soon as I mentioned the make and model, Dantz said
"no."
David C.
>
>
> That's good to know. The Retrospect I have that didn't support
> firewire was 4.1.
>
> Can anyone verify that Retrospect for OSX supports a yamaha firewire
> 8824? I've been unable to ensure *specific* support that at their
> site. Backwhen I first got the drive and the software, everyone here
> seemed assured it would work or would work RSN. It didn't for over a
> year. I'm surprised it really does now. I might just try to call
> Dantz, if anyone there has a phone...
http://www.dantz.com/index.php3?SCREEN=compatibility_list_detail&id=8120
According to this page, the Yamaha CRW8824FXZ is supported. On the
Mac platform, you need at least Restrospect Express version 4.3 and
driver update 2.1.
If you buy the latest version (5.0) and download the latest updates
there shouldn't be any problem.
-- David
David C.
>
> it didn't seem like a great solution:
> - Exabyte VXA-1 Drive $1k (msrp)
> - VXA-1 33GB tape $67
>
> It seems that $/GB is about even between an IDE hard drive
> and the tapes. The drive cost is on top of that.
Drop a tape cartridge on the floor and you won't lose any data. I
wouldn't try that with a hard drive.
> My impression is that tape backups don't really hit their stride until
> you're looking into an autoloader system and fully automated backups.
> But still seems kinda expensive.
Yes, tape is expensive. But it's not meant to be used like a hard
drive. It was designed for a specific purpose - for backups.
There are drives that are cheaper than VXA drives. I mentioned it
because it's the most economical drive that can store such a huge
quantity of data on one tape. I figure that if my backups get bigger
than 3 DVDs (about 16G, at the compression ratios I've been getting),
then the inconvenince of having to swap media once an hour will be
much worse than the price tag for the drive.
With 33GB of uncompressed storage per tape, I doubt I'll ever have a
need for more than one tape per backup set.
For a much better price, there are inexpensive Travan NS-20 drives.
They hold 10GB of uncompressed storage per tape and cost much less
than the VXA drive. I'm not considering one because by the time I
decide to switch away from DVD, my backup sets will already be too
big for a single NS-20 tape.
My other computers currently use tape drives (a DDS-2 DAT drive on my
Linux PC and a Travan TR-4 drive on my Mac Quadra.) These both use
4G (uncompressed) media, which works OK because the PC has a 9G drive
and the Mac has a 2G drive. I usually don't need more than one tape
and never need more than two. But with my G4's 80G drive, and
DVD-RW's 4.7G (uncompressed) capacity, tape drives of that size just
don't make a lot of sense.
-- David
Matthew David Hills
> A cheap IDE drive, while it doesn't give me any
> kind of archiving, can mirror all the data, and cheaply.
Why wouldn't hard drives work for archiving?
The VXA tapes were listed at $67 for 33gig.
I've been seeing 120gig internal hard drives for $130.
Buy a firewire case, and a new hard drive any time you
want to set aside your old drive for posterity?
Advantages:
- less cost up-front (similar long-term cost)
- seems to be faster than the r/w rates I've seen
quoted for tape
- larger capacity per unit.
- not tied to a single manufacturer
- flexible for other purposes--assembling large video projects, etc.
Disadvantages:
- recovery more laborious--inserting a tape
vs. screwing in the old HD mech.
- long-term stability of HD mechanism?
(occasionally I hear about stuck drive heads,
but then I also hear about faded dyes on
CDR's, etc.)
Also, I'm not sure how survivability compares... (ie, due to media
corruption/destruction... HFS/HFS+ hard drives aren't great -- lose a couple
of sectors of your catalog or extents files and you have problems. Does
Retrospect use an interesting file format that would aid recovery? I would
think that tape systems have a more recoverable format)
David--feel free to add to the list; I haven't been using
tapes enough to really know the ins and outs of them...
the above was more my own deliberation as I considered
what routes I wanted to go
Matt
Pat Janes
hi...@Stanford.EDU (Matthew David Hills) wrote:
> Also, I'm not sure how survivability compares... (ie, due to media
> corruption/destruction... HFS/HFS+ hard drives aren't great -- lose a couple
> of sectors of your catalog or extents files and you have problems. Does
> Retrospect use an interesting file format that would aid recovery? I would
> think that tape systems have a more recoverable format)
The VXA drive in particular uses something they call packet-based data
or some such. The media reliability/recoverability claims made on their
website (vxa.com) are incredible - tapes immersed in hot coffee, frozen
and thawed etc. I've recently installed a VXA-1 firewire tape drive but
have not yet had reason to test such claims.
Gerry Scott-Moore
wrote:
> If you buy the latest version (5.0) and download the latest updates
> there shouldn't be any problem.
I did, and there wasn't. Actually I just called them on the phone to
talk to a human. I bumped into one of the nicest and most helpful
customer service reps in many a year, who hunted down the appropriate
tech guy who could verify that a 8824 Yamaha Firewire would work.
I paid for the upgrade to desktop (which has a 30 day money-back
guarantee) downloaded it, used it. It isn't exactly fast, but seems
fully operative.
Thanks for the input from all.
David C.
>
> listed at $67 for 33gig. I've been seeing 120gig internal hard
> drives for $130. Buy a firewire case, and a new hard drive any time
> you want to set aside your old drive for posterity?
True. But I don't think anybody ever buys tape drives because of the
price.
> Advantages:
> - less cost up-front (similar long-term cost)
> - seems to be faster than the r/w rates I've seen
> quoted for tape
> - larger capacity per unit.
> - not tied to a single manufacturer
> - flexible for other purposes--assembling large video projects, etc.
Tape drives shouldn't be tied to a single manufacturer. SCSI-2 tape
drives are all compatible with each other. So are ATAPI drives. I
would hope that FireWire is too, since FW uses SCSI signaling
protocols.
> Disadvantages:
> - recovery more laborious--inserting a tape
> vs. screwing in the old HD mech.
> - long-term stability of HD mechanism?
> (occasionally I hear about stuck drive heads,
> but then I also hear about faded dyes on
> CDR's, etc.)
Decent tape drives have quick-seek mechanisms. As long as your
backup program keeps an index file on your hard drive, restoring a
single file usually doesn't take very long. But I'll admit it's even
faster using disc media (hard drive or optical).
As for stability of the drives and media, it depends greatly on what
brand/model drive and what brand/model media.
I personally despise everything made by Sony. But I love my Aiwa DAT
drive, and a lot of people I know swear by Exabyte's 8mm drives.
If you've got truly huge amounts of data, you get get autoloader
versions of most of the high-end tape formats. But you pay big time
for something like that. They're typically only sold to corporate IT
departments, and are priced accordingly.
> Also, I'm not sure how survivability compares... (ie, due to media
> corruption/destruction... HFS/HFS+ hard drives aren't great -- lose
> a couple of sectors of your catalog or extents files and you have
> problems. Does Retrospect use an interesting file format that would
> aid recovery? I would think that tape systems have a more
> recoverable format)
Hard drives can get damaged if they are subject to shock. Don't drop
one off of a table. Some modern drives (especially 2.5" laptop
drives) can withstand quite a bit, but there is still a rather
delecate magnetic head assembly inside each one which can get damaged
if it suffers too much of an impact.
Tapes, on the other hand, are separate from their drive mechanism,
and so can usually withstand a lot more abuse.
Other removable magnetic systems can often survive a lot as well.
Although certain kinds (like SyQuest) are more sensitive than modern
hard drives.
> David--feel free to add to the list; I haven't been using tapes
> enough to really know the ins and outs of them... the above was
> more my own deliberation as I considered what routes I wanted to
> go
They're designed for one purpose and one purpose only - making
backups of large amounts of data. Their nature (a spool of tape)
means that they will not work well for anything other than sequential
access.
You definitely get more flexibility with hard drives.
The price may even be competitive with large tapes, although I'm not
so sure of that, if you start factoring in the price of FireWire
cases. Note that corporate users of tape drives are going to have
many tapes - typically weekly full backups and daily incremental
backups. And several months worth of historic archives. Which means
a lot of tapes and a lot of run-time.
Hard drives with firewire cases (since you won't be swapping drives
in the cases every week) are still more expensive than large tapes.
Also, the cost in terms of labor for someone to swap tapes can easily
be way more than the cost of media - especially if he is forced to
stay late at night or work on a weekend to do the tape swapping.
This is the primary motivation for corporations paying the high price
for autoloaders.
Even for personal use, I agree, up to a point. When I back up my PC
(everything usually fits on a single 4G tape), I can insert a tape,
start the program, and go to bed. I put the tape away when I get up
in the morning.
When I back up my Mac, however (where I don't have a tape drive), I
can't do this. It takes two DVDs to hold everything, so I have to
swap media three times (once during the backup, then re-insert the
first disc at the start of verification, and then a swap during
verification). This means I have to start it early enough so that I
can remain awake while it's running, and I can't use the computer
during that time. Much much more inconvenient. When the backup set
gets big enough to require three discs (and therefore five disk
swaps) it will be inconvenient enough that I will welcome a large
tape drive, even if it does cost $1000.
-- David
heron stone
> > listed at $67 for 33gig. I've been seeing 120gig internal hard
> > drives for $130. Buy a firewire case, and a new hard drive any time
> > you want to set aside your old drive for posterity?
my opinion...
.i've been backing up to hds for years
.i have two drives for everything...
.2 80 gb maxtors inside my g4/400
.they are identical and i use retrospect's
duplicate function every couple of days
to keep them that way
.these drives contain all my system stuff,
applications and documents (not including
media stuff... mp3s and video)
.if anything goes wrong with my startup drive,
i can simply boot to the clone and get
back to work
.i also have a lot of mp3s on an ext. maxtor fw
250gb and an ext. maxtor 160gb fw
.both of them are duplicated on duplicate drives
.the maxtor 250s cost $400 each
.that's way cheaper than backing up to tape when
you take the price of a good tape drive into
consideration
.and it's much simpler
heron
--
Nature, heron stone
to be commanded, heron...@attbi.com
must be obeyed. http://home.attbi.com/~heronstone/
CC Zona
heron stone <heron...@attbi.com> wrote:
> .i've been backing up to hds for years
> .i have two drives for everything...
>
> .2 80 gb maxtors inside my g4/400
<snip>
>
> .the maxtor 250s cost $400 each
> .that's way cheaper than backing up to tape when
> you take the price of a good tape drive into
> consideration
Hard drives are the cheaper solution, but tape offers other advantages as a
backup medium to justify its cost.
When you can't access the original data because of an accidental overwrite
or directory corruption or disk failure, the backup on the internal is
handy because it's accessible and bootable. In those everyday
circumstances, it's a terrific solution.
When you can't access the original data due to damage caused by a virus,
fire, natural disaster, theft, etc. those backups to the internal HD are
just as likely to be a loss as the originals. In those catastrophic
circumstances, you could be pretty much screwed. Whereas the customary way
tapes are used is by keeping multiple rotating sets, in which at least one
is always being stored offsite in a heat/moisture/theft-resistant container
until its next turn in the rotation. Try taking one of those internal
drive for a walk every night--not fun. Backing up to some form of
removable/portable media is a Good Thing(TM), pricier though it may be.
--
CC
heron stone
CC Zona <ccz...@nospam.invalid> wrote:
.you're right
.that's why i back up my docs to CDs and keep them
at a friend's house
.all my really important docs take up less than 1gb
.and all my cleaned-up MP3s are also on the same
friend's 250 gb hd
.he bought the drive... i gave him the music
.and i think he bought another drive and gave a copy
to his daughter
.it's not hard to get people to back up my music for
me
Tony Lawrence
But it's not a good backup.
See http://aplawrence.com/Unixart/diffback.html for why.
--
Tony Lawrence
Free SCO, Mac OS X and Linux Skills Tests:
http://aplawrence.com/skillstest.html
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> See http://aplawrence.com/Unixart/diffback.html for why.
One serious error in your analysis: incremental backups also -delete-
files. I use incremental backs on both VMS and Macintosh. They're the
best way. Your concept of "differential" sounds like it'd waste a lot
of tape, or you'd be making a lot of full backups that the incremental
crowd won't need.
Incremental backups require a certain amount of recovery time, sure. On
the Mac, Retrospect handles that just fine. On VMS, you restore them
backwards and it's handled for you. No biggie.
--
Today, on Paper-view: The World Origami Championship
Robert Peirce
yymmdd.pax.gz file which I write to CD with Toast. To avoid an initial
huge backup of system stuff, where I had the CD, I set my first
timestamp for the day I received the computer.
The real test of a backup system is restoring. I have already had to
restore a couple of download files I needed to reinstall. This worked
like a charm.
As with any incremental system, you have to know where the stuff is, so
I create indexes that I leave on-line.
This is about the cheapest thing I can think of. CD blanks are less the
20 cents each in bulk and Toast was pretty cheap as I recall.
Furthermore, the latest Mac knoledgebase mail says the Mac can burn
sessions so you really could get by without using Toast at all!
Everything you need except the CD blanks is already on the system.
--
Robert B. Peirce Venetia, PA 724-941-6883
inv...@stargate.net [Mac] r...@pgh.net [PC]
m...@venetia.pgh.pa.us [NeXT]
r...@cooksonpeirce.com [Office]
Howard S Shubs
jdoh...@null.com (John Doherty) wrote:
> Mine certainly don't, and never have. Once a file is written to a
> backup tape, it stays there until the tape is recycled, usually
> for at least a year.
On -recovery-, guy. They delete files on recovery. If a file didn't
exist at the time of an incremental, and that incremental is restored,
any such files not there will be deleted. You're not going to over-fill
a disk by an incremental restore. That's why the VMS "incremental
backup" restore instructions say to do it backwards.
> You get greater redundancy, though, i.e., more copies of each file
> that's backed up.
This form of redundancy's not useful, though. Several copies of the
same file with no changes to the content is just less efficient, not in
a good way, either.
> Using the standard unix dump, you can have both of what he calls
> "incremental" and "differential" backups just by choosing your
> dump levels appropriately, and in fact, that's what I do. Some of
> my backups are what he calls incremental and others are what he
> calls differential.
Consistancy is important. If you don't stay consistant, recovery of a
whole volume will be a problem.
> I can't tell what the point of his article is supposed to be. It seems
> he wants to argue that all backups should be full backups, but for
> me, that's just completely impractical and useless advice. If we
> did that, we'd spend about $50,000 a year on tape. :-)
Yeah, it's a problem. Ideally, you'd only have failures just after a
full backup. I wish it were possible to plan ahead that way. :-/ You
could then create a mirror set with the drive that's about to fail in
time to get everything copied and have zero downtime! mmmmm...
Howard S Shubs
> In article <m23cnrk...@qqqq.invalid>, sha...@techie.com (David C.) wrote:
>
> > I personally despise everything made by Sony.
>
> Their AIT tape drives are terrific, if you ask me. We've been using
> them for three years or so now.
I just started using an AIT-1 library last June. It's great for my
purposes.
> You couldn't pay me to use DAT, except for audio. Opinions differ,
> I guess. :-)
Me too. I used to swear by DAT, but then it became too small and slow
for the size of the source. Given that DAT has reached end-of-life,
it's not a way to go at this time.
> Sony's Super AIT will put up to a terabyte on one tape.
And a youngster some tomorrow will look at that and tell you that his
removable media is larger than that. :->
> Too bad you hate Sony, because you can pick up a nice AIT1 drive
> for under $1,000, with either FireWire or SCSI. :-)
That's what I did. Got an AIT-1 library on eBay, purchased an enclosure
for it, and have been using it since. With the enclosure, cable, and
controller, the total came to a bit under US$1K IIRC.
Tony Lawrence
> In article <5B9W9.724321$%m4.34...@rwcrnsc52.ops.asp.att.net>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>See http://aplawrence.com/Unixart/diffback.html for why.
>
>
> One serious error in your analysis: incremental backups also -delete-
> files.
I think you are misunderstanding something here. How can a backup know
whether or not a file that it backed up should or should not be
restored? It can't. There's also the matter of losing one piece of
thev differential, as the article explained.
I use incremental backs on both VMS and Macintosh. They're the
> best way. Your concept of "differential" sounds like it'd waste a lot
> of tape, or you'd be making a lot of full backups that the incremental
> crowd won't need.
Differentials are NOT the best way. They are acceptable if there is no
other way for you to do complete backups. Period. If you don't
understand that, you'll someday wish that you did.
Trust me: I've seen more disasters than you'd ever want to know about.
If it is at alll possible, you should be backing up every single file on
every single backup. If it isn't, well, that's different. Like
anything else, you settle for less when you have to. However, as tapes
are faster and bigger than ever before, and the static files are usually
the least of your problems for speed or space, differentials are hard to
excuse.
Tony Lawrence
> In article <howard-DDFC27....@enews.newsguy.com>, Howard S
> Shubs <how...@shubs.net> wrote:
>
>
>>In article <5B9W9.724321$%m4.34...@rwcrnsc52.ops.asp.att.net>,
>> Tony Lawrence <to...@pcunix.com> wrote:
>>
>>
>>>See http://aplawrence.com/Unixart/diffback.html for why.
>>
>>One serious error in your analysis: incremental backups also -delete-
>>files.
>
>
> backup tape, it stays there until the tape is recycled, usually
> for at least a year.
>
>
>>best way. Your concept of "differential" sounds like it'd waste a lot
>>of tape, or you'd be making a lot of full backups that the incremental
>>crowd won't need.
>
>
> that's backed up.
>
> "incremental" and "differential" backups just by choosing your
> dump levels appropriately, and in fact, that's what I do. Some of
> my backups are what he calls incremental and others are what he
> calls differential.
>
> all files created or modified since the last level N-1 dump, N-2
> if there was no N-1, etc. Simple, but effective.)
>
> For example, consider the following dump pattern: 0, 2, 2, 3, 3, 1.
>
> The level 0 backup gets everything. The level 2 backups are what
> he would call both incremental and differential. The first level
> 3 backup is incremental and the second one is both incremental and
> differential, and finally, the level 1 is strictly differential.
>
> I can't tell what the point of his article is supposed to be. It seems
> he wants to argue that all backups should be full backups, but for
> me, that's just completely impractical and useless advice. If we
> did that, we'd spend about $50,000 a year on tape. :-)
And how much is your data worth ?
But wow, you must be buying some pretty expensive tapes.
Tony Lawrence
> This form of redundancy's not useful, though. Several copies of the
> same file with no changes to the content is just less efficient, not in
> a good way, either.
That is completely ridiculous.
I'm sorry, but you just don't get it. Unfortunately, I've "been there
and seen that". Too many times. Tapes snap. Data degrades. Tapes get
lost, misplaced, overwritten, stepped on, burned, chewed by rats and
drowned. The more reduncacy you have the less your recovery costs are.
> Yeah, it's a problem. Ideally, you'd only have failures just after a
> full backup. I wish it were possible to plan ahead that way. :-/ You
> could then create a mirror set with the drive that's about to fail in
> time to get everything copied and have zero downtime! mmmmm...
>
Sigh. Exactly the point: If every backup IS a full backup, then that's
exactly what you have: only failures after a full backup.
But never mind: You do it your way, and I'll keep my customers doing it
my way. You have a right to do whatever you want.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> That is completely ridiculous.
If you say so. I've been doing this for 16 years.
> I'm sorry, but you just don't get it. Unfortunately, I've "been there
> and seen that". Too many times. Tapes snap. Data degrades. Tapes get
> lost, misplaced, overwritten, stepped on, burned, chewed by rats and
> drowned. The more reduncacy you have the less your recovery costs are.
That's why you have multiple sets of tapes. Extra redundancy on -one-
set of tapes doesn't help you nearly as much.
> Sigh. Exactly the point: If every backup IS a full backup, then that's
> exactly what you have: only failures after a full backup.
Full-backups every time is ridiculous. Multiple backup sets with
incrementals is the way.
> But never mind: You do it your way, and I'll keep my customers doing it
> my way. You have a right to do whatever you want.
No kidding. I'll keep your name in mind.
Howard S Shubs
> Maybe the VMS stuff worked that way, but I don't think it's a general
> characteristic.
That's also how Retrospect seems to work.
> By default, the unix restore doesn't work that way. If you're doing
> a full recovery, and try it in the wrong order, it just tells you
> that. It doesn't delete any files.
Sounds silly.
> It is if one of the tapes turns out to be unreadable. Or if one of
> them is off-site, and not conveniently at hand.
I keep 7 sets of backups. One set breaking or being unavailable isn't a
problem.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I think you are misunderstanding something here. How can a backup know
> whether or not a file that it backed up should or should not be
> restored? It can't. There's also the matter of losing one piece of
> thev differential, as the article explained.
Okay, I'll explain how it's done, if I can remember.
A backup set, be it a full backup or an incremental, stores file
attributes as well as data. By comparing the "date of last
modification", you can determine if a file on disk is to be replaced by
one on tape. That covers restoring only the latest copy, yes?
So, how does it know to -delete- old files? IIRC, by storing directory
information as well, a directory being a kind of file. If a newer
version of a directory file doesn't contain file X.Y, then file X.Y
shouldn't continue to exist, and it gets deleted. I might be
remembering wrong, though. I'll try cross-posting to a group which
would know.
> Differentials are NOT the best way. They are acceptable if there is no
> other way for you to do complete backups. Period. If you don't
> understand that, you'll someday wish that you did.
No, that's what I was saying. We're in violent agreement. Full backups
followed by a series of incrementals is the way to go.
> Trust me: I've seen more disasters than you'd ever want to know about.
I haven't, because I've always been able to recover from data loss, or
near as dammit, since my first few weeks in the industry.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> But wow, you must be buying some pretty expensive tapes.
Or he just has a lot of data.
Tony Lawrence
> In article <4wlW9.731686$%m4.34...@rwcrnsc52.ops.asp.att.net>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>I think you are misunderstanding something here. How can a backup know
>>whether or not a file that it backed up should or should not be
>>restored? It can't. There's also the matter of losing one piece of
>>thev differential, as the article explained.
>
>
> Okay, I'll explain how it's done, if I can remember.
>
> A backup set, be it a full backup or an incremental, stores file
> attributes as well as data. By comparing the "date of last
> modification", you can determine if a file on disk is to be replaced by
> one on tape. That covers restoring only the latest copy, yes?
>
> So, how does it know to -delete- old files? IIRC, by storing directory
> information as well, a directory being a kind of file. If a newer
> version of a directory file doesn't contain file X.Y, then file X.Y
> shouldn't continue to exist, and it gets deleted. I might be
> remembering wrong, though. I'll try cross-posting to a group which
> would know.
>
Sheesh. I understand the algorithm it could use, what I'm trying to get
through to you is that it is insufficient. The whole point of backup is
recover data. Sometimes files get deleted that should not have been and
that fact isn't always discovered immediately. Sometimes the fact that
something is gone isn't even known: all that is known is that something
that used to work does not. Couple that with some bad timing and a full
restore and this smart differential backup finds the file on its master
but then deletes it yet again on the differerential.. wonderful.
>
>
>>Differentials are NOT the best way. They are acceptable if there is no
>>other way for you to do complete backups. Period. If you don't
>>understand that, you'll someday wish that you did.
>
>
> No, that's what I was saying. We're in violent agreement. Full backups
> followed by a series of incrementals is the way to go.
No. FULL BACKUPS EACH AND EVERY TIME.
:-)
>
>
>
>>Trust me: I've seen more disasters than you'd ever want to know about.
>
>
> I haven't, because I've always been able to recover from data loss, or
> near as dammit, since my first few weeks in the industry.
>
I don't mean to offend, but that's incredibly naive. I've been involved
with computer systems since 1967. I have heard the wailing and seen the
gnashing of teeth. Bad things happen to backup systems, and when the
shit has really hit the fan and splattered all over the room, the
ability to pull a complete backup from three days back or even last
month can be the difference between a lot of hard work and complete
disaster.
If you can't do full backups because it takes too long or you don't have
big enough storage, fine: do differentials. As I have said though, the
usual case is that the static files are often the least of your problems
in space and time, and large, fast archival systems get cheaper and
cheaper all the time anyway.
Tony Lawrence
> In article <2ylW9.731278$P31.542469@rwcrnsc53>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>But wow, you must be buying some pretty expensive tapes.
>
>
> Or he just has a lot of data.
>
Sure. But strange that most of it is so static that full backups
supposedly cost this much but differentials don't?
I can envision such data sets: tons of historical data. That's a
special case we haven't touched on here: data that could be kept on
non-volatile media if it were not for the need for fast access.
In that case, it makes perfect sense to put that data on optical disks,
cd's, dvd's, microfilm or granite etchings :-) and not back it up. But
that isn't differential backup: it's just deliberate and reasonable
exclusion of unchanging data.
John Santos
to...@pcunix.com says...
Err, if you know a particular file is missing, you restore it
explicitly. You don't do a full restore unless you are recovering a
trashed disk.
For some reason this thread is cross-posted between comp.os.vms and
comp.sys.mac.apps. I don't know anything about Mac backups, but
VMS backup certainly allows you to do selective restores of any
file or group of files from an image, differential or incremental
backup. If you do a selective restore, it restores the files you
ask for (including everything if you ask it nicely) and doesn't
delete anything. It is explicitly *not* trying to recreate the
disk at the time of the most recent backup in this case.
>
> >
> >
> >>Differentials are NOT the best way. They are acceptable if there is no
> >>other way for you to do complete backups. Period. If you don't
> >>understand that, you'll someday wish that you did.
> >
> >
> > No, that's what I was saying. We're in violent agreement. Full backups
> > followed by a series of incrementals is the way to go.
>
> No. FULL BACKUPS EACH AND EVERY TIME.
>
>
> :-)
>
>
> >
> >
> >
> >>Trust me: I've seen more disasters than you'd ever want to know about.
> >
> >
> > I haven't, because I've always been able to recover from data loss, or
> > near as dammit, since my first few weeks in the industry.
> >
>
> I don't mean to offend, but that's incredibly naive. I've been involved
> with computer systems since 1967. I have heard the wailing and seen the
> gnashing of teeth. Bad things happen to backup systems, and when the
> shit has really hit the fan and splattered all over the room, the
> ability to pull a complete backup from three days back or even last
> month can be the difference between a lot of hard work and complete
> disaster.
>
> If you can't do full backups because it takes too long or you don't have
> big enough storage, fine: do differentials. As I have said though, the
> usual case is that the static files are often the least of your problems
> in space and time, and large, fast archival systems get cheaper and
> cheaper all the time anyway.
>
>
>
>
--
John
Tony Lawrence
I said: " Couple that with some bad timing and a full restore" ..
>
> For some reason this thread is cross-posted between comp.os.vms and
> comp.sys.mac.apps. I don't know anything about Mac backups, but
> VMS backup certainly allows you to do selective restores of any
> file or group of files from an image, differential or incremental
> backup. If you do a selective restore, it restores the files you
> ask for (including everything if you ask it nicely) and doesn't
> delete anything. It is explicitly *not* trying to recreate the
> disk at the time of the most recent backup in this case.
Sigh. NOT THE POINT.
Systems crash. It's funny in a way, but the reliability of modern
systems contributes to poor backup practice. Disk failures used to be a
fairly common thing for me to deal with, but now I can go a year or
more without seeing even one. Because of this, people tend to get
sloppy: I have customers now who have NEVER had a hardware disk failure.
So you take someone like that who doesn't have a paranoid
administrator or a PITA consultant who yells at them when he finds
backups in disarray :-) and you have a recipe for disaster. Tapes are
lost, improperly labeled, overwritten, unverified, frozen, whatever.
Now your "smart differential" does its restore dance, and deletes a file
that never should have been deleted. If you are lucky, the problem is
noticed immediately. If you are not lucky, the need for that file
doesn't come up until every tape that had it has been long overwritten.
Or, you are even less lucky and one of your differentials snaps, is
defective, is lost, mislabeled whatever. Lucky you. What now? Go back
to the previous set of course - how old is that?
The more redundancy, the better. Period. I really cannot understand
why something so simple and obvious has caused such contention.
The BEST backup is a COMPLETE backup each and every time. If it is
impossible to do that because of space or time issues, then AND ONLY
THEN should you use differentials.
Sheesh :-)
James L. Ryan
(in message <d8yW9.738625$%m4.34...@rwcrnsc52.ops.asp.att.net>):
> The BEST backup is a COMPLETE backup each and every time. If it is
> impossible to do that because of space or time issues, then AND ONLY
> THEN should you use differentials.
Perhaps it has already been mentioned in this thread of discussion, but
Retrospect (and Retrospect Express) allow one to make three different kinds
of backups:
1) an incremental backup which augments the backup with only those items
which are new or which have changed. Such a backup allows one to selectively
restore any version of an item that at any time has been included in the
backup set. The structure of an incremental backup is a proprietary single
file which requires Retrospect or Retrospect Express when recovering items.
2) a duplicate backup which replaces in the backup only those items that have
are new or which have changed. A duplicate backup can only be made to a hard
disk or to memory. The structure of the duplicate backup is logically
identical to the original and items may be freely moved, without the need for
Retrospect or Retrospect Express, at any time from the backup back into to
the active set.
3) an archival backup which is similar to an incremental backup with the
difference being that designated items are also removed from the active set.
Depending on the user's choice, as many backup sets can be specified as the
user wishes, the backup sets can be a mixture of the above, and the backup
schedule can be automated.
-- James L. Ryan -- TaliesinSoft
David C.
> David C. wrote:
>>
>> I personally despise everything made by Sony.
>
> them for three years or so now.
I haven't used an AIT drive, but every helical-scan drive mechanism
I've ever used by Sony (VHS, DAT and 8mm video) has failed after 2-3
years. I don't think they have a clue on how to make a reliable
helical-scan drive, and I won't trust any such drive made by them
ever again.
I personally had a backup trashed by a Sony DDS-2 drive that failed
after less than three years of light use. When I complained they told
me (in nicer weasel language) to go screw myself, because the one year
warantee had expired.
I've had similar reliability problems in many other Sony devices as
well, including amplifiers, CD players, and computers. In my
experience, the only Sony products that don't fall apart after a short
period of time are their professional monitors. (I have had quality
problems with their inexpensive consumer-grade monitors, however.)
I don't think anybody in their entire corporation has a clue about
quality control. There's no way I'd ever trust my data to a Sony
product. If you had my experience, I'm sure you'd agree with me.
>> But I love my Aiwa DAT drive
>
> You couldn't pay me to use DAT, except for audio. Opinions differ, I
> guess. :-)
Mine has been working flawlessly for quit a long time now.
Unfortunately, Aiwa isn't making data-DAT drives anymore, which is a
pity because I really like mine.
Also unfortunately, Sony makes the lion's share of DAT drives sold
these days. Most drives you find sold with other manufacturer's
names are also made by Sony.
But I won't buy DAT in the future because there isn't any DAT format
with a large enough capacity for today's hard drives, and an
auto-loader is just a waste.
>> and a lot of people I know swear by Exabyte's 8mm drives.
>
> We used Exabytes for years before AIT, going back to the days of
> minicomputers. We still have three working 8505XLs, and an unused
> spare on the shelf. Those things are workhorses.
IIRC, Exabyte's "Mammoth" drives are the upgrade path for their
classic 8mm drives. The M1 has a 20G uncompressed capacity and the M2
has a 60G uncompressed capacity. But these are way too expensive for
my budget.
But I think their most economical drives are the VXA drives. 33G
VXA-1 or 80G VXA-2.
> Too bad you hate Sony, because you can pick up a nice AIT1 drive
> for under $1,000, with either FireWire or SCSI. :-)
At MSRP, a bare SCSI Exabyte VXA-1 drive costs $700. External SCSI is
$900. External FireWire is $1000.
Seems competitively priced to an AIT1 drive.
See also:
http://www.exabyte.com/products/products/vxa_firewire.cfm
http://www.storagebysony.com/products/productmain.asp?id=139
Sony advertises AIT-1 as 90G capacity, with 2.6:1 compression (a
completely unrealistic ratio, IMO), which equates to 34.6G without
compression. Meaning that the capacities of AIT-1 and VXA-1 are
close enough as to be considered equivalent.
-- David
David C.
>
> As with any incremental system, you have to know where the stuff is, so
> I create indexes that I leave on-line.
And any backup software worth using will automatically maintain these
kinds of indexes for you.
With Retrospect, a selective restore is simple. You pick the backup
set, and you're given a list of all the files/directories. Select the
ones you want and go. It will prompt you for the tapes/discs that it
needs in order to restore the selected files.
> This is about the cheapest thing I can think of. CD blanks are less
> the 20 cents each in bulk and Toast was pretty cheap as I recall.
Cheap, until you start making full backups. 700M per disc isn't a
lot. I've got about 7GB on my hard drive, which equates to about 10
CDs (assuming no file-system overhead and no compression.) My
Retrospect backups have been getting about 14% compression (1.14:1),
so that would equate to about 800M per disc - or 9 discs for the
entire set.)
Admittedly, this is still cheap - 9 discs is less than two dollars.
But there's the cost in terms of time. A 9-disc backup means you
need to swap media 17 times (8 during the backup, 8 during the
verify, and once between backup and verify). This means you're
practically tethered to the computer until it finishes.
I don't speak for anybody else, but my time is worth a lot more than
the media-cost savings from using CDs.
-- David
Tony Lawrence
Cd's aren't great for backup. DVD-RAM is a lot better, though still
limited to set compressible to 4.7 GB and with a time window of apppx
3GB per hour. As I said, much better than CD, but sometimes not good
enough. See http://aplawrence.com/Reviews/dvdram.html too.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> Sheesh. I understand the algorithm it could use, what I'm trying to get
> through to you is that it is insufficient. The whole point of backup is
> recover data. Sometimes files get deleted that should not have been and
> that fact isn't always discovered immediately. Sometimes the fact that
Which is when you go recover a specific file, which capability isn't
lost from using incrementals. I'm talking about the ability to restore
entire volumes.
> :-)
<rolls eyes>
> I don't mean to offend, but that's incredibly naive. I've been involved
> with computer systems since 1967. I have heard the wailing and seen the
> gnashing of teeth. Bad things happen to backup systems, and when the
> shit has really hit the fan and splattered all over the room, the
> ability to pull a complete backup from three days back or even last
> month can be the difference between a lot of hard work and complete
> disaster.
Sure, and I can do it.
Tony Lawrence
> In article <3RvW9.615140$GR5.4...@rwcrnsc51.ops.asp.att.net>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>Sheesh. I understand the algorithm it could use, what I'm trying to get
>>through to you is that it is insufficient. The whole point of backup is
>>recover data. Sometimes files get deleted that should not have been and
>>that fact isn't always discovered immediately. Sometimes the fact that
>
>
> Which is when you go recover a specific file, which capability isn't
> lost from using incrementals. I'm talking about the ability to restore
> entire volumes.
I said (but apparently have to keep saying it over and over) that I am
referring to the case where YOU DON'T KNOW THE FILE IS MISDSING.
>
>
>
>>:-)
>
>
> <rolls eyes>
>
>
>>I don't mean to offend, but that's incredibly naive. I've been involved
>>with computer systems since 1967. I have heard the wailing and seen the
>>gnashing of teeth. Bad things happen to backup systems, and when the
>>shit has really hit the fan and splattered all over the room, the
>>ability to pull a complete backup from three days back or even last
>>month can be the difference between a lot of hard work and complete
>>disaster.
>
>
> Sure, and I can do it.
>
Best of luck to you. May you live long enough to regret this decision.
Never wrestle with a pig. It gets you dirty and annoys the pig.
Howard S Shubs
John Santos <john....@post.harvard.edu> wrote:
> For some reason this thread is cross-posted between comp.os.vms and
> comp.sys.mac.apps.
Yes, I did that. We're talking about backup methods, and I know backups
are backups, on VMS or Macintosh, where I know both systems, so I
cross-posted in order to get more contributors to discuss backup
methodology.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I said: " Couple that with some bad timing and a full restore" ..
I don't see it. Since you have multiple backup sets, and because
restoring a volume doesn't destroy the restore tapes, what's the problem?
> Sheesh :-)
I can't say you're clueless, 'cause you're not, but you're ignoring our
point.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> Sure. But strange that most of it is so static that full backups
> supposedly cost this much but differentials don't?
Say you're making backups of an Oracle db. If you do a "differential",
you're effectively doing a full backup every time, since the files are
changing every time.
If you only record the set of changed records, you're doing something
closer to an incremental backup, using the tools Oracle provides (I
think) for this task.
> In that case, it makes perfect sense to put that data on optical disks,
> cd's, dvd's, microfilm or granite etchings :-) and not back it up. But
> that isn't differential backup: it's just deliberate and reasonable
> exclusion of unchanging data.
Much too small, compared to the size of the data that's changing.
Tony Lawrence
> In article <d8yW9.738625$%m4.34...@rwcrnsc52.ops.asp.att.net>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>I said: " Couple that with some bad timing and a full restore" ..
>
>
> I don't see it. Since you have multiple backup sets, and because
> restoring a volume doesn't destroy the restore tapes, what's the problem?
>
>
>
>>Sheesh :-)
>
>
> I can't say you're clueless, 'cause you're not, but you're ignoring our
> point.
>
Exactly what I am thinking of you, strangely enough..
But I'm tired of slapping this horse's rump. I remain convinced that
anything other than complete backups is inferior and should never be
done unless there is no other choice.
You remain convinced otherwise.
May neither of us need any damn backups anyway :-)
JF Mezei
> But I'm tired of slapping this horse's rump. I remain convinced that
> anything other than complete backups is inferior and should never be
> done unless there is no other choice.
It depends on the backup utility, and the robustness of the operating system
and file system. For VMS, one can have a full backup complemented with
incremental backups. It saves time every day when backups are done, but it
takes longer to find the file you need on those rare occasions when you need
to restore a file.
With VMS, you can generate a backup journal which lists all the files that are
being written to tape, so it is fairly easy to check which backup contains the
file you want.
If managed properly, a full backup complemented with incrementals is not a
problem to restore when you have the right robust OS. Just takes longer to do
and better planning/understanding, as well as properly scheduling how often
you do a full backup.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I said (but apparently have to keep saying it over and over) that I am
> referring to the case where YOU DON'T KNOW THE FILE IS MISDSING.
And as I said, the backup set isn't erased when you restore it, so you
can go back and restore files from earlier snapshots if some file was
deleted by accident. Clear yet?
> Best of luck to you. May you live long enough to regret this decision.
Don't think it'll happen, guy. I lose disks all the time and the data's
fine.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> You remain convinced otherwise.
Yup. I'm mired in reality, rather than ivory tower theorising.
> May neither of us need any damn backups anyway :-)
But we do! We do! That's the biggest system management task of them
all. I think we're both aware of that.
Tony Lawrence
> In article <lXDW9.56181$1q3.7145@sccrnsc01>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>You remain convinced otherwise.
>
>
> Yup. I'm mired in reality, rather than ivory tower theorising.
THAT I cannot ignore :-)
I'm an independent, and have been since 1983. I'm not a var pushing
product, I'm not a contract admin who stays at the same place for months
at a time: I'm a guy who is out in the trenches every day, flitting from
customer to customer ('cept nowadays my flitting is more ssh than
automobile).
My comments about backup are based on my experience with literally
thousands of customers. In the old days (not all that long ago) I'd be
involved in hard drive failures at least once a month. Nowadays it's
once a year, if that, but I've seen the problems with differentials and
incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
If that's "ivory tower", I'm a 15th century blacksmith having
hallucinatory visions.
That's it. Go and sin no more :-)
Rob Young
>
> My comments about backup are based on my experience with literally
> thousands of customers. In the old days (not all that long ago) I'd be
> involved in hard drive failures at least once a month. Nowadays it's
> once a year, if that, but I've seen the problems with differentials and
> incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
>
> If that's "ivory tower", I'm a 15th century blacksmith having
> hallucinatory visions.
>
No, just very out of date.
I do incrementals - always. Folks are somewhat uncomfortable with
that. The warm and fuzzy is to do (typical) weekly fulls, daily
incrementals back to the weeklies. With TSM, you are always
doing incrementals.
The problem is conceptual. Think of your backups taking place to
a fault-tolerant database. Each day, an export goes off-site so
your backed up files reside in two places (this is typical for most
of us regardless of backup solution).
What is occuring is files that are *not* changing are only
backed up initially! incrementals forever
http://adsm.nerdc.ufl.edu/local/incrementals-forever.html
Picking a file at random, one created in 1999, you can see when it
was last backed up:
$ abc show backup site$disk:[username.routines]TST_AA_IF.TXT
Archive Backup Client for ADSM on OpenVMS, Version V3.1.0.1
Copyright 1996-2001, Storage Solutions Specialists, Inc.
DISK$SITE_USERS:[USERS.USERNAME.ROUTINES]TST_AA_IF.TXT;1 (A) 6097 20-SEP-2001
10:40:43
Last backed up over a year and a half ago. I know I can restore it,
as the database/tapepool containing it is fault-tolerant.
So, I disagree with this:
> once a year, if that, but I've seen the problems with differentials and
> incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
You have a poor backup solution in use, would be my first guess as
to why you so strongly oppose incrementals.
Rob
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> My comments about backup are based on my experience with literally
> thousands of customers. In the old days (not all that long ago) I'd be
> involved in hard drive failures at least once a month. Nowadays it's
> once a year, if that, but I've seen the problems with differentials and
> incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
It's almost always unavoidable.
When you've got TB, or even a few hundred GB, to backup every night and
it has to go over a network, even GB ethernet isn't enough to get it all
done in the wee hours. You have to reduce the needed bandwidth to get
it done. Incrementals are sufficient.
Howard S Shubs
you...@encompasserve.org (Rob Young) wrote:
> $ abc show backup site$disk:[username.routines]TST_AA_IF.TXT
> Archive Backup Client for ADSM on OpenVMS, Version V3.1.0.1
> Copyright 1996-2001, Storage Solutions Specialists, Inc.
> DISK$SITE_USERS:[USERS.USERNAME.ROUTINES]TST_AA_IF.TXT;1 (A) 6097 20-SEP-2001
> 10:40:43
Note that this particular syntax is very system specific. I've been
trying to keep the discussion more general when I added the
cross-posting to COV.
Rob Young
There is a TSM client available for most all OSes. The
statement about incrementals is not correct.
Rob
Tony Lawrence
A year and a half ago?????????
No comment.
>
> So, I disagree with this:
>
>
>>once a year, if that, but I've seen the problems with differentials and
>>incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
>
>
> You have a poor backup solution in use, would be my first guess as
> to why you so strongly oppose incrementals.
>
> Rob
>
I love people who join a conversation late and make wild assumptiopns.
Go read http://aplawrence.com/Unixart/diffback.html
*My* personal backup solution is bit-level verified DVD-RAM. Complete
bootable backup with recovery tools on board, able to restore the entire
thing to a virgin disk with one keystroke. My *customers* solutions
include that, qic-02, dat, helical scan, autochangers.. etc. and
software from tar to Arcserve. Generally they (at my advice) do
complete, bit verified backups.
Let me ask YOU a question: which do you think is better: backup up
everything that has changed since the last master, or just files changed
since the last backup? Why?
BTW, when was the last time you verified your 18 month old backup?
Tony Lawrence
Oh for crying out loud. How many times do I have to say it: IF YOU ARE
CONSTRAINED BY SPACE OR TIME THEN INCREMENTALS OR DIFFERENTIALS ARE
OBVIOUSLY NECESSARY.
Cripes.
Nevertheless: in many systems, as I said at
http://aplawrence.com/Unixart/diffback.html, the unchanging stuff is the
LEAST of your problems: the 300 MB of OS static files etc. is pretty
unimportant when you are backing up terabytes. If you are backing up
that kind of set, you probably have pretty damn fast storage too, so the
puny 300 MB or whatever probably doesn't add much to that.
But (because obviously the attention span of Usenet requires this) I
will once again say: IF YOU ARE CONSTRAINED BY SPACE OR TIME THEN
INCREMENTALS OR DIFFERENTIALS ARE OBVIOUSLY NECESSARY.
Cripes. Again.
Rob Young
> Rob Young wrote:
>>
>> Picking a file at random, one created in 1999, you can see when it
>> was last backed up:
>>
>> $ abc show backup site$disk:[username.routines]TST_AA_IF.TXT
>> Archive Backup Client for ADSM on OpenVMS, Version V3.1.0.1
>> Copyright 1996-2001, Storage Solutions Specialists, Inc.
>> DISK$SITE_USERS:[USERS.USERNAME.ROUTINES]TST_AA_IF.TXT;1 (A) 6097 20-SEP-2001
>> 10:40:43
>>
>> Last backed up over a year and a half ago. I know I can restore it,
>> as the database/tapepool containing it is fault-tolerant.
>
> A year and a half ago?????????
>
> No comment.
>
Backed up then, and still available today. In other words, I
could restore it right now, in less than 2 minutes if I wanted to.
Remember, I said that it is like backing up to a database.
>>
>> So, I disagree with this:
>>
>>
>>>once a year, if that, but I've seen the problems with differentials and
>>>incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
>>
>>
>> You have a poor backup solution in use, would be my first guess as
>> to why you so strongly oppose incrementals.
>>
>> Rob
>>
>
>
> Let me ask YOU a question: which do you think is better: backup up
> everything that has changed since the last master, or just files changed
> since the last backup? Why?
>
What master?
Only back the file up if it has changed.
> BTW, when was the last time you verified your 18 month old backup?
You don't understand how TSM works and overlooked that I mentioned
the files are backed up to a fault-tolerant database with offsite
copies.
If you can list it via a show command, you can restore it. Period.
Rob
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> But (because obviously the attention span of Usenet requires this) I
> will once again say: IF YOU ARE CONSTRAINED BY SPACE OR TIME THEN
> INCREMENTALS OR DIFFERENTIALS ARE OBVIOUSLY NECESSARY.
This is why I referred to the Ivory Tower, because you are ALWAYS
constrained by SPACE AND TIME. Always, no exceptions at all outside of
the Ivory Tower.
Tony Lawrence
> In article <s9ZW9.66836$hl1.7232@sccrnsc04>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>But (because obviously the attention span of Usenet requires this) I
>>will once again say: IF YOU ARE CONSTRAINED BY SPACE OR TIME THEN
>>INCREMENTALS OR DIFFERENTIALS ARE OBVIOUSLY NECESSARY.
>
>
> This is why I referred to the Ivory Tower, because you are ALWAYS
> constrained by SPACE AND TIME. Always, no exceptions at all outside of
> the Ivory Tower.
>
I have no idea what you are referring to. Many businesses have a 8 to
16 hour window in which to do backup. If their entire system can be
backed up to removable media in that time frame, they are not
constrained by space or time.
I do have customers with 24 x 7 operations, and they are harder to deal
with. If they can afford the cost, I like to handle that with logical
volume software that can do a filesystem snapshot that can be backed up
at leisure. If they can't do that, then we do what we can for the money
they can affford to spend. Sometimes incrementals are the only choice,
but I work hard to find a better way.
But the great majority of my client base are 9 to 5, leaving plenty of
time for backup. Tapes get bigger and bigger all the time, too.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I have no idea what you are referring to. Many businesses have a 8 to
> 16 hour window in which to do backup. If their entire system can be
> backed up to removable media in that time frame, they are not
> constrained by space or time.
In my experience, this is vanishingly rare. Most people like to get
home rather than watch tapes spin, even if they can take down time in
order to get a nice cold backup.
Tony Lawrence
> In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>I have no idea what you are referring to. Many businesses have a 8 to
>>16 hour window in which to do backup. If their entire system can be
>>backed up to removable media in that time frame, they are not
>>constrained by space or time.
>
>
> In my experience, this is vanishingly rare. Most people like to get
> home rather than watch tapes spin, even if they can take down time in
> order to get a nice cold backup.
>
You have to be kidding me. Your "experience" can't amount to much.
Who sits and watches tapes spin? Cron jobs do the whole thing - the
backup, the verify, the printing or emailing of results. Even paging
people if need be. What are computers for?
Even in places where there is no real down time, there is ordinarily no
human involvement. For example, we might set backups to happpen twice a
day. At 15 minutes before backup, people get an email or a screen flash
reminding them that they will have to close out for a few minutes at
whatever time has been set. When that time comes, the system checks
that everybody is out, and then nags if they aren't, and finally ejects
them forcibly if necessary. Then the snapshot is made (for those who
don't know what that is, see http://aplawrence.com/Linux/lvm.html ) and
people can get back on. The backup then begins. The whole thing takes
but a few minutes, and most of that is the nagging. More important is
that every bit of it is automatic, untouched by human hands.
Tony Lawrence
> In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>I have no idea what you are referring to. Many businesses have a 8 to
>>16 hour window in which to do backup. If their entire system can be
>>backed up to removable media in that time frame, they are not
>>constrained by space or time.
>
>
> In my experience, this is vanishingly rare. Most people like to get
> home rather than watch tapes spin, even if they can take down time in
> order to get a nice cold backup.
>
I must be an idiot. I've been arguing with you about the relative value
of backup methods and you obviously are so wet behind the ears that you
think people do backups manually.
I've been involved with computers since 1967, and have been self
employed since 1983. I have done work at thousands of sites, from one
person offices to Fidelity. I'm certified in Solaris, SCO, and Windows.
Enterprise level, not product, by the way :
http://aplawrence.com/certification.html
(I don't normally put this kind of stuff up on the bar for measurement
but this foolishness has just gone on too long).
Now tell me what in hell makes you think that anything you have to say
should be given any weight at all? I really don't mean to be offensive
here, but this "Ivory Tower" crap coming from someone so plainly
inexperienced has really ticked me off.
I make my living doing this. Do you? I work with dozens of different
backup systems from tape to dvd to jukeboxes. Do you? I work with
multiple backup software, from tar and cpio to Arcserve and beyond. Do
you? I have performed HUNDREDS of disk recoveries, rebuilds, reinstalls
and upgrades on multiple OS platforms from TRSDOS to Microsoft Xenix,
DOS, Windows NT, SCO, Solaris, and Linux. Have you?
I'm sorry to get so rabid. I'm willing to at least respect the opinions
of someone like the VMS admin who posted earlier, though I disagree with
his reliance on a database based incremental backup. I'm no longer
willing to give any respect to your opinions on this.
Sorry for the rant.
"In your experience" indeed.
Tony Lawrence
I wouldn't count on that without knowing the details of the software.
For example, many a person has beeen led down the path of doom by
assuming that a "tar tvf" listing was sufficient indication that the
files listed could in fact be restored.
I think a fault tolerant database (though too pricey for most of us)
combined with offsite copies is certainly a reasonable backup method.
That doesn't necessarily change anything though: it's still better to
backup everything every single time unless it is impossible to do so.
However, that does depend on how the offsite copies are generated. If
they are in fact full copies of everything, then this method would
satisfy me: the backup is indirect, but it is still complete. If the
copies are incrementals,, then you have the same silliness and
difficulties of the more common tape incrementals.
I would guess that the copies probably are complete?
Rob Young
>
> I'm sorry to get so rabid. I'm willing to at least respect the opinions
> of someone like the VMS admin who posted earlier, though I disagree with
> his reliance on a database based incremental backup. I'm no longer
> willing to give any respect to your opinions on this.
>
> Sorry for the rant.
>
That's okay. I agree you know your stuff. But don't
be afraid to pick up on something else totally foreign and
arguably much better.
My whole point about the incrementals is it isn't as bad as you come
off about it. And it isn't as if TSM isn't used in very
large situations.
Here is an example, easy to find dozens of similar examples:
Client is looking for a Tivoli Storage Manager Administrator to
1. Provide day to day adminstration of the TSM environment at MTO (4 TSM
Servers, STK silo, 16 STK drives, 5000 tapes, 225 clients (HP UX, VMS, NT) with
20 TB of storage).
2. Assist in indentification, evaluation, and implementation of improvements to
overall TSM environment
3. Transition knowledge to client team members so we can be self sufficient.
Day to Day Administration Activities. Day to day support activities the TSM
Administrator will perform include:
- Monitoring the environment to detect any hardware or software issues and
taking appropriate corrective action
- Monitoring system jobs to ensure backups complete in timely manner, TSM
database backups run successfully, and off-site tapes are generated
---
etc. The point is that it is an Enterprise Backup solution. There
are similar Enterprise Backups out there.
Again, incrementals are not a bad thing.
I'm not the only one relying on it. It is used in hundreds of
places. I suppose if there were any large "issues" with it, we
would hear about it. There aren't. It really is a comfort
level and I understand why many cling to their backup schemas.
Rob
Rob Young
Please don't insult us. One of the worst features of Eunuchs
is a pitiful backup design (or lack thereof).
Yes, it is possible to write something and know it got there
and to know you can read it back.
> I think a fault tolerant database (though too pricey for most of us)
> combined with offsite copies is certainly a reasonable backup method.
>
> That doesn't necessarily change anything though: it's still better to
> backup everything every single time unless it is impossible to do so.
But that is mostly unrealistic. The problem is with 200, 400
, 500 clients (servers) and the storage associated with them, there
aren't enough hours in the day to backup everything all the
time. Secondly, and just as importantly, to shrink the
backup window... if the file isn't changing, why not skip backing
it up? This is a problem. What if your only copy goes bad? Well
let us make sure we have a fault tolerant backup solution and lets
make sure we do a daily DR migration.
This isn't everybody, it is larger shops. You most likely
wouldn't have an Enterprise Backup product if you are small. There
is a transition point where even with 20-80 servers things get
unwieldy.
> However, that does depend on how the offsite copies are generated. If
> they are in fact full copies of everything, then this method would
> satisfy me: the backup is indirect, but it is still complete. If the
> copies are incrementals,, then you have the same silliness and
> difficulties of the more common tape incrementals.
I'm not a TSM administrator. I do know DR migration (or
offsite) takes about 3-4 hours and begins after
all backups are complete. This keeps your DR copy in synch
with what is onsite.
You might have better results in shooting holes in TSM if
you read a bit more about it. Here is the de facto TSM
admin site, seems to be a good resource:
> I would guess that the copies probably are complete?
Yes. You can't move the tapes until the DR migration (again, often
referred to as "offsite") is complete. I'm sure you can find
details at the above link if interested.
Rob
Rob Young
>
>> I would guess that the copies probably are complete?
>
I realize I misinterpreted that question. The copies aren't
complete copies. The database and associated files that
were backed up, lives in two places at the same time. Only the
parts that change go offsite everyday. Obviously, the database
changes and some subset of backed up storage.
Christopher Lynn
I work in a Datacenter and we've recieved five of the Xserve boxes
(very sexy looking machines and easy to use). Now the problem is that
retrospect doesn't even come close to doing what it needs to do, add
on the fact that we've already bought around a million dollars worth
of backup equipment. SUN StorEdge robotic backup system (looks like a
big purple fridge) and Legato Enterprise. Problem is the Xserve
solutions .Mac and Retrospect just aren't enterprise level and the
idea of backup to a DVD-CRW or singular connected tape drive defeats
the purpose of a large investment into (a relatively new) backup
system nor are they that secure. Now in my chunk of the datacenter I
take care of 40 servers, now 45 with the five Xserver's. I have other
things to do during the day than change tapes for five boxes, that's
why the DataCenter I work for bought a big, honkin' tape library robot
to do that and one guy manages it (not me, unfortunately).
I've worked around it by mapping drives with a korn shell script with
cron and doing a little dump of the files I know that need to be
backed up on to the SUN V120. From there I backup the files from the
SUN server. Not exactly elegant but it does the trick. Although, I
get the feeling that if push came to shove I doubt that the restores
wouldn't be that -erm- stable.
I think Apple needs to gets some partnerships inline with what's
really required for enterprise. This isn't a dig, I LOVE the gear,
it's just that I need better tools to work with this stuff.
Cheers!
Christopher
Gerry Scott-Moore <222...@adelphia.net.invalid> wrote in message news:<110120031325534317%222...@adelphia.net.invalid>...
> Sadly I only have a firewire CD burner and retrospect gathering dust in
> a dsk drawer. Since I got the G4 and the FW CDRW I simply make copies
> of directories and get as many as I can on a CD.
>
> Now, with OSX, and all it's myriad invisible files, can I simply drag a
> directory to Toas (actually operative under os 9.2.2) in order to back
> up my osx partition's directories?
>
> I can't figure out in any other way to back up and with recent Safari
> scares, I need to make my first back up and soon!
>
> Any help appreciated...
Tony Lawrence
> In article <_0cX9.770577$P31.567195@rwcrnsc53>, Tony Lawrence <to...@pcunix.com> writes:
>
>>Rob Young wrote:
>>
>>>In article <o4ZW9.62086$Yq3.12889@sccrnsc02>, Tony Lawrence <to...@pcunix.com> writes:
>>>
>>>
>>>>Rob Young wrote:
>>> If you can list it via a show command, you can restore it. Period.
>>>
>>
>>I wouldn't count on that without knowing the details of the software.
>>For example, many a person has beeen led down the path of doom by
>>assuming that a "tar tvf" listing was sufficient indication that the
>>files listed could in fact be restored.
>>
>
>
> Please don't insult us. One of the worst features of Eunuchs
> is a pitiful backup design (or lack thereof).
I'm sorry, you misunderstand. The reason I included the reference to
tar was to make it plain that I was not saying that your specific
assertion was invalid, but rather that people cannot rely on that as a
general fact. Sorry to upset you :-)
As to Unix backup, I have no idea what "backup design" is inherent to
the OS. There isn't. Nor should there be.
>
> Yes, it is possible to write something and know it got there
> and to know you can read it back.
>
>
>>I think a fault tolerant database (though too pricey for most of us)
>>combined with offsite copies is certainly a reasonable backup method.
>>
>>That doesn't necessarily change anything though: it's still better to
>>backup everything every single time unless it is impossible to do so.
>
>
> But that is mostly unrealistic. The problem is with 200, 400
> , 500 clients (servers) and the storage associated with them, there
> aren't enough hours in the day to backup everything all the
> time. Secondly, and just as importantly, to shrink the
> backup window... if the file isn't changing, why not skip backing
> it up? This is a problem. What if your only copy goes bad? Well
> let us make sure we have a fault tolerant backup solution and lets
> make sure we do a daily DR migration.
Again, I have said this over and over again: SOMETIMES THERE IS NO
OTHER OPTION TO INCREMENTAL BACKUP BECAUSE OF LIMITATIONS OF SPACE AND TIME.
Why is it necessary to repeat this over and over? Is the attention span
here really that short?
>
> This isn't everybody, it is larger shops. You most likely
> wouldn't have an Enterprise Backup product if you are small. There
> is a transition point where even with 20-80 servers things get
> unwieldy.
>
>
>>However, that does depend on how the offsite copies are generated. If
>>they are in fact full copies of everything, then this method would
>>satisfy me: the backup is indirect, but it is still complete. If the
>>copies are incrementals,, then you have the same silliness and
>>difficulties of the more common tape incrementals.
>
>
> I'm not a TSM administrator. I do know DR migration (or
> offsite) takes about 3-4 hours and begins after
> all backups are complete. This keeps your DR copy in synch
> with what is onsite.
>
> You might have better results in shooting holes in TSM if
> you read a bit more about it. Here is the de facto TSM
> admin site, seems to be a good resource:
The fact is, my argument has absolutely nothing to do with TSM. That
product is completely irrelevant to this discussion because it is
nothing but a convenient go-between to the real backup, which is the
archival of the TSM database. You may very well think of that as yout
backup, and it may even function as such under normal conditions, but it
is conceptually no different (though of course practically very
different) than simply copying data to a network drive and backing up THAT.
If you backup the TSM incrementally, then you are cruising for trouble
for all the reasons I outlined at
http://aplawrence.com/Unixart/diffback.html. If you are in fact backing
that up non-incrementally, then you are in fact doing complete backups:
the fact that the TSM feed is incremental is completly unimportant.
Rob Young
> Rob Young wrote:
>>
>> You might have better results in shooting holes in TSM if
>> you read a bit more about it. Here is the de facto TSM
>> admin site, seems to be a good resource:
>
>
> The fact is, my argument has absolutely nothing to do with TSM. That
> product is completely irrelevant to this discussion because it is
> nothing but a convenient go-between to the real backup, which is the
> archival of the TSM database.
No.
> You may very well think of that as yout
> backup, and it may even function as such under normal conditions, but it
> is conceptually no different (though of course practically very
> different) than simply copying data to a network drive and backing up THAT.
>
> If you backup the TSM incrementally, then you are cruising for trouble
> for all the reasons I outlined at
> http://aplawrence.com/Unixart/diffback.html. If you are in fact backing
> that up non-incrementally, then you are in fact doing complete backups:
> the fact that the TSM feed is incremental is completly unimportant.
>
You aren't "backing up the TSM incrementally". TSM has a database
that tracks all changes/files and each tape and disk in the
various pools are dependent on the database to track them. The
database itself is copied and DR migrated every day. And
no, you aren't doing complete backups as the servers it is tracking
are sending files that have been modified/changed, you are
performing incrementals:
http://adsm.nerdc.ufl.edu/local/incrementals-forever.html
Every backup in TSM is viewed as an incremental, even the first. The initial
backup is simply an incremental in which, coincidentally, every file is needed
to be copied. Thereafter, the backup client downloads a list of all the files
and modification times that are recorded on the server. This map is used to
determine what should be copied.
And yes, if you narrow down to the small world you are talking about,
do full backups of everything all the time - because you can, and
because they are easier to understand and support!
Rob
Tony Lawrence
> In article <pAhX9.774093$WL3.765385@rwcrnsc54>, Tony Lawrence <to...@pcunix.com> writes:
>
>>Rob Young wrote:
>
>
>>> You might have better results in shooting holes in TSM if
>>> you read a bit more about it. Here is the de facto TSM
>>> admin site, seems to be a good resource:
>>
>>
>>The fact is, my argument has absolutely nothing to do with TSM. That
>>product is completely irrelevant to this discussion because it is
>>nothing but a convenient go-between to the real backup, which is the
>>archival of the TSM database.
>
>
> No.
Yes :-)
TSM is just a more sophisticated method of copying files to another hard
drive. As you well know, as convenient and wonderful as that is, it
puts all your eggs in one basket and would be completely unacceptable if
it were not backed up to removable media in turn.
>
>
>>You may very well think of that as yout
>>backup, and it may even function as such under normal conditions, but it
>>is conceptually no different (though of course practically very
>>different) than simply copying data to a network drive and backing up THAT.
>>
>>If you backup the TSM incrementally, then you are cruising for trouble
>>for all the reasons I outlined at
>>http://aplawrence.com/Unixart/diffback.html. If you are in fact backing
>>that up non-incrementally, then you are in fact doing complete backups:
>>the fact that the TSM feed is incremental is completly unimportant.
>>
>
>
> You aren't "backing up the TSM incrementally". TSM has a database
> that tracks all changes/files and each tape and disk in the
> various pools are dependent on the database to track them. The
> database itself is copied and DR migrated every day. And
> no, you aren't doing complete backups as the servers it is tracking
> are sending files that have been modified/changed, you are
> performing incrementals:
You are missing what I am saying: how you send data to TSM is
irrelevant. How you backup TSM ITSELF is what is important.
>
> http://adsm.nerdc.ufl.edu/local/incrementals-forever.html
>
> Every backup in TSM is viewed as an incremental, even the first. The initial
> backup is simply an incremental in which, coincidentally, every file is needed
> to be copied. Thereafter, the backup client downloads a list of all the files
> and modification times that are recorded on the server. This map is used to
> determine what should be copied.
>
> And yes, if you narrow down to the small world you are talking about,
> do full backups of everything all the time - because you can, and
> because they are easier to understand and support!
Which is what I have been saying. But there's more reason than that, as
I detailed at http://aplawrence.com/Unixart/diffbacks.html : damage to a
tape in the middle of an incremental set can be a very serious loss
because the next valid tape that has certain data may be much older and
in some cases (related files) may cause the entire set to be abandoned.
Been there, seen that. More than once.
Rob Young
Yes, in your small world. The TSM backend is fault tolerant,
keep that in mind. I'm sure there is a document explaining
how it works somewhere but you really aren't interested.
Rob
Rob Young
Clarifying. A tape may fail while writing to it. That counts
as a miss. You have to run the backup again from the client
side. The tape that failed twin comes back from offsite to synch
back up.
Rob
David J. Dachtera
>
> Howard S Shubs wrote:
> > In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
> > Tony Lawrence <to...@pcunix.com> wrote:
> >
> >
> >>I have no idea what you are referring to. Many businesses have a 8 to
> >>16 hour window in which to do backup. If their entire system can be
> >>backed up to removable media in that time frame, they are not
> >>constrained by space or time.
> >
> >
> > In my experience, this is vanishingly rare. Most people like to get
> > home rather than watch tapes spin, even if they can take down time in
> > order to get a nice cold backup.
> >
>
> I must be an idiot. I've been arguing with you about the relative value
> of backup methods and you obviously are so wet behind the ears that you
> think people do backups manually.
If you know how to automate backups including:
o Shutdown the running system
o Boot up the stand-alone CD
o Kick-off a system-disk backup
o Shutdown the stand-alone system
o Reboot the live, production system, sans app.'s
o Backup the production application disks
o Restart the applications.
...you'll not only become a very wealthy man, you'll fulfill the dreams
of many denizens of this group.
Actually, I could see doing that with a very sophisticated Reflection
script (RCL or RBS, take your choice).
Oh, yeah, there's that zero-downtime thing to consider...
--
David J. Dachtera
dba DJE Systems
http://www.djesys.com/
Unofficial Affordable OpenVMS Home Page:
http://www.djesys.com/vms/soho/
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I make my living doing this.
I do system management for a living these days, yes.
> I work with dozens of different
> backup systems from tape to dvd to jukeboxes. Do you?
Backups at my current location are mainly done by others in my group,
using TSM and a good sized robotic solution tied to an AIX box.
> and upgrades on multiple OS platforms from TRSDOS to Microsoft Xenix,
> DOS, Windows NT, SCO, Solaris, and Linux. Have you?
I've worked with Mac and VMS, mainly. Done a little with Linux and AIX
as well. I skip the MS stuff as much as possible.
> I'm sorry to get so rabid. I'm willing to at least respect the opinions
> of someone like the VMS admin who posted earlier, though I disagree with
> his reliance on a database based incremental backup. I'm no longer
> willing to give any respect to your opinions on this.
As you wish.
> "In your experience" indeed.
You'd prefer I didn't use the qualifier?
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> Who sits and watches tapes spin? Cron jobs do the whole thing - the
Okay, so I used a weak argument. Geez.
Ron Shepard
has always been some kind of media that is cheaper than the primary
disk/drum/etc storage. Back in the 70's for example, disk space
cost about $100/MB and tape storage cost about $1/MB. In the past
few years, the cost of disk space has decreased tremendously, about
a factor of 100,000 since the 70's. HD storage is now about $1/GB.
However, tape costs have not changed as much, a GB of tape storage
now costs maybe 4 or 5 times less than it did 10 years ago. DVD-R
is still about $2/GB (not counting the drive, just the media), and
CD-R is comparable. DLT is about $1/GB.
We now have a situation where hard disk storage is either cheaper,
or comprable, to the cost of the media that is used to back it up.
It makes you wonder what is the point. Why not simply buy more hard
drives, back up the data, power them down and store them in a vault?
How will this situation change in the future? Disk storage will
almost certainly get cheaper in the future. Is there any new
storage media that has the ability to keep up with disk storage
costs? What happens in 10 years if disk storage is cheaper by a
factor of 1000 than current prices, but tape/DVD costs are only 4 or
5 times cheaper?
$.02 -Ron Shepard
Eberhard Heuser-Hofmann
In article <ron-shepard-B21C...@netnews.attbi.com>, Ron
Shepard <ron-s...@NOSPAM.attbi.com> writes:
|>Throughout the history of computing (since the 50's at least), there
|>has always been some kind of media that is cheaper than the primary
|>disk/drum/etc storage. Back in the 70's for example, disk space
|>cost about $100/MB and tape storage cost about $1/MB. In the past
|>few years, the cost of disk space has decreased tremendously, about
|>a factor of 100,000 since the 70's. HD storage is now about $1/GB.
|>However, tape costs have not changed as much, a GB of tape storage
|>now costs maybe 4 or 5 times less than it did 10 years ago. DVD-R
|>is still about $2/GB (not counting the drive, just the media), and
|>CD-R is comparable. DLT is about $1/GB.
|>
|>We now have a situation where hard disk storage is either cheaper,
|>or comprable, to the cost of the media that is used to back it up.
|>It makes you wonder what is the point. Why not simply buy more hard
|>drives, back up the data, power them down and store them in a vault?
|>
I've read an article in a german computer magazine, that states,
that it is nowadays cheaper to throw off the DLT-Tape library and to buy
a IDE-based raidsystem.
|>How will this situation change in the future? Disk storage will
|>almost certainly get cheaper in the future. Is there any new
|>storage media that has the ability to keep up with disk storage
|>costs? What happens in 10 years if disk storage is cheaper by a
|>factor of 1000 than current prices, but tape/DVD costs are only 4 or
|>5 times cheaper?
There is difference between backup and archiving. For archiving
purpose use CD/DVD's for backup tape or disks. DVD's with
much higher capacity than today will be out this or next year
(keyword "blue laser"). But the drives will be very expensive
in the beginning as usual.
|>
|>$.02 -Ron Shepard
|>
eberhard
Tony Lawrence
As I've tried to explain, TSM is completely irrelevant to this. It's a
wonderful thing, yes, but it would be HOW YOU BACK THAT UP that is
important and relevant.
And my world is hardly small :-) Kind of a funny thing for a VMS guy to
say, you know?
Tony Lawrence
A tape may fail ANYTIME. Tapes fail while being read. Tapes get eaten
by rats (well, yeah, that wasn't one of my higher class clients), lost,
mislabeled, stepped on etc. That's why the more redundancy you have
with tapes, the safer you are. Again, I've been there and seen this
kind of thing first hand. Apparently in your oh so larger than my world
the unexpected never happens.
Tony Lawrence
> Tony Lawrence wrote:
>
>>Howard S Shubs wrote:
>>
>>>In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
>>> Tony Lawrence <to...@pcunix.com> wrote:
>>>
>>>
>>>
>>>>I have no idea what you are referring to. Many businesses have a 8 to
>>>>16 hour window in which to do backup. If their entire system can be
>>>>backed up to removable media in that time frame, they are not
>>>>constrained by space or time.
>>>
>>>
>>>In my experience, this is vanishingly rare. Most people like to get
>>>home rather than watch tapes spin, even if they can take down time in
>>>order to get a nice cold backup.
>>>
>>
>>I must be an idiot. I've been arguing with you about the relative value
>>of backup methods and you obviously are so wet behind the ears that you
>>think people do backups manually.
>
>
> If you know how to automate backups including:
>
> o Shutdown the running system
> o Boot up the stand-alone CD
Not necessary under any conditions. I suspect you haven't much
experience at this.
> o Kick-off a system-disk backup
> o Shutdown the stand-alone system
> o Reboot the live, production system, sans app.'s
> o Backup the production application disks
> o Restart the applications.
>
> ...you'll not only become a very wealthy man, you'll fulfill the dreams
> of many denizens of this group.
Sigh. All of those things are done every day. Your first two steps
aren't necessary, but even that could be easily automated and in fact
I've done things like that for other reasons. Not necessary for backup
though.
>
> Actually, I could see doing that with a very sophisticated Reflection
> script (RCL or RBS, take your choice).
>
> Oh, yeah, there's that zero-downtime thing to consider...
Few of my clients get that, although we can come close with disk
snapshots. If you REALLY need zero down time, that requires cooperation
from the application software: it has to be able to bring all data files
into sync and buffer all transactions while the snapshot is being
prepared. As that literally takes seconds, that should be no hardship,
but I don't have anything with that feature.
There's nothing technically difficult about it though and everything
else is simple and BEING DONE EVERY DAY. Nothing particularly
sophisticated about it.
Tony Lawrence
If the drives are cheap enough that you can maintain the depth of backup
that you need, why not? Indeed, many of us could afford to do that now:
while I don't particularly want to tie up the cost of a dozen or so disk
drives, that really isn't all that much money nowadays. As the disks
are much more rugged than they used to be, it wouldn't be scary to
transport them home offsite casually.
But if we free up the radio spectrum and have high speed "pipes"
everywhere, maybe we don't need to physically move those drives around
at all?
Tony Lawrence
> In article <ron-shepard-B21C...@netnews.attbi.com>, Ron
> Shepard <ron-s...@NOSPAM.attbi.com> writes:
> |>Throughout the history of computing (since the 50's at least), there
> |>has always been some kind of media that is cheaper than the primary
> |>disk/drum/etc storage. Back in the 70's for example, disk space
> |>cost about $100/MB and tape storage cost about $1/MB. In the past
> |>few years, the cost of disk space has decreased tremendously, about
> |>a factor of 100,000 since the 70's. HD storage is now about $1/GB.
> |>However, tape costs have not changed as much, a GB of tape storage
> |>now costs maybe 4 or 5 times less than it did 10 years ago. DVD-R
> |>is still about $2/GB (not counting the drive, just the media), and
> |>CD-R is comparable. DLT is about $1/GB.
> |>
> |>We now have a situation where hard disk storage is either cheaper,
> |>or comprable, to the cost of the media that is used to back it up.
> |>It makes you wonder what is the point. Why not simply buy more hard
> |>drives, back up the data, power them down and store them in a vault?
> |>
>
> I've read an article in a german computer magazine, that states,
> that it is nowadays cheaper to throw off the DLT-Tape library and to buy
> a IDE-based raidsystem.
>
But it's not suitable for backup. Not by itself. That's just a cheaper
and less functional TSM system: you still need to back that up to
removable media. As you say:
>
> There is difference between backup and archiving. For archiving
> purpose use CD/DVD's for backup tape or disks. DVD's with
> much higher capacity than today will be out this or next year
> (keyword "blue laser"). But the drives will be very expensive
> in the beginning as usual.
Right. I like DVDRAM for small systems and if they can boost the
capacity up it has advantages over tapes in some ways (though it
probably will never match tape write speeds).
Paul Repacholi
> I've read an article in a german computer magazine, that states,
> that it is nowadays cheaper to throw off the DLT-Tape library and to
> buy a IDE-based raidsystem.
RIGHT...
Now burn down the building. Tell me when you get the data off your
IDE raid.
BTW, you do know why it is called IDE, don't you :)
--
Paul Repacholi 1 Crescent Rd.,
+61 (08) 9257-1001 Kalamunda.
West Australia 6076
comp.os.vms,- The Older, Grumpier Slashdot
Raw, Cooked or Well-done, it's all half baked.
EPIC, The Architecture of the future, always has been, always will be.
Eberhard Heuser-Hofmann
In article <87vg0hb...@prep.synonet.com>, Paul Repacholi
<pr...@prep.synonet.com> writes:
|>vax...@chclu.chemie.uni-konstanz.de (Eberhard Heuser-Hofmann) writes:
|>
|>> I've read an article in a german computer magazine, that states,
|>> that it is nowadays cheaper to throw off the DLT-Tape library and to
|>> buy a IDE-based raidsystem.
|>
|>RIGHT...
|>
|>Now burn down the building. Tell me when you get the data off your
|>IDE raid.
And tell me when you get the data off your DLT-Library if
there is fire in that room.
|>
|>BTW, you do know why it is called IDE, don't you :)
Use raid10 (Raid5+mirroring). Even if a IDE-disk fails,
you will have no data loss.
eberhard
Tony Lawrence
> In article <87vg0hb...@prep.synonet.com>, Paul Repacholi
> <pr...@prep.synonet.com> writes:
> |>vax...@chclu.chemie.uni-konstanz.de (Eberhard Heuser-Hofmann) writes:
> |>
> |>> I've read an article in a german computer magazine, that states,
> |>> that it is nowadays cheaper to throw off the DLT-Tape library and to
> |>> buy a IDE-based raidsystem.
> |>
> |>RIGHT...
> |>
> |>Now burn down the building. Tell me when you get the data off your
> |>IDE raid.
>
> And tell me when you get the data off your DLT-Library if
> there is fire in that room.
You get it from the tapes you rotated off site, of course.
> |>
> |>BTW, you do know why it is called IDE, don't you :)
>
> Use raid10 (Raid5+mirroring). Even if a IDE-disk fails,
> you will have no data loss.
Well, no data loss unless the two drives of the raid5 fail..
Backing up to a disk drive is quick, convenient, and insufficient. You
ALWAYS need removable media for archival storage. Before some damn fool
says "just buy a bigger drive and store multiple sets there", THAT puts
too much stuff at risk in one place.
Bob Koehler
> We now have a situation where hard disk storage is either cheaper,
> or comprable, to the cost of the media that is used to back it up.
> It makes you wonder what is the point. Why not simply buy more hard
> drives, back up the data, power them down and store them in a vault?
Back when it was still the VAXLUG we had a presentation near here
from a thrid party backup vendor. He was asked why do tape backups
instead of disk to disk and proceeded to knock a 9-track tape from
the podium onto the floor.
"I'll guarantee I can get most of the data, probably all of it, off
that tape. Just try that with a backup on disk."
Bob Koehler
> vax...@chclu.chemie.uni-konstanz.de (Eberhard Heuser-Hofmann) writes:
>
>> I've read an article in a german computer magazine, that states,
>> that it is nowadays cheaper to throw off the DLT-Tape library and to
>> buy a IDE-based raidsystem.
>
Hm. I've got a 4mm drive and about ½ TB worth of tape for free.
You can get disk cheaper than that?
Keith A. Lewis
> Hm. I've got a 4mm drive and about ½ TB worth of tape for free.
> You can get disk cheaper than that?
I cut my HW maintenance (contract) costs in half for several Alphas by
removing the 4mm drive. They break a lot.
--Keith Lewis klewis$mitre.org
The above may not (yet) represent the opinions of my employer.
JF Mezei
If you have volume shadowing across buildings, the issue of off-site backup is
not as bad. Although you would have to keep enough drives to keep enough
backups (eg, if you need 2 months at once a day, that would be 60 drives and
that makes managing the drives "interesting". Drives are generally not
designed to be removable media that you switch every day.
The advantage of tapes is that it makes it easy to store in a vault. Although
I guess one could also do the same with drives, but again, everytime you
handle a disk drive, you have greater risks of damaging it and/or the data.
David J. Dachtera
>
> David J. Dachtera wrote:
> > Tony Lawrence wrote:
> >
> >>Howard S Shubs wrote:
> >>
> >>>In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
> >>> Tony Lawrence <to...@pcunix.com> wrote:
> >>>
> >>>
> >>>
> >>>>I have no idea what you are referring to. Many businesses have a 8 to
> >>>>16 hour window in which to do backup. If their entire system can be
> >>>>backed up to removable media in that time frame, they are not
> >>>>constrained by space or time.
> >>>
> >>>
> >>>In my experience, this is vanishingly rare. Most people like to get
> >>>home rather than watch tapes spin, even if they can take down time in
> >>>order to get a nice cold backup.
> >>>
> >>
> >>I must be an idiot. I've been arguing with you about the relative value
> >>of backup methods and you obviously are so wet behind the ears that you
> >>think people do backups manually.
> >
> >
> > If you know how to automate backups including:
> >
> > o Shutdown the running system
> > o Boot up the stand-alone CD
>
> Not necessary under any conditions.
I'll refer you to OpenVMS engineering for a correction.
> I suspect you haven't much
> experience at this.
19 years with VMS, 24 years in the business. See my website. Former
OpenVMS ASE.
Try again.
> > o Kick-off a system-disk backup
> > o Shutdown the stand-alone system
> > o Reboot the live, production system, sans app.'s
> > o Backup the production application disks
> > o Restart the applications.
> >
> > ...you'll not only become a very wealthy man, you'll fulfill the dreams
> > of many denizens of this group.
>
> Sigh. All of those things are done every day. Your first two steps
> aren't necessary,
Well, actually they are. Most of us "cheat" at it by doing a /IMAGE
BACKUP of our live system disk while the cluster is up and running since
we simply cannot afford the downtime it requires, nor do we have savvy
personnel on hand at 00:00 to perform the task.
Again, contact OpenVMS Engineering or the CSC for the supported system
disk backup procedure as documented.
> but even that could be easily automated and in fact
> I've done things like that for other reasons. Not necessary for backup
> though.
Well, actually yes it is. It's a gamble (calculated risk, really) to do
it the way we all do it, but the instances of a failure at DR test time
have been so low as to not prohibit using the "cheat".
> >
> > Actually, I could see doing that with a very sophisticated Reflection
> > script (RCL or RBS, take your choice).
> >
> > Oh, yeah, there's that zero-downtime thing to consider...
>
> Few of my clients get that, although we can come close with disk
> snapshots. If you REALLY need zero down time, that requires cooperation
> from the application software: it has to be able to bring all data files
> into sync and buffer all transactions while the snapshot is being
> prepared. As that literally takes seconds, that should be no hardship,
> but I don't have anything with that feature.
No one has it. It's eutopia.
> There's nothing technically difficult about it though and everything
> else is simple and BEING DONE EVERY DAY. Nothing particularly
> sophisticated about it.
That last comment suggest YOU may lack some experience with large-scale
commercial systems.
I spent two years writing BACKUP automation for large Chicago-area
healthcare concern. It runs with tape library automation (*NOT* SLS) and
splits and rejoins shadow- and mirror-sets as needed, and rotates
through sets of production environments, three a night to cover seven
production environments.
There's frequently a very large difference between what CAN be done and
what SHOULD be done. Say, "compromise".