How to Backup OSX
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I think you are misunderstanding something here. How can a backup know
> whether or not a file that it backed up should or should not be
> restored? It can't. There's also the matter of losing one piece of
> thev differential, as the article explained.
Okay, I'll explain how it's done, if I can remember.
A backup set, be it a full backup or an incremental, stores file
attributes as well as data. By comparing the "date of last
modification", you can determine if a file on disk is to be replaced by
one on tape. That covers restoring only the latest copy, yes?
So, how does it know to -delete- old files? IIRC, by storing directory
information as well, a directory being a kind of file. If a newer
version of a directory file doesn't contain file X.Y, then file X.Y
shouldn't continue to exist, and it gets deleted. I might be
remembering wrong, though. I'll try cross-posting to a group which
would know.
> Differentials are NOT the best way. They are acceptable if there is no
> other way for you to do complete backups. Period. If you don't
> understand that, you'll someday wish that you did.
No, that's what I was saying. We're in violent agreement. Full backups
followed by a series of incrementals is the way to go.
> Trust me: I've seen more disasters than you'd ever want to know about.
I haven't, because I've always been able to recover from data loss, or
near as dammit, since my first few weeks in the industry.
--
Today, on Paper-view: The World Origami Championship
Tony Lawrence
> In article <4wlW9.731686$%m4.34...@rwcrnsc52.ops.asp.att.net>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>I think you are misunderstanding something here. How can a backup know
>>whether or not a file that it backed up should or should not be
>>restored? It can't. There's also the matter of losing one piece of
>>thev differential, as the article explained.
>
>
> Okay, I'll explain how it's done, if I can remember.
>
> A backup set, be it a full backup or an incremental, stores file
> attributes as well as data. By comparing the "date of last
> modification", you can determine if a file on disk is to be replaced by
> one on tape. That covers restoring only the latest copy, yes?
>
> So, how does it know to -delete- old files? IIRC, by storing directory
> information as well, a directory being a kind of file. If a newer
> version of a directory file doesn't contain file X.Y, then file X.Y
> shouldn't continue to exist, and it gets deleted. I might be
> remembering wrong, though. I'll try cross-posting to a group which
> would know.
>
Sheesh. I understand the algorithm it could use, what I'm trying to get
through to you is that it is insufficient. The whole point of backup is
recover data. Sometimes files get deleted that should not have been and
that fact isn't always discovered immediately. Sometimes the fact that
something is gone isn't even known: all that is known is that something
that used to work does not. Couple that with some bad timing and a full
restore and this smart differential backup finds the file on its master
but then deletes it yet again on the differerential.. wonderful.
>
>
>>Differentials are NOT the best way. They are acceptable if there is no
>>other way for you to do complete backups. Period. If you don't
>>understand that, you'll someday wish that you did.
>
>
> No, that's what I was saying. We're in violent agreement. Full backups
> followed by a series of incrementals is the way to go.
No. FULL BACKUPS EACH AND EVERY TIME.
:-)
>
>
>
>>Trust me: I've seen more disasters than you'd ever want to know about.
>
>
> I haven't, because I've always been able to recover from data loss, or
> near as dammit, since my first few weeks in the industry.
>
I don't mean to offend, but that's incredibly naive. I've been involved
with computer systems since 1967. I have heard the wailing and seen the
gnashing of teeth. Bad things happen to backup systems, and when the
shit has really hit the fan and splattered all over the room, the
ability to pull a complete backup from three days back or even last
month can be the difference between a lot of hard work and complete
disaster.
If you can't do full backups because it takes too long or you don't have
big enough storage, fine: do differentials. As I have said though, the
usual case is that the static files are often the least of your problems
in space and time, and large, fast archival systems get cheaper and
cheaper all the time anyway.
--
Tony Lawrence
Free SCO, Mac OS X and Linux Skills Tests:
http://aplawrence.com/skillstest.html
John Santos
to...@pcunix.com says...
Err, if you know a particular file is missing, you restore it
explicitly. You don't do a full restore unless you are recovering a
trashed disk.
For some reason this thread is cross-posted between comp.os.vms and
comp.sys.mac.apps. I don't know anything about Mac backups, but
VMS backup certainly allows you to do selective restores of any
file or group of files from an image, differential or incremental
backup. If you do a selective restore, it restores the files you
ask for (including everything if you ask it nicely) and doesn't
delete anything. It is explicitly *not* trying to recreate the
disk at the time of the most recent backup in this case.
>
> >
> >
> >>Differentials are NOT the best way. They are acceptable if there is no
> >>other way for you to do complete backups. Period. If you don't
> >>understand that, you'll someday wish that you did.
> >
> >
> > No, that's what I was saying. We're in violent agreement. Full backups
> > followed by a series of incrementals is the way to go.
>
> No. FULL BACKUPS EACH AND EVERY TIME.
>
>
> :-)
>
>
> >
> >
> >
> >>Trust me: I've seen more disasters than you'd ever want to know about.
> >
> >
> > I haven't, because I've always been able to recover from data loss, or
> > near as dammit, since my first few weeks in the industry.
> >
>
> I don't mean to offend, but that's incredibly naive. I've been involved
> with computer systems since 1967. I have heard the wailing and seen the
> gnashing of teeth. Bad things happen to backup systems, and when the
> shit has really hit the fan and splattered all over the room, the
> ability to pull a complete backup from three days back or even last
> month can be the difference between a lot of hard work and complete
> disaster.
>
> If you can't do full backups because it takes too long or you don't have
> big enough storage, fine: do differentials. As I have said though, the
> usual case is that the static files are often the least of your problems
> in space and time, and large, fast archival systems get cheaper and
> cheaper all the time anyway.
>
>
>
>
--
John
Tony Lawrence
I said: " Couple that with some bad timing and a full restore" ..
>
> For some reason this thread is cross-posted between comp.os.vms and
> comp.sys.mac.apps. I don't know anything about Mac backups, but
> VMS backup certainly allows you to do selective restores of any
> file or group of files from an image, differential or incremental
> backup. If you do a selective restore, it restores the files you
> ask for (including everything if you ask it nicely) and doesn't
> delete anything. It is explicitly *not* trying to recreate the
> disk at the time of the most recent backup in this case.
Sigh. NOT THE POINT.
Systems crash. It's funny in a way, but the reliability of modern
systems contributes to poor backup practice. Disk failures used to be a
fairly common thing for me to deal with, but now I can go a year or
more without seeing even one. Because of this, people tend to get
sloppy: I have customers now who have NEVER had a hardware disk failure.
So you take someone like that who doesn't have a paranoid
administrator or a PITA consultant who yells at them when he finds
backups in disarray :-) and you have a recipe for disaster. Tapes are
lost, improperly labeled, overwritten, unverified, frozen, whatever.
Now your "smart differential" does its restore dance, and deletes a file
that never should have been deleted. If you are lucky, the problem is
noticed immediately. If you are not lucky, the need for that file
doesn't come up until every tape that had it has been long overwritten.
Or, you are even less lucky and one of your differentials snaps, is
defective, is lost, mislabeled whatever. Lucky you. What now? Go back
to the previous set of course - how old is that?
The more redundancy, the better. Period. I really cannot understand
why something so simple and obvious has caused such contention.
The BEST backup is a COMPLETE backup each and every time. If it is
impossible to do that because of space or time issues, then AND ONLY
THEN should you use differentials.
Sheesh :-)
James L. Ryan
(in message <d8yW9.738625$%m4.34...@rwcrnsc52.ops.asp.att.net>):
> The BEST backup is a COMPLETE backup each and every time. If it is
> impossible to do that because of space or time issues, then AND ONLY
> THEN should you use differentials.
Perhaps it has already been mentioned in this thread of discussion, but
Retrospect (and Retrospect Express) allow one to make three different kinds
of backups:
1) an incremental backup which augments the backup with only those items
which are new or which have changed. Such a backup allows one to selectively
restore any version of an item that at any time has been included in the
backup set. The structure of an incremental backup is a proprietary single
file which requires Retrospect or Retrospect Express when recovering items.
2) a duplicate backup which replaces in the backup only those items that have
are new or which have changed. A duplicate backup can only be made to a hard
disk or to memory. The structure of the duplicate backup is logically
identical to the original and items may be freely moved, without the need for
Retrospect or Retrospect Express, at any time from the backup back into to
the active set.
3) an archival backup which is similar to an incremental backup with the
difference being that designated items are also removed from the active set.
Depending on the user's choice, as many backup sets can be specified as the
user wishes, the backup sets can be a mixture of the above, and the backup
schedule can be automated.
-- James L. Ryan -- TaliesinSoft
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> Sheesh. I understand the algorithm it could use, what I'm trying to get
> through to you is that it is insufficient. The whole point of backup is
> recover data. Sometimes files get deleted that should not have been and
> that fact isn't always discovered immediately. Sometimes the fact that
Which is when you go recover a specific file, which capability isn't
lost from using incrementals. I'm talking about the ability to restore
entire volumes.
> :-)
<rolls eyes>
> I don't mean to offend, but that's incredibly naive. I've been involved
> with computer systems since 1967. I have heard the wailing and seen the
> gnashing of teeth. Bad things happen to backup systems, and when the
> shit has really hit the fan and splattered all over the room, the
> ability to pull a complete backup from three days back or even last
> month can be the difference between a lot of hard work and complete
> disaster.
Sure, and I can do it.
Tony Lawrence
> In article <3RvW9.615140$GR5.4...@rwcrnsc51.ops.asp.att.net>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>Sheesh. I understand the algorithm it could use, what I'm trying to get
>>through to you is that it is insufficient. The whole point of backup is
>>recover data. Sometimes files get deleted that should not have been and
>>that fact isn't always discovered immediately. Sometimes the fact that
>
>
> Which is when you go recover a specific file, which capability isn't
> lost from using incrementals. I'm talking about the ability to restore
> entire volumes.
I said (but apparently have to keep saying it over and over) that I am
referring to the case where YOU DON'T KNOW THE FILE IS MISDSING.
>
>
>
>>:-)
>
>
> <rolls eyes>
>
>
>>I don't mean to offend, but that's incredibly naive. I've been involved
>>with computer systems since 1967. I have heard the wailing and seen the
>>gnashing of teeth. Bad things happen to backup systems, and when the
>>shit has really hit the fan and splattered all over the room, the
>>ability to pull a complete backup from three days back or even last
>>month can be the difference between a lot of hard work and complete
>>disaster.
>
>
> Sure, and I can do it.
>
Best of luck to you. May you live long enough to regret this decision.
Never wrestle with a pig. It gets you dirty and annoys the pig.
Howard S Shubs
John Santos <john....@post.harvard.edu> wrote:
> For some reason this thread is cross-posted between comp.os.vms and
> comp.sys.mac.apps.
Yes, I did that. We're talking about backup methods, and I know backups
are backups, on VMS or Macintosh, where I know both systems, so I
cross-posted in order to get more contributors to discuss backup
methodology.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I said: " Couple that with some bad timing and a full restore" ..
I don't see it. Since you have multiple backup sets, and because
restoring a volume doesn't destroy the restore tapes, what's the problem?
> Sheesh :-)
I can't say you're clueless, 'cause you're not, but you're ignoring our
point.
Tony Lawrence
> In article <d8yW9.738625$%m4.34...@rwcrnsc52.ops.asp.att.net>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>I said: " Couple that with some bad timing and a full restore" ..
>
>
> I don't see it. Since you have multiple backup sets, and because
> restoring a volume doesn't destroy the restore tapes, what's the problem?
>
>
>
>>Sheesh :-)
>
>
> I can't say you're clueless, 'cause you're not, but you're ignoring our
> point.
>
Exactly what I am thinking of you, strangely enough..
But I'm tired of slapping this horse's rump. I remain convinced that
anything other than complete backups is inferior and should never be
done unless there is no other choice.
You remain convinced otherwise.
May neither of us need any damn backups anyway :-)
JF Mezei
> But I'm tired of slapping this horse's rump. I remain convinced that
> anything other than complete backups is inferior and should never be
> done unless there is no other choice.
It depends on the backup utility, and the robustness of the operating system
and file system. For VMS, one can have a full backup complemented with
incremental backups. It saves time every day when backups are done, but it
takes longer to find the file you need on those rare occasions when you need
to restore a file.
With VMS, you can generate a backup journal which lists all the files that are
being written to tape, so it is fairly easy to check which backup contains the
file you want.
If managed properly, a full backup complemented with incrementals is not a
problem to restore when you have the right robust OS. Just takes longer to do
and better planning/understanding, as well as properly scheduling how often
you do a full backup.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I said (but apparently have to keep saying it over and over) that I am
> referring to the case where YOU DON'T KNOW THE FILE IS MISDSING.
And as I said, the backup set isn't erased when you restore it, so you
can go back and restore files from earlier snapshots if some file was
deleted by accident. Clear yet?
> Best of luck to you. May you live long enough to regret this decision.
Don't think it'll happen, guy. I lose disks all the time and the data's
fine.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> You remain convinced otherwise.
Yup. I'm mired in reality, rather than ivory tower theorising.
> May neither of us need any damn backups anyway :-)
But we do! We do! That's the biggest system management task of them
all. I think we're both aware of that.
Tony Lawrence
> In article <lXDW9.56181$1q3.7145@sccrnsc01>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>You remain convinced otherwise.
>
>
> Yup. I'm mired in reality, rather than ivory tower theorising.
THAT I cannot ignore :-)
I'm an independent, and have been since 1983. I'm not a var pushing
product, I'm not a contract admin who stays at the same place for months
at a time: I'm a guy who is out in the trenches every day, flitting from
customer to customer ('cept nowadays my flitting is more ssh than
automobile).
My comments about backup are based on my experience with literally
thousands of customers. In the old days (not all that long ago) I'd be
involved in hard drive failures at least once a month. Nowadays it's
once a year, if that, but I've seen the problems with differentials and
incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
If that's "ivory tower", I'm a 15th century blacksmith having
hallucinatory visions.
That's it. Go and sin no more :-)
Rob Young
>
> My comments about backup are based on my experience with literally
> thousands of customers. In the old days (not all that long ago) I'd be
> involved in hard drive failures at least once a month. Nowadays it's
> once a year, if that, but I've seen the problems with differentials and
> incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
>
> If that's "ivory tower", I'm a 15th century blacksmith having
> hallucinatory visions.
>
No, just very out of date.
I do incrementals - always. Folks are somewhat uncomfortable with
that. The warm and fuzzy is to do (typical) weekly fulls, daily
incrementals back to the weeklies. With TSM, you are always
doing incrementals.
The problem is conceptual. Think of your backups taking place to
a fault-tolerant database. Each day, an export goes off-site so
your backed up files reside in two places (this is typical for most
of us regardless of backup solution).
What is occuring is files that are *not* changing are only
backed up initially! incrementals forever
http://adsm.nerdc.ufl.edu/local/incrementals-forever.html
Picking a file at random, one created in 1999, you can see when it
was last backed up:
$ abc show backup site$disk:[username.routines]TST_AA_IF.TXT
Archive Backup Client for ADSM on OpenVMS, Version V3.1.0.1
Copyright 1996-2001, Storage Solutions Specialists, Inc.
DISK$SITE_USERS:[USERS.USERNAME.ROUTINES]TST_AA_IF.TXT;1 (A) 6097 20-SEP-2001
10:40:43
Last backed up over a year and a half ago. I know I can restore it,
as the database/tapepool containing it is fault-tolerant.
So, I disagree with this:
> once a year, if that, but I've seen the problems with differentials and
> incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
You have a poor backup solution in use, would be my first guess as
to why you so strongly oppose incrementals.
Rob
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> My comments about backup are based on my experience with literally
> thousands of customers. In the old days (not all that long ago) I'd be
> involved in hard drive failures at least once a month. Nowadays it's
> once a year, if that, but I've seen the problems with differentials and
> incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
It's almost always unavoidable.
When you've got TB, or even a few hundred GB, to backup every night and
it has to go over a network, even GB ethernet isn't enough to get it all
done in the wee hours. You have to reduce the needed bandwidth to get
it done. Incrementals are sufficient.
Howard S Shubs
you...@encompasserve.org (Rob Young) wrote:
> $ abc show backup site$disk:[username.routines]TST_AA_IF.TXT
> Archive Backup Client for ADSM on OpenVMS, Version V3.1.0.1
> Copyright 1996-2001, Storage Solutions Specialists, Inc.
> DISK$SITE_USERS:[USERS.USERNAME.ROUTINES]TST_AA_IF.TXT;1 (A) 6097 20-SEP-2001
> 10:40:43
Note that this particular syntax is very system specific. I've been
trying to keep the discussion more general when I added the
cross-posting to COV.
Rob Young
There is a TSM client available for most all OSes. The
statement about incrementals is not correct.
Rob
Tony Lawrence
A year and a half ago?????????
No comment.
>
> So, I disagree with this:
>
>
>>once a year, if that, but I've seen the problems with differentials and
>>incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
>
>
> You have a poor backup solution in use, would be my first guess as
> to why you so strongly oppose incrementals.
>
> Rob
>
I love people who join a conversation late and make wild assumptiopns.
Go read http://aplawrence.com/Unixart/diffback.html
*My* personal backup solution is bit-level verified DVD-RAM. Complete
bootable backup with recovery tools on board, able to restore the entire
thing to a virgin disk with one keystroke. My *customers* solutions
include that, qic-02, dat, helical scan, autochangers.. etc. and
software from tar to Arcserve. Generally they (at my advice) do
complete, bit verified backups.
Let me ask YOU a question: which do you think is better: backup up
everything that has changed since the last master, or just files changed
since the last backup? Why?
BTW, when was the last time you verified your 18 month old backup?
Tony Lawrence
Oh for crying out loud. How many times do I have to say it: IF YOU ARE
CONSTRAINED BY SPACE OR TIME THEN INCREMENTALS OR DIFFERENTIALS ARE
OBVIOUSLY NECESSARY.
Cripes.
Nevertheless: in many systems, as I said at
http://aplawrence.com/Unixart/diffback.html, the unchanging stuff is the
LEAST of your problems: the 300 MB of OS static files etc. is pretty
unimportant when you are backing up terabytes. If you are backing up
that kind of set, you probably have pretty damn fast storage too, so the
puny 300 MB or whatever probably doesn't add much to that.
But (because obviously the attention span of Usenet requires this) I
will once again say: IF YOU ARE CONSTRAINED BY SPACE OR TIME THEN
INCREMENTALS OR DIFFERENTIALS ARE OBVIOUSLY NECESSARY.
Cripes. Again.
Rob Young
> Rob Young wrote:
>>
>> Picking a file at random, one created in 1999, you can see when it
>> was last backed up:
>>
>> $ abc show backup site$disk:[username.routines]TST_AA_IF.TXT
>> Archive Backup Client for ADSM on OpenVMS, Version V3.1.0.1
>> Copyright 1996-2001, Storage Solutions Specialists, Inc.
>> DISK$SITE_USERS:[USERS.USERNAME.ROUTINES]TST_AA_IF.TXT;1 (A) 6097 20-SEP-2001
>> 10:40:43
>>
>> Last backed up over a year and a half ago. I know I can restore it,
>> as the database/tapepool containing it is fault-tolerant.
>
> A year and a half ago?????????
>
> No comment.
>
Backed up then, and still available today. In other words, I
could restore it right now, in less than 2 minutes if I wanted to.
Remember, I said that it is like backing up to a database.
>>
>> So, I disagree with this:
>>
>>
>>>once a year, if that, but I've seen the problems with differentials and
>>>incrementals and I DO NOT RECOMMEND THEIR USE UNLESS IT IS UNAVOIDABLE.
>>
>>
>> You have a poor backup solution in use, would be my first guess as
>> to why you so strongly oppose incrementals.
>>
>> Rob
>>
>
>
> Let me ask YOU a question: which do you think is better: backup up
> everything that has changed since the last master, or just files changed
> since the last backup? Why?
>
What master?
Only back the file up if it has changed.
> BTW, when was the last time you verified your 18 month old backup?
You don't understand how TSM works and overlooked that I mentioned
the files are backed up to a fault-tolerant database with offsite
copies.
If you can list it via a show command, you can restore it. Period.
Rob
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> But (because obviously the attention span of Usenet requires this) I
> will once again say: IF YOU ARE CONSTRAINED BY SPACE OR TIME THEN
> INCREMENTALS OR DIFFERENTIALS ARE OBVIOUSLY NECESSARY.
This is why I referred to the Ivory Tower, because you are ALWAYS
constrained by SPACE AND TIME. Always, no exceptions at all outside of
the Ivory Tower.
Tony Lawrence
> In article <s9ZW9.66836$hl1.7232@sccrnsc04>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>But (because obviously the attention span of Usenet requires this) I
>>will once again say: IF YOU ARE CONSTRAINED BY SPACE OR TIME THEN
>>INCREMENTALS OR DIFFERENTIALS ARE OBVIOUSLY NECESSARY.
>
>
> This is why I referred to the Ivory Tower, because you are ALWAYS
> constrained by SPACE AND TIME. Always, no exceptions at all outside of
> the Ivory Tower.
>
I have no idea what you are referring to. Many businesses have a 8 to
16 hour window in which to do backup. If their entire system can be
backed up to removable media in that time frame, they are not
constrained by space or time.
I do have customers with 24 x 7 operations, and they are harder to deal
with. If they can afford the cost, I like to handle that with logical
volume software that can do a filesystem snapshot that can be backed up
at leisure. If they can't do that, then we do what we can for the money
they can affford to spend. Sometimes incrementals are the only choice,
but I work hard to find a better way.
But the great majority of my client base are 9 to 5, leaving plenty of
time for backup. Tapes get bigger and bigger all the time, too.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I have no idea what you are referring to. Many businesses have a 8 to
> 16 hour window in which to do backup. If their entire system can be
> backed up to removable media in that time frame, they are not
> constrained by space or time.
In my experience, this is vanishingly rare. Most people like to get
home rather than watch tapes spin, even if they can take down time in
order to get a nice cold backup.
Shane Smith
specialties, that may skew your sample set.
Shane
Tony Lawrence
> In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>I have no idea what you are referring to. Many businesses have a 8 to
>>16 hour window in which to do backup. If their entire system can be
>>backed up to removable media in that time frame, they are not
>>constrained by space or time.
>
>
> In my experience, this is vanishingly rare. Most people like to get
> home rather than watch tapes spin, even if they can take down time in
> order to get a nice cold backup.
>
You have to be kidding me. Your "experience" can't amount to much.
Who sits and watches tapes spin? Cron jobs do the whole thing - the
backup, the verify, the printing or emailing of results. Even paging
people if need be. What are computers for?
Even in places where there is no real down time, there is ordinarily no
human involvement. For example, we might set backups to happpen twice a
day. At 15 minutes before backup, people get an email or a screen flash
reminding them that they will have to close out for a few minutes at
whatever time has been set. When that time comes, the system checks
that everybody is out, and then nags if they aren't, and finally ejects
them forcibly if necessary. Then the snapshot is made (for those who
don't know what that is, see http://aplawrence.com/Linux/lvm.html ) and
people can get back on. The backup then begins. The whole thing takes
but a few minutes, and most of that is the nagging. More important is
that every bit of it is automatic, untouched by human hands.
Tony Lawrence
> In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
> Tony Lawrence <to...@pcunix.com> wrote:
>
>
>>I have no idea what you are referring to. Many businesses have a 8 to
>>16 hour window in which to do backup. If their entire system can be
>>backed up to removable media in that time frame, they are not
>>constrained by space or time.
>
>
> In my experience, this is vanishingly rare. Most people like to get
> home rather than watch tapes spin, even if they can take down time in
> order to get a nice cold backup.
>
I must be an idiot. I've been arguing with you about the relative value
of backup methods and you obviously are so wet behind the ears that you
think people do backups manually.
I've been involved with computers since 1967, and have been self
employed since 1983. I have done work at thousands of sites, from one
person offices to Fidelity. I'm certified in Solaris, SCO, and Windows.
Enterprise level, not product, by the way :
http://aplawrence.com/certification.html
(I don't normally put this kind of stuff up on the bar for measurement
but this foolishness has just gone on too long).
Now tell me what in hell makes you think that anything you have to say
should be given any weight at all? I really don't mean to be offensive
here, but this "Ivory Tower" crap coming from someone so plainly
inexperienced has really ticked me off.
I make my living doing this. Do you? I work with dozens of different
backup systems from tape to dvd to jukeboxes. Do you? I work with
multiple backup software, from tar and cpio to Arcserve and beyond. Do
you? I have performed HUNDREDS of disk recoveries, rebuilds, reinstalls
and upgrades on multiple OS platforms from TRSDOS to Microsoft Xenix,
DOS, Windows NT, SCO, Solaris, and Linux. Have you?
I'm sorry to get so rabid. I'm willing to at least respect the opinions
of someone like the VMS admin who posted earlier, though I disagree with
his reliance on a database based incremental backup. I'm no longer
willing to give any respect to your opinions on this.
Sorry for the rant.
"In your experience" indeed.
Tony Lawrence
I wouldn't count on that without knowing the details of the software.
For example, many a person has beeen led down the path of doom by
assuming that a "tar tvf" listing was sufficient indication that the
files listed could in fact be restored.
I think a fault tolerant database (though too pricey for most of us)
combined with offsite copies is certainly a reasonable backup method.
That doesn't necessarily change anything though: it's still better to
backup everything every single time unless it is impossible to do so.
However, that does depend on how the offsite copies are generated. If
they are in fact full copies of everything, then this method would
satisfy me: the backup is indirect, but it is still complete. If the
copies are incrementals,, then you have the same silliness and
difficulties of the more common tape incrementals.
I would guess that the copies probably are complete?
Rob Young
>
> I'm sorry to get so rabid. I'm willing to at least respect the opinions
> of someone like the VMS admin who posted earlier, though I disagree with
> his reliance on a database based incremental backup. I'm no longer
> willing to give any respect to your opinions on this.
>
> Sorry for the rant.
>
That's okay. I agree you know your stuff. But don't
be afraid to pick up on something else totally foreign and
arguably much better.
My whole point about the incrementals is it isn't as bad as you come
off about it. And it isn't as if TSM isn't used in very
large situations.
Here is an example, easy to find dozens of similar examples:
Client is looking for a Tivoli Storage Manager Administrator to
1. Provide day to day adminstration of the TSM environment at MTO (4 TSM
Servers, STK silo, 16 STK drives, 5000 tapes, 225 clients (HP UX, VMS, NT) with
20 TB of storage).
2. Assist in indentification, evaluation, and implementation of improvements to
overall TSM environment
3. Transition knowledge to client team members so we can be self sufficient.
Day to Day Administration Activities. Day to day support activities the TSM
Administrator will perform include:
- Monitoring the environment to detect any hardware or software issues and
taking appropriate corrective action
- Monitoring system jobs to ensure backups complete in timely manner, TSM
database backups run successfully, and off-site tapes are generated
---
etc. The point is that it is an Enterprise Backup solution. There
are similar Enterprise Backups out there.
Again, incrementals are not a bad thing.
I'm not the only one relying on it. It is used in hundreds of
places. I suppose if there were any large "issues" with it, we
would hear about it. There aren't. It really is a comfort
level and I understand why many cling to their backup schemas.
Rob
Rob Young
Please don't insult us. One of the worst features of Eunuchs
is a pitiful backup design (or lack thereof).
Yes, it is possible to write something and know it got there
and to know you can read it back.
> I think a fault tolerant database (though too pricey for most of us)
> combined with offsite copies is certainly a reasonable backup method.
>
> That doesn't necessarily change anything though: it's still better to
> backup everything every single time unless it is impossible to do so.
But that is mostly unrealistic. The problem is with 200, 400
, 500 clients (servers) and the storage associated with them, there
aren't enough hours in the day to backup everything all the
time. Secondly, and just as importantly, to shrink the
backup window... if the file isn't changing, why not skip backing
it up? This is a problem. What if your only copy goes bad? Well
let us make sure we have a fault tolerant backup solution and lets
make sure we do a daily DR migration.
This isn't everybody, it is larger shops. You most likely
wouldn't have an Enterprise Backup product if you are small. There
is a transition point where even with 20-80 servers things get
unwieldy.
> However, that does depend on how the offsite copies are generated. If
> they are in fact full copies of everything, then this method would
> satisfy me: the backup is indirect, but it is still complete. If the
> copies are incrementals,, then you have the same silliness and
> difficulties of the more common tape incrementals.
I'm not a TSM administrator. I do know DR migration (or
offsite) takes about 3-4 hours and begins after
all backups are complete. This keeps your DR copy in synch
with what is onsite.
You might have better results in shooting holes in TSM if
you read a bit more about it. Here is the de facto TSM
admin site, seems to be a good resource:
> I would guess that the copies probably are complete?
Yes. You can't move the tapes until the DR migration (again, often
referred to as "offsite") is complete. I'm sure you can find
details at the above link if interested.
Rob
Rob Young
>
>> I would guess that the copies probably are complete?
>
I realize I misinterpreted that question. The copies aren't
complete copies. The database and associated files that
were backed up, lives in two places at the same time. Only the
parts that change go offsite everyday. Obviously, the database
changes and some subset of backed up storage.
Tony Lawrence
> In article <_0cX9.770577$P31.567195@rwcrnsc53>, Tony Lawrence <to...@pcunix.com> writes:
>
>>Rob Young wrote:
>>
>>>In article <o4ZW9.62086$Yq3.12889@sccrnsc02>, Tony Lawrence <to...@pcunix.com> writes:
>>>
>>>
>>>>Rob Young wrote:
>>> If you can list it via a show command, you can restore it. Period.
>>>
>>
>>I wouldn't count on that without knowing the details of the software.
>>For example, many a person has beeen led down the path of doom by
>>assuming that a "tar tvf" listing was sufficient indication that the
>>files listed could in fact be restored.
>>
>
>
> Please don't insult us. One of the worst features of Eunuchs
> is a pitiful backup design (or lack thereof).
I'm sorry, you misunderstand. The reason I included the reference to
tar was to make it plain that I was not saying that your specific
assertion was invalid, but rather that people cannot rely on that as a
general fact. Sorry to upset you :-)
As to Unix backup, I have no idea what "backup design" is inherent to
the OS. There isn't. Nor should there be.
>
> Yes, it is possible to write something and know it got there
> and to know you can read it back.
>
>
>>I think a fault tolerant database (though too pricey for most of us)
>>combined with offsite copies is certainly a reasonable backup method.
>>
>>That doesn't necessarily change anything though: it's still better to
>>backup everything every single time unless it is impossible to do so.
>
>
> But that is mostly unrealistic. The problem is with 200, 400
> , 500 clients (servers) and the storage associated with them, there
> aren't enough hours in the day to backup everything all the
> time. Secondly, and just as importantly, to shrink the
> backup window... if the file isn't changing, why not skip backing
> it up? This is a problem. What if your only copy goes bad? Well
> let us make sure we have a fault tolerant backup solution and lets
> make sure we do a daily DR migration.
Again, I have said this over and over again: SOMETIMES THERE IS NO
OTHER OPTION TO INCREMENTAL BACKUP BECAUSE OF LIMITATIONS OF SPACE AND TIME.
Why is it necessary to repeat this over and over? Is the attention span
here really that short?
>
> This isn't everybody, it is larger shops. You most likely
> wouldn't have an Enterprise Backup product if you are small. There
> is a transition point where even with 20-80 servers things get
> unwieldy.
>
>
>>However, that does depend on how the offsite copies are generated. If
>>they are in fact full copies of everything, then this method would
>>satisfy me: the backup is indirect, but it is still complete. If the
>>copies are incrementals,, then you have the same silliness and
>>difficulties of the more common tape incrementals.
>
>
> I'm not a TSM administrator. I do know DR migration (or
> offsite) takes about 3-4 hours and begins after
> all backups are complete. This keeps your DR copy in synch
> with what is onsite.
>
> You might have better results in shooting holes in TSM if
> you read a bit more about it. Here is the de facto TSM
> admin site, seems to be a good resource:
The fact is, my argument has absolutely nothing to do with TSM. That
product is completely irrelevant to this discussion because it is
nothing but a convenient go-between to the real backup, which is the
archival of the TSM database. You may very well think of that as yout
backup, and it may even function as such under normal conditions, but it
is conceptually no different (though of course practically very
different) than simply copying data to a network drive and backing up THAT.
If you backup the TSM incrementally, then you are cruising for trouble
for all the reasons I outlined at
http://aplawrence.com/Unixart/diffback.html. If you are in fact backing
that up non-incrementally, then you are in fact doing complete backups:
the fact that the TSM feed is incremental is completly unimportant.
Rob Young
> Rob Young wrote:
>>
>> You might have better results in shooting holes in TSM if
>> you read a bit more about it. Here is the de facto TSM
>> admin site, seems to be a good resource:
>
>
> The fact is, my argument has absolutely nothing to do with TSM. That
> product is completely irrelevant to this discussion because it is
> nothing but a convenient go-between to the real backup, which is the
> archival of the TSM database.
No.
> You may very well think of that as yout
> backup, and it may even function as such under normal conditions, but it
> is conceptually no different (though of course practically very
> different) than simply copying data to a network drive and backing up THAT.
>
> If you backup the TSM incrementally, then you are cruising for trouble
> for all the reasons I outlined at
> http://aplawrence.com/Unixart/diffback.html. If you are in fact backing
> that up non-incrementally, then you are in fact doing complete backups:
> the fact that the TSM feed is incremental is completly unimportant.
>
You aren't "backing up the TSM incrementally". TSM has a database
that tracks all changes/files and each tape and disk in the
various pools are dependent on the database to track them. The
database itself is copied and DR migrated every day. And
no, you aren't doing complete backups as the servers it is tracking
are sending files that have been modified/changed, you are
performing incrementals:
http://adsm.nerdc.ufl.edu/local/incrementals-forever.html
Every backup in TSM is viewed as an incremental, even the first. The initial
backup is simply an incremental in which, coincidentally, every file is needed
to be copied. Thereafter, the backup client downloads a list of all the files
and modification times that are recorded on the server. This map is used to
determine what should be copied.
And yes, if you narrow down to the small world you are talking about,
do full backups of everything all the time - because you can, and
because they are easier to understand and support!
Rob
Tony Lawrence
> In article <pAhX9.774093$WL3.765385@rwcrnsc54>, Tony Lawrence <to...@pcunix.com> writes:
>
>>Rob Young wrote:
>
>
>>> You might have better results in shooting holes in TSM if
>>> you read a bit more about it. Here is the de facto TSM
>>> admin site, seems to be a good resource:
>>
>>
>>The fact is, my argument has absolutely nothing to do with TSM. That
>>product is completely irrelevant to this discussion because it is
>>nothing but a convenient go-between to the real backup, which is the
>>archival of the TSM database.
>
>
> No.
Yes :-)
TSM is just a more sophisticated method of copying files to another hard
drive. As you well know, as convenient and wonderful as that is, it
puts all your eggs in one basket and would be completely unacceptable if
it were not backed up to removable media in turn.
>
>
>>You may very well think of that as yout
>>backup, and it may even function as such under normal conditions, but it
>>is conceptually no different (though of course practically very
>>different) than simply copying data to a network drive and backing up THAT.
>>
>>If you backup the TSM incrementally, then you are cruising for trouble
>>for all the reasons I outlined at
>>http://aplawrence.com/Unixart/diffback.html. If you are in fact backing
>>that up non-incrementally, then you are in fact doing complete backups:
>>the fact that the TSM feed is incremental is completly unimportant.
>>
>
>
> You aren't "backing up the TSM incrementally". TSM has a database
> that tracks all changes/files and each tape and disk in the
> various pools are dependent on the database to track them. The
> database itself is copied and DR migrated every day. And
> no, you aren't doing complete backups as the servers it is tracking
> are sending files that have been modified/changed, you are
> performing incrementals:
You are missing what I am saying: how you send data to TSM is
irrelevant. How you backup TSM ITSELF is what is important.
>
> http://adsm.nerdc.ufl.edu/local/incrementals-forever.html
>
> Every backup in TSM is viewed as an incremental, even the first. The initial
> backup is simply an incremental in which, coincidentally, every file is needed
> to be copied. Thereafter, the backup client downloads a list of all the files
> and modification times that are recorded on the server. This map is used to
> determine what should be copied.
>
> And yes, if you narrow down to the small world you are talking about,
> do full backups of everything all the time - because you can, and
> because they are easier to understand and support!
Which is what I have been saying. But there's more reason than that, as
I detailed at http://aplawrence.com/Unixart/diffbacks.html : damage to a
tape in the middle of an incremental set can be a very serious loss
because the next valid tape that has certain data may be much older and
in some cases (related files) may cause the entire set to be abandoned.
Been there, seen that. More than once.
Rob Young
Yes, in your small world. The TSM backend is fault tolerant,
keep that in mind. I'm sure there is a document explaining
how it works somewhere but you really aren't interested.
Rob
Rob Young
Clarifying. A tape may fail while writing to it. That counts
as a miss. You have to run the backup again from the client
side. The tape that failed twin comes back from offsite to synch
back up.
Rob
David J. Dachtera
>
> Howard S Shubs wrote:
> > In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
> > Tony Lawrence <to...@pcunix.com> wrote:
> >
> >
> >>I have no idea what you are referring to. Many businesses have a 8 to
> >>16 hour window in which to do backup. If their entire system can be
> >>backed up to removable media in that time frame, they are not
> >>constrained by space or time.
> >
> >
> > In my experience, this is vanishingly rare. Most people like to get
> > home rather than watch tapes spin, even if they can take down time in
> > order to get a nice cold backup.
> >
>
> I must be an idiot. I've been arguing with you about the relative value
> of backup methods and you obviously are so wet behind the ears that you
> think people do backups manually.
If you know how to automate backups including:
o Shutdown the running system
o Boot up the stand-alone CD
o Kick-off a system-disk backup
o Shutdown the stand-alone system
o Reboot the live, production system, sans app.'s
o Backup the production application disks
o Restart the applications.
...you'll not only become a very wealthy man, you'll fulfill the dreams
of many denizens of this group.
Actually, I could see doing that with a very sophisticated Reflection
script (RCL or RBS, take your choice).
Oh, yeah, there's that zero-downtime thing to consider...
--
David J. Dachtera
dba DJE Systems
http://www.djesys.com/
Unofficial Affordable OpenVMS Home Page:
http://www.djesys.com/vms/soho/
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> I make my living doing this.
I do system management for a living these days, yes.
> I work with dozens of different
> backup systems from tape to dvd to jukeboxes. Do you?
Backups at my current location are mainly done by others in my group,
using TSM and a good sized robotic solution tied to an AIX box.
> and upgrades on multiple OS platforms from TRSDOS to Microsoft Xenix,
> DOS, Windows NT, SCO, Solaris, and Linux. Have you?
I've worked with Mac and VMS, mainly. Done a little with Linux and AIX
as well. I skip the MS stuff as much as possible.
> I'm sorry to get so rabid. I'm willing to at least respect the opinions
> of someone like the VMS admin who posted earlier, though I disagree with
> his reliance on a database based incremental backup. I'm no longer
> willing to give any respect to your opinions on this.
As you wish.
> "In your experience" indeed.
You'd prefer I didn't use the qualifier?
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> Who sits and watches tapes spin? Cron jobs do the whole thing - the
Okay, so I used a weak argument. Geez.
Ron Shepard
has always been some kind of media that is cheaper than the primary
disk/drum/etc storage. Back in the 70's for example, disk space
cost about $100/MB and tape storage cost about $1/MB. In the past
few years, the cost of disk space has decreased tremendously, about
a factor of 100,000 since the 70's. HD storage is now about $1/GB.
However, tape costs have not changed as much, a GB of tape storage
now costs maybe 4 or 5 times less than it did 10 years ago. DVD-R
is still about $2/GB (not counting the drive, just the media), and
CD-R is comparable. DLT is about $1/GB.
We now have a situation where hard disk storage is either cheaper,
or comprable, to the cost of the media that is used to back it up.
It makes you wonder what is the point. Why not simply buy more hard
drives, back up the data, power them down and store them in a vault?
How will this situation change in the future? Disk storage will
almost certainly get cheaper in the future. Is there any new
storage media that has the ability to keep up with disk storage
costs? What happens in 10 years if disk storage is cheaper by a
factor of 1000 than current prices, but tape/DVD costs are only 4 or
5 times cheaper?
$.02 -Ron Shepard
Eberhard Heuser-Hofmann
In article <ron-shepard-B21C...@netnews.attbi.com>, Ron
Shepard <ron-s...@NOSPAM.attbi.com> writes:
|>Throughout the history of computing (since the 50's at least), there
|>has always been some kind of media that is cheaper than the primary
|>disk/drum/etc storage. Back in the 70's for example, disk space
|>cost about $100/MB and tape storage cost about $1/MB. In the past
|>few years, the cost of disk space has decreased tremendously, about
|>a factor of 100,000 since the 70's. HD storage is now about $1/GB.
|>However, tape costs have not changed as much, a GB of tape storage
|>now costs maybe 4 or 5 times less than it did 10 years ago. DVD-R
|>is still about $2/GB (not counting the drive, just the media), and
|>CD-R is comparable. DLT is about $1/GB.
|>
|>We now have a situation where hard disk storage is either cheaper,
|>or comprable, to the cost of the media that is used to back it up.
|>It makes you wonder what is the point. Why not simply buy more hard
|>drives, back up the data, power them down and store them in a vault?
|>
I've read an article in a german computer magazine, that states,
that it is nowadays cheaper to throw off the DLT-Tape library and to buy
a IDE-based raidsystem.
|>How will this situation change in the future? Disk storage will
|>almost certainly get cheaper in the future. Is there any new
|>storage media that has the ability to keep up with disk storage
|>costs? What happens in 10 years if disk storage is cheaper by a
|>factor of 1000 than current prices, but tape/DVD costs are only 4 or
|>5 times cheaper?
There is difference between backup and archiving. For archiving
purpose use CD/DVD's for backup tape or disks. DVD's with
much higher capacity than today will be out this or next year
(keyword "blue laser"). But the drives will be very expensive
in the beginning as usual.
|>
|>$.02 -Ron Shepard
|>
eberhard
Tony Lawrence
As I've tried to explain, TSM is completely irrelevant to this. It's a
wonderful thing, yes, but it would be HOW YOU BACK THAT UP that is
important and relevant.
And my world is hardly small :-) Kind of a funny thing for a VMS guy to
say, you know?
Tony Lawrence
A tape may fail ANYTIME. Tapes fail while being read. Tapes get eaten
by rats (well, yeah, that wasn't one of my higher class clients), lost,
mislabeled, stepped on etc. That's why the more redundancy you have
with tapes, the safer you are. Again, I've been there and seen this
kind of thing first hand. Apparently in your oh so larger than my world
the unexpected never happens.
Tony Lawrence
> Tony Lawrence wrote:
>
>>Howard S Shubs wrote:
>>
>>>In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
>>> Tony Lawrence <to...@pcunix.com> wrote:
>>>
>>>
>>>
>>>>I have no idea what you are referring to. Many businesses have a 8 to
>>>>16 hour window in which to do backup. If their entire system can be
>>>>backed up to removable media in that time frame, they are not
>>>>constrained by space or time.
>>>
>>>
>>>In my experience, this is vanishingly rare. Most people like to get
>>>home rather than watch tapes spin, even if they can take down time in
>>>order to get a nice cold backup.
>>>
>>
>>I must be an idiot. I've been arguing with you about the relative value
>>of backup methods and you obviously are so wet behind the ears that you
>>think people do backups manually.
>
>
> If you know how to automate backups including:
>
> o Shutdown the running system
> o Boot up the stand-alone CD
Not necessary under any conditions. I suspect you haven't much
experience at this.
> o Kick-off a system-disk backup
> o Shutdown the stand-alone system
> o Reboot the live, production system, sans app.'s
> o Backup the production application disks
> o Restart the applications.
>
> ...you'll not only become a very wealthy man, you'll fulfill the dreams
> of many denizens of this group.
Sigh. All of those things are done every day. Your first two steps
aren't necessary, but even that could be easily automated and in fact
I've done things like that for other reasons. Not necessary for backup
though.
>
> Actually, I could see doing that with a very sophisticated Reflection
> script (RCL or RBS, take your choice).
>
> Oh, yeah, there's that zero-downtime thing to consider...
Few of my clients get that, although we can come close with disk
snapshots. If you REALLY need zero down time, that requires cooperation
from the application software: it has to be able to bring all data files
into sync and buffer all transactions while the snapshot is being
prepared. As that literally takes seconds, that should be no hardship,
but I don't have anything with that feature.
There's nothing technically difficult about it though and everything
else is simple and BEING DONE EVERY DAY. Nothing particularly
sophisticated about it.
Tony Lawrence
If the drives are cheap enough that you can maintain the depth of backup
that you need, why not? Indeed, many of us could afford to do that now:
while I don't particularly want to tie up the cost of a dozen or so disk
drives, that really isn't all that much money nowadays. As the disks
are much more rugged than they used to be, it wouldn't be scary to
transport them home offsite casually.
But if we free up the radio spectrum and have high speed "pipes"
everywhere, maybe we don't need to physically move those drives around
at all?
Tony Lawrence
> In article <ron-shepard-B21C...@netnews.attbi.com>, Ron
> Shepard <ron-s...@NOSPAM.attbi.com> writes:
> |>Throughout the history of computing (since the 50's at least), there
> |>has always been some kind of media that is cheaper than the primary
> |>disk/drum/etc storage. Back in the 70's for example, disk space
> |>cost about $100/MB and tape storage cost about $1/MB. In the past
> |>few years, the cost of disk space has decreased tremendously, about
> |>a factor of 100,000 since the 70's. HD storage is now about $1/GB.
> |>However, tape costs have not changed as much, a GB of tape storage
> |>now costs maybe 4 or 5 times less than it did 10 years ago. DVD-R
> |>is still about $2/GB (not counting the drive, just the media), and
> |>CD-R is comparable. DLT is about $1/GB.
> |>
> |>We now have a situation where hard disk storage is either cheaper,
> |>or comprable, to the cost of the media that is used to back it up.
> |>It makes you wonder what is the point. Why not simply buy more hard
> |>drives, back up the data, power them down and store them in a vault?
> |>
>
> I've read an article in a german computer magazine, that states,
> that it is nowadays cheaper to throw off the DLT-Tape library and to buy
> a IDE-based raidsystem.
>
But it's not suitable for backup. Not by itself. That's just a cheaper
and less functional TSM system: you still need to back that up to
removable media. As you say:
>
> There is difference between backup and archiving. For archiving
> purpose use CD/DVD's for backup tape or disks. DVD's with
> much higher capacity than today will be out this or next year
> (keyword "blue laser"). But the drives will be very expensive
> in the beginning as usual.
Right. I like DVDRAM for small systems and if they can boost the
capacity up it has advantages over tapes in some ways (though it
probably will never match tape write speeds).
Paul Repacholi
> I've read an article in a german computer magazine, that states,
> that it is nowadays cheaper to throw off the DLT-Tape library and to
> buy a IDE-based raidsystem.
RIGHT...
Now burn down the building. Tell me when you get the data off your
IDE raid.
BTW, you do know why it is called IDE, don't you :)
--
Paul Repacholi 1 Crescent Rd.,
+61 (08) 9257-1001 Kalamunda.
West Australia 6076
comp.os.vms,- The Older, Grumpier Slashdot
Raw, Cooked or Well-done, it's all half baked.
EPIC, The Architecture of the future, always has been, always will be.
Eberhard Heuser-Hofmann
In article <87vg0hb...@prep.synonet.com>, Paul Repacholi
<pr...@prep.synonet.com> writes:
|>vax...@chclu.chemie.uni-konstanz.de (Eberhard Heuser-Hofmann) writes:
|>
|>> I've read an article in a german computer magazine, that states,
|>> that it is nowadays cheaper to throw off the DLT-Tape library and to
|>> buy a IDE-based raidsystem.
|>
|>RIGHT...
|>
|>Now burn down the building. Tell me when you get the data off your
|>IDE raid.
And tell me when you get the data off your DLT-Library if
there is fire in that room.
|>
|>BTW, you do know why it is called IDE, don't you :)
Use raid10 (Raid5+mirroring). Even if a IDE-disk fails,
you will have no data loss.
eberhard
Tony Lawrence
> In article <87vg0hb...@prep.synonet.com>, Paul Repacholi
> <pr...@prep.synonet.com> writes:
> |>vax...@chclu.chemie.uni-konstanz.de (Eberhard Heuser-Hofmann) writes:
> |>
> |>> I've read an article in a german computer magazine, that states,
> |>> that it is nowadays cheaper to throw off the DLT-Tape library and to
> |>> buy a IDE-based raidsystem.
> |>
> |>RIGHT...
> |>
> |>Now burn down the building. Tell me when you get the data off your
> |>IDE raid.
>
> And tell me when you get the data off your DLT-Library if
> there is fire in that room.
You get it from the tapes you rotated off site, of course.
> |>
> |>BTW, you do know why it is called IDE, don't you :)
>
> Use raid10 (Raid5+mirroring). Even if a IDE-disk fails,
> you will have no data loss.
Well, no data loss unless the two drives of the raid5 fail..
Backing up to a disk drive is quick, convenient, and insufficient. You
ALWAYS need removable media for archival storage. Before some damn fool
says "just buy a bigger drive and store multiple sets there", THAT puts
too much stuff at risk in one place.
Shane Smith
-----Original Message-----
From: Paul Repacholi [mailto:pr...@prep.synonet.com]
Sent: Wednesday, January 22, 2003 6:28 AM
To: Info...@Mvb.Saic.Com
Subject: Re: How to Backup OSX
vax...@chclu.chemie.uni-konstanz.de (Eberhard Heuser-Hofmann) writes:
> I've read an article in a german computer magazine, that states,
> that it is nowadays cheaper to throw off the DLT-Tape library and to
> buy a IDE-based raidsystem.
RIGHT...
Now burn down the building. Tell me when you get the data off your
IDE raid.
BTW, you do know why it is called IDE, don't you :)
--
Bob Koehler
> We now have a situation where hard disk storage is either cheaper,
> or comprable, to the cost of the media that is used to back it up.
> It makes you wonder what is the point. Why not simply buy more hard
> drives, back up the data, power them down and store them in a vault?
Back when it was still the VAXLUG we had a presentation near here
from a thrid party backup vendor. He was asked why do tape backups
instead of disk to disk and proceeded to knock a 9-track tape from
the podium onto the floor.
"I'll guarantee I can get most of the data, probably all of it, off
that tape. Just try that with a backup on disk."
Bob Koehler
>
>> I've read an article in a german computer magazine, that states,
>> that it is nowadays cheaper to throw off the DLT-Tape library and to
>> buy a IDE-based raidsystem.
>
Hm. I've got a 4mm drive and about ½ TB worth of tape for free.
You can get disk cheaper than that?
Keith A. Lewis
> Hm. I've got a 4mm drive and about ½ TB worth of tape for free.
> You can get disk cheaper than that?
I cut my HW maintenance (contract) costs in half for several Alphas by
removing the 4mm drive. They break a lot.
--Keith Lewis klewis$mitre.org
The above may not (yet) represent the opinions of my employer.
JF Mezei
If you have volume shadowing across buildings, the issue of off-site backup is
not as bad. Although you would have to keep enough drives to keep enough
backups (eg, if you need 2 months at once a day, that would be 60 drives and
that makes managing the drives "interesting". Drives are generally not
designed to be removable media that you switch every day.
The advantage of tapes is that it makes it easy to store in a vault. Although
I guess one could also do the same with drives, but again, everytime you
handle a disk drive, you have greater risks of damaging it and/or the data.
David J. Dachtera
>
> David J. Dachtera wrote:
> > Tony Lawrence wrote:
> >
> >>Howard S Shubs wrote:
> >>
> >>>In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
> >>> Tony Lawrence <to...@pcunix.com> wrote:
> >>>
> >>>
> >>>
> >>>>I have no idea what you are referring to. Many businesses have a 8 to
> >>>>16 hour window in which to do backup. If their entire system can be
> >>>>backed up to removable media in that time frame, they are not
> >>>>constrained by space or time.
> >>>
> >>>
> >>>In my experience, this is vanishingly rare. Most people like to get
> >>>home rather than watch tapes spin, even if they can take down time in
> >>>order to get a nice cold backup.
> >>>
> >>
> >>I must be an idiot. I've been arguing with you about the relative value
> >>of backup methods and you obviously are so wet behind the ears that you
> >>think people do backups manually.
> >
> >
> > If you know how to automate backups including:
> >
> > o Shutdown the running system
> > o Boot up the stand-alone CD
>
> Not necessary under any conditions.
I'll refer you to OpenVMS engineering for a correction.
> I suspect you haven't much
> experience at this.
19 years with VMS, 24 years in the business. See my website. Former
OpenVMS ASE.
Try again.
> > o Kick-off a system-disk backup
> > o Shutdown the stand-alone system
> > o Reboot the live, production system, sans app.'s
> > o Backup the production application disks
> > o Restart the applications.
> >
> > ...you'll not only become a very wealthy man, you'll fulfill the dreams
> > of many denizens of this group.
>
> Sigh. All of those things are done every day. Your first two steps
> aren't necessary,
Well, actually they are. Most of us "cheat" at it by doing a /IMAGE
BACKUP of our live system disk while the cluster is up and running since
we simply cannot afford the downtime it requires, nor do we have savvy
personnel on hand at 00:00 to perform the task.
Again, contact OpenVMS Engineering or the CSC for the supported system
disk backup procedure as documented.
> but even that could be easily automated and in fact
> I've done things like that for other reasons. Not necessary for backup
> though.
Well, actually yes it is. It's a gamble (calculated risk, really) to do
it the way we all do it, but the instances of a failure at DR test time
have been so low as to not prohibit using the "cheat".
> >
> > Actually, I could see doing that with a very sophisticated Reflection
> > script (RCL or RBS, take your choice).
> >
> > Oh, yeah, there's that zero-downtime thing to consider...
>
> Few of my clients get that, although we can come close with disk
> snapshots. If you REALLY need zero down time, that requires cooperation
> from the application software: it has to be able to bring all data files
> into sync and buffer all transactions while the snapshot is being
> prepared. As that literally takes seconds, that should be no hardship,
> but I don't have anything with that feature.
No one has it. It's eutopia.
> There's nothing technically difficult about it though and everything
> else is simple and BEING DONE EVERY DAY. Nothing particularly
> sophisticated about it.
That last comment suggest YOU may lack some experience with large-scale
commercial systems.
I spent two years writing BACKUP automation for large Chicago-area
healthcare concern. It runs with tape library automation (*NOT* SLS) and
splits and rejoins shadow- and mirror-sets as needed, and rotates
through sets of production environments, three a night to cover seven
production environments.
There's frequently a very large difference between what CAN be done and
what SHOULD be done. Say, "compromise".
Carl Perkins
}Backing up to a disk drive is quick, convenient, and insufficient. You
}ALWAYS need removable media for archival storage. Before some damn fool
Disk drives are removeable - it's not like they are welded in or anything.
You can take them offsite almost as easily as you can take tapes off site.
They just take a bit more storage space.
By the way, what dufus crossposted this thread to comp.sys.mac.apps?
--- Carl
Tony Lawrence
> Tony Lawrence wrote:
>
>>David J. Dachtera wrote:
>>
>>>Tony Lawrence wrote:
>>>
>>>
>>>>Howard S Shubs wrote:
>>>>
>>>>
>>>>>In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
>>>>>Tony Lawrence <to...@pcunix.com> wrote:
I've got to trim a bit of this because it's really out of hand and
getting all twisty.
>>>If you know how to automate backups including:
>>>
>>>o Shutdown the running system
>>>o Boot up the stand-alone CD
>>
>>Not necessary under any conditions.
>
>
> I'll refer you to OpenVMS engineering for a correction.
I'm posting from a Unix group. If you have to shut down a VMS system to
back it up, I am nothing short of stunned.
>
>
>>I suspect you haven't much
>>experience at this.
>
>
> 19 years with VMS, 24 years in the business. See my website. Former
> OpenVMS ASE.
>
> Try again.
Try again? You read comments directed at a specific person (Howard
Shaubs) and think that they apply to you? OK, the thread has gotten a
little confusing, but I didn't think it was THAT bad.
>>>
>>>Oh, yeah, there's that zero-downtime thing to consider...
>>
>>Few of my clients get that, although we can come close with disk
>>snapshots. If you REALLY need zero down time, that requires cooperation
>>from the application software: it has to be able to bring all data files
>>into sync and buffer all transactions while the snapshot is being
>>prepared. As that literally takes seconds, that should be no hardship,
>>but I don't have anything with that feature.
>
>
> No one has it. It's eutopia.
Nonsense. It may very well be that no commercial software exists with
such features, but I have written small things with similar ideas for
specific purposes. There's nothing difficult about it. It would be
hellishly difficult to modify an existing package of any complexity, but
if you started from scratch with this in mind it's no brain strain.
>
>
>>There's nothing technically difficult about it though and everything
>>else is simple and BEING DONE EVERY DAY. Nothing particularly
>>sophisticated about it.
>
>
> That last comment suggest YOU may lack some experience with large-scale
> commercial systems.
>
> I spent two years writing BACKUP automation for large Chicago-area
> healthcare concern. It runs with tape library automation (*NOT* SLS) and
> splits and rejoins shadow- and mirror-sets as needed, and rotates
> through sets of production environments, three a night to cover seven
> production environments.
>
> There's frequently a very large difference between what CAN be done and
> what SHOULD be done. Say, "compromise".
>
I believe I already covered the only compromise necessary: a very few
minutes with the database down, a quick snapshot, database back up, and
the backup is taken from the virtualized snapshot. Most of the downtime
of that is just people: getting them to finish up whatever they are
doing and get out. The snapshot itself is seconds.
Doc.Cypher
>David J. Dachtera wrote:
>> Tony Lawrence wrote:
>>
>>>>If you know how to automate backups including:
>>>>
>>>>o Shutdown the running system
>>>>o Boot up the stand-alone CD
>>>
>>>Not necessary under any conditions.
>> I'll refer you to OpenVMS engineering for a correction.
>I'm posting from a Unix group. If you have to shut down a VMS system to
>back it up, I am nothing short of stunned.
To create a *perfect* copy of a VMS system disk, yes. You need to boot from
another device. It's the sort of thing you would be strongly recommended to
do following an operating system upgrade.
However, as David stated in the portion you snipped, it is very, very, rare
for a VMS system disk backup made whilst the machine is running to be
unusable.
Doc.
--
Time and money, the psychotropics of the business world...
~ VAXman https://vmsbox.cjb.net
Tony Lawrence
> On Thu, 23 Jan 2003, Tony Lawrence <to...@pcunix.com> wrote:
>
>>David J. Dachtera wrote:
>>
>>>Tony Lawrence wrote:
>>>
>>>
>>>>David J. Dachtera wrote:
>>>>
>>>>>If you know how to automate backups including:
>>>>>
>>>>>o Shutdown the running system
>>>>>o Boot up the stand-alone CD
>>>>
>>>>Not necessary under any conditions.
>
>
>>>I'll refer you to OpenVMS engineering for a correction.
>
>
>>I'm posting from a Unix group. If you have to shut down a VMS system to
>>back it up, I am nothing short of stunned.
>
>
> To create a *perfect* copy of a VMS system disk, yes. You need to boot from
> another device. It's the sort of thing you would be strongly recommended to
> do following an operating system upgrade.
>
> However, as David stated in the portion you snipped, it is very, very, rare
> for a VMS system disk backup made whilst the machine is running to be
> unusable.
Well, note the title of this thread. Why it ever got crossposted to a
VMS group is buried in here somewhere, but I don't really care: 99% of
what we have been talking about has only been clouded and made more
confusing by the fact that we are talking about two very different
operating systems and often about different conditions of use.
Mike Rosenberg
> |>Now burn down the building. Tell me when you get the data off your
> |>IDE raid.
>
> And tell me when you get the data off your DLT-Library if
> there is fire in that room.
If you keep the backups in the same room as the computer, you're out of
luck, of course, but if you're serious about backing up, you always have
at least one backup set stored offsite.
--
Mike Rosenberg
<http://www.macconsult.com>
"If you ever get annoyed, look at me, I'm self employed..."
Paul Repacholi
> In article <87vg0hb...@prep.synonet.com>, Paul Repacholi
> <pr...@prep.synonet.com> writes:
> |>vax...@chclu.chemie.uni-konstanz.de (Eberhard Heuser-Hofmann) writes:
> |>
> |>> I've read an article in a german computer magazine, that states,
> |>> that it is nowadays cheaper to throw off the DLT-Tape library and to
> |>> buy a IDE-based raidsystem.
> |>
> |>RIGHT...
> |>
> |>Now burn down the building. Tell me when you get the data off your
> |>IDE raid.
>
> And tell me when you get the data off your DLT-Library if
> there is fire in that room.
1) Fire safe.
2) Off-site.
Howard S Shubs
Ron Shepard <ron-s...@NOSPAM.attbi.com> wrote:
> How will this situation change in the future? Disk storage will
> almost certainly get cheaper in the future. Is there any new
> storage media that has the ability to keep up with disk storage
> costs? What happens in 10 years if disk storage is cheaper by a
> factor of 1000 than current prices, but tape/DVD costs are only 4 or
> 5 times cheaper?
I'm expecting a major technology change to solid media, which will have
enormously higher capacity and access speed, as well as tremendously
higher reliablility. The only way to back one of these "cubes" up will
be to copy it to another one. Disks and tapes will be past-tense. I
expect this w/in 20 years.
Tim Llewellyn
Mike Rosenberg wrote:
>
> Eberhard Heuser-Hofmann <vax...@chclu.chemie.uni-konstanz.de> wrote:
>
> > |>Now burn down the building. Tell me when you get the data off your
> > |>IDE raid.
> >
> > And tell me when you get the data off your DLT-Library if
> > there is fire in that room.
>
> If you keep the backups in the same room as the computer, you're out of
> luck, of course, but if you're serious about backing up, you always have
> at least one backup set stored offsite.
>
A firesafe is also a good thing to have to store your onsite tapes.
regards,
* PLEASE NOTE tim.ll...@cableinet.co.uk address is NO LONGER VALID *
Tony Lawrence
Until the "cubes" fit in a shirt pocket, it won't happen.
Bob Koehler
> koe...@eisner.nospam.encompasserve.org (Bob Koehler) writes in article <iI7f$hTI...@eisner.encompasserve.org> dated 22 Jan 2003 14:37:22 -0600:
>> Hm. I've got a 4mm drive and about ½ TB worth of tape for free.
>> You can get disk cheaper than that?
>
> I cut my HW maintenance (contract) costs in half for several Alphas by
> removing the 4mm drive. They break a lot.
I've only had 2 break over the last 10 or so years. Must have
at least a dozen.
Bob Koehler
>
> I'm posting from a Unix group. If you have to shut down a VMS system to
> back it up, I am nothing short of stunned.
It's just like putting UNIX in single-user mode to get a clean
backup of the root partition. For everythiung else, you don't have
to, and many times you can get away with it for the OS, too.
David J. Dachtera
>
> David J. Dachtera wrote:
> > Tony Lawrence wrote:
> >
> >>David J. Dachtera wrote:
> >>
> >>>Tony Lawrence wrote:
> >>>
> >>>
> >>>>Howard S Shubs wrote:
> >>>>
> >>>>
> >>>>>In article <Lj%W9.71429$1q3.10159@sccrnsc01>,
> >>>>>Tony Lawrence <to...@pcunix.com> wrote:
>
> I've got to trim a bit of this because it's really out of hand and
> getting all twisty.
>
> >>>If you know how to automate backups including:
> >>>
> >>>o Shutdown the running system
> >>>o Boot up the stand-alone CD
> >>
> >>Not necessary under any conditions.
> >
> >
> > I'll refer you to OpenVMS engineering for a correction.
>
> I'm posting from a Unix group. If you have to shut down a VMS system to
> back it up, I am nothing short of stunned.
>
> >
> >
> >>I suspect you haven't much
> >>experience at this.
> >
> >
> > 19 years with VMS, 24 years in the business. See my website. Former
> > OpenVMS ASE.
> >
> > Try again.
>
> Try again? You read comments directed at a specific person (Howard
> Shaubs) and think that they apply to you?
AH! That's where you went wrong. Probably would have been better to
reply by private e-mail if it was not intended for the main group or any
cross-postings.
Howard S Shubs
Tony Lawrence <to...@pcunix.com> wrote:
> Until the "cubes" fit in a shirt pocket, it won't happen.
Current disks don't fit there, why do these have to? Just curious.
Ron Shepard
Howard S Shubs <how...@shubs.net> wrote:
> > Until the "cubes" fit in a shirt pocket, it won't happen.
>
> Current disks don't fit there, why do these have to? Just curious.
This has little to do with the backup discussion, but there are 2.5
inch hard drives that do fit in your shirt pocket. Also, I saw an
article recently about a small 4GB drive that was about the size of
a postage stamp and about 2mm thick. Imagine putting a few dozen
(or a few hundred) of these on a card with a RAID controller for
fault tolerance.
$.02 -Ron Shepard
John Wallace
"Ron Shepard" <ron-s...@NOSPAM.attbi.com> wrote in message
news:ron-shepard-C079...@netnews.attbi.com...
FYI OT ETC: Beware. Some of these "microdrives" make it *very* clear that
they are not intended for active long term data storage and retrieval,
rather for data transportation between systems. My Tosh 2GB PCMCIA certainly
does; I had assumed that other similar "microdrives" from IBM (they do a
CompactFlash one meeting your description) and others had similar
restrictions. Even if it were solidstate memory CompactFlash, they still
have a limited number of writes before they wear out. This isn't the kind of
RAID I'd like to see. Maybe OK for backing up laptops (and saving somewhere
else) though.
Shane Smith
themselves as disks to the OS. I think what Tony meant was that for the
cubes to replace /all/ storage media, fitting them in the pocket was a
pre-requisite. One of many.
Shane
-----Original Message-----
From: Howard S Shubs [mailto:how...@shubs.net]
Sent: Thursday, January 23, 2003 7:39 PM
To: Info...@Mvb.Saic.Com
Subject: Re: How to Backup OSX
In article <d3ZX9.10400$4y2.1291@sccrnsc04>,
Tony Lawrence <to...@pcunix.com> wrote:
> Until the "cubes" fit in a shirt pocket, it won't happen.
Current disks don't fit there, why do these have to? Just curious.
--
Bill Todd
"John Wallace" <johnwa...@yahoo.dotco.dotuk> wrote in message
news:b0re18$h3$1...@venus.btinternet.com...
>
> "Ron Shepard" <ron-s...@NOSPAM.attbi.com> wrote in message
> news:ron-shepard-C079...@netnews.attbi.com...
> > In article <howard-37C75E....@enews.newsguy.com>,
> > Howard S Shubs <how...@shubs.net> wrote:
> >
> > > > Until the "cubes" fit in a shirt pocket, it won't happen.
> > >
> > > Current disks don't fit there, why do these have to? Just curious.
> >
> > This has little to do with the backup discussion, but there are 2.5
> > inch hard drives that do fit in your shirt pocket. Also, I saw an
> > article recently about a small 4GB drive that was about the size of
> > a postage stamp and about 2mm thick. Imagine putting a few dozen
> > (or a few hundred) of these on a card with a RAID controller for
> > fault tolerance.
> >
> > $.02 -Ron Shepard
>
> FYI OT ETC: Beware. Some of these "microdrives" make it *very* clear that
> they are not intended for active long term data storage and retrieval,
> rather for data transportation between systems. My Tosh 2GB PCMCIA
certainly
> does; I had assumed that other similar "microdrives" from IBM (they do a
> CompactFlash one meeting your description) and others had similar
> restrictions.
4 GB in CompactFlash? I hadn't seen anything close to that in a 1" form
factor, though I'm not well-acquainted with the area.
There is, however, a 1" form factor disk drive (3600 rpm, 12 ms. average
seek, etc.) called a Microdrive. Current capacities range up to 1 GB, but I
think I read recently that a larger one is on the way. And my guess is that
its reliability/lifetime characteristics are considerably closer to those
for larger disks than to flash.
- bill
Paul Repacholi
> Mike Rosenberg wrote:
> > Eberhard Heuser-Hofmann <vax...@chclu.chemie.uni-konstanz.de> wrote:
> > > |>Now burn down the building. Tell me when you get the data off
> > > |>your IDE raid.
> > > And tell me when you get the data off your DLT-Library if there
> > > is fire in that room.
> > If you keep the backups in the same room as the computer, you're
> > out of luck, of course, but if you're serious about backing up,
> > you always have at least one backup set stored offsite.
> A firesafe is also a good thing to have to store your onsite tapes.
However, PLEASE don't think that if you put it in a fire safe, they
are safe...
This is easy to fix though. Grab a tape, and head down to your local
supermarket. Look in the kitchenware section for some rectangular
storage canisters the right size for you tapes. Yes, this may mean
getting different one. Also get different coloured lids for each
day/system whatever. You want each one to hold a full set of tapes for
each backup. So the oper takes out a canister, opens it and uses the
tapes to do a backup. The tapes go back in, and it is sealed and
replaced in the safe.
OK, why...
When you do have a fire, the safe interior WILL heat up. Not a lot,
but some. Then it will cool, and start sucking in a wonderfull mix of
water vapour, HCN, HCl, and more crap than we want to think
about. This will promptly condense out on anthing cool. Like the
contents of your safe. :( DATs do not like this. Does not do DLTs any
good either!
So go and get a nice airtight container, and not have an `interesting
time' at you recovery site.
This is what you do NOT want to see
http://www.mso.anu.edu.au/colless
Howard S Shubs
Paul Repacholi <pr...@prep.synonet.com> wrote:
> So go and get a nice airtight container, and not have an `interesting
> time' at you recovery site.
Interesting indeed! I'll have to remember that one.
Paul Sture
Just for everyone's amusement, I stumbled across the following
page about tapes::
http://storageconference.org/2001/EmergTechPanel/Schwarz.pdf
"1972
IBM begins development on its last
tape drive (3480) ever because
of the declining cost of disk drives"
There are graphs of prices trends as well.
--
Paul Sture
Tony Lawrence
I think it's *possible* that dvdram or other optical devices might
someday replace tape (and who knows, might even replace hard drives if
you could ever make them fast enough) but the big advantage to tape is
that you can always increase capacity by making the media a little
thinner and a lot longer (disregarding the obvious phyical limits, of
course). That's not equivalent to packing bits tighter to increase
storage on a disk drive - as long as you can fit longer tape into the
drive, you can store more. In theory, you could take a 4mm dat tape and
design a tape that ran for hundreds miles and was fed in from outside
(obviously some clever engineering would be necessary). The same drive
could write much larger data.
For some reason that made me think of those car cd adaptors that let you
play a cd through your tape player. I'm thinking of that in reverse:
take a "tape" that actually feeds its data off to something else when
written. Nothing to do with the previous point; just a random though
looking for an application :-)
If I had to bet on it, I'd put my money on tape being around even after
hard drives are solid state or quantum devices or whatever. They just
keep getting bigger and faster.
Paul Repacholi
> This is what you do NOT want to see
> http://www.mso.anu.edu.au/colless
Sorry, wrong link, use
http://www.mso.anu.edu.au/colless/StromloFire/
Howard S Shubs
Paul Repacholi <pr...@prep.synonet.com> wrote:
> Sorry, wrong link, use
> http://www.mso.anu.edu.au/colless/StromloFire/
What HAPPENED??
Howard S Shubs
b...@world.std.com (Brian 'Jarai' Chase) wrote:
> In article <howard-5D5596....@enews.newsguy.com>,
> Howard S Shubs <how...@shubs.net> wrote:
> > In article <87wuksw...@prep.synonet.com>,
> > Paul Repacholi <pr...@prep.synonet.com> wrote:
>
> > > Sorry, wrong link, use
> > > http://www.mso.anu.edu.au/colless/StromloFire/
> >
> > What HAPPENED??
>
> Aliens.
You look up their nostrils long enough, they get annoyed?