Upgrading to Sco Openserver 5.0.5
kevi...@my-dejanews.com
SCO Openserver 5.0.5? So far my attempts to upgrade a 5.0.4 system to
5.0.5 have failed miserably. A fresh install works fine everytime...
--
-krs
-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own
Jean-Pierre Radley
| I was wondering what kind of success people are having upgrading to
| SCO Openserver 5.0.5? So far my attempts to upgrade a 5.0.4 system to
| 5.0.5 have failed miserably. A fresh install works fine everytime...
I did an In-Place Upgrade from 5.0.4 to 5.0.5. To my mind, it was 98%
satisfactory -- no real problems, a bit of cruft left behind, and some
really obscure changes I made in the some SCO-supplied files that were
not preserved.
I would *still* advocate running my savefiles script and doing a fresh
install.
OTOH, "failed miserably" is more than a little vague. What problems did
you actually find?
--
Jean-Pierre Radley <j...@jpr.com> XC/XT Custodian Sysop, CompuServe SCOForum
kevi...@my-dejanews.com
Jean-Pierre Radley <j...@jpr.com> wrote:
> OTOH, "failed miserably" is more than a little vague. What problems did
> you actually find?
Yes, I am sorry that was very vague. I've since had some more success.
Ok, I don't have all the fine details here but I had 3 test systems to
upgrade. 2 of them were 5.0.4, we'll call them A and B. A third was 5.0.2,
lets call it C.
Did the 'upgrade' on A. Seemed to work fine. However during the first boot up
after the install, it had a double kernel panic. Reset and again the same
double kernel panic. This is repeated for every attempt to reboot. Also on
the screen it said failed to write the dump (although I am not sure if it
would have even been useful). Apparently from SCO documentation this could be
caused by either failure in ram or a software failure. Not much help there.
So I moved on to B. System B was the one I would describe as "failed
miserably" because it failed during the install part. Note again I selected
Upgrade and it did report the system was suitable for an upgrade. After the
failure, I tried to reboot and they system wouldn't boot. I don't recall
exactly what happened but I don't believe it even made it to the Boot:
prompt. So I tried an upgrade again and it said there was no OS to upgrade. I
then decide it would be best to do a fresh install at this point. Done.
Back to system A. I decided to reinstall 5.0.4. I did so. Fine. I attempted
an upgrade to 5.0.5 again. And it worked fine. Ok. So far everything seems to
be ok with it.
System C. It had 5.0.2 on it and I did the upgrade. Things went fine and I was
surpized the upgrade went without a hitch. So far so good with this one too.
In summary, this may still sound vague, :-) but thats a rough overview as to
what happened. Since they were test machines, I didn't do any backups and
losing the data on them was not important. But I do have about 8 machines,
some with Openserver and others with 3.2v4.2. These results were inconsistant
at best which scares me when I think about upgrading fully functioning
servers. Looks like I will have to do full backups, make use of your
savefiles script and pray. :-)
Jean-Pierre Radley
| Looks like I will have to do full backups, make use of your
| savefiles script and pray. :-)
Uh, no matter *which* method(s) you use to move to a morre recent
release of *any* operating system, I do not see how you can avoid
making full backups. That's just part of the job, you tell one of Bru,
LoneTar, Ctar or BackupEDGE to make and verify at least two archives
before you make any major changes at all.
Mark A. Davis
>
> kevi...@my-dejanews.com averred (on Fri, Nov 20, 1998 at 03:00:10AM +0000):
> | Looks like I will have to do full backups, make use of your
> | savefiles script and pray. :-)
>
> Uh, no matter *which* method(s) you use to move to a morre recent
> release of *any* operating system, I do not see how you can avoid
> making full backups. That's just part of the job, you tell one of Bru,
> LoneTar, Ctar or BackupEDGE to make and verify at least two archives
> before you make any major changes at all.
I hate to ask the question again, but won't a
cd /
find . -mount -print > !tape_all
find ./stand -print >> !tape_all
cat !tape_all | cpio -oHcrc -C 10240 > /dev/rct0
suffice as a complete system backup?
I know that if there were a total systems failure, I would have to
reinstall the old OS enough to get to the tape drive, and then worry
about disk partitions (we have a RAID system, though). But otherwise,
are there any other obvious dangers?
I have the 5.05 sitting on my desk, but I am not too excited about
installing it on my mission-critical systems. A "fresh install" scares
me much more than an upgrade, since our systems are so incredibly large
and there are always things lost or hosed about a fresh install.
--
/--------------------------------------------------------------------\
| Mark A. Davis, |Lake Taylor| Voice: (757)-461-5001x431 8-4:30ET |
| Director of | Hospital | ma...@yylaketaylor.org ***to reply |
|Information Systems|Norfolk, VA| from USENET remove anti-spam "yy" |
\--------------------------------------------------------------------/
Jean-Pierre Radley
| cd /
| find . -mount -print > !tape_all
| find ./stand -print >> !tape_all
| cat !tape_all | cpio -oHcrc -C 10240 > /dev/rct0
|
| suffice as a complete system backup?
Not in my book, it wouldn't.
I will only accept backups that have been verified bit for bit.
Tony Lawrence
> I hate to ask the question again, but won't a
>
> cd /
> find . -mount -print > !tape_all
> find ./stand -print >> !tape_all
> cat !tape_all | cpio -oHcrc -C 10240 > /dev/rct0
>
> suffice as a complete system backup?
>
> I know that if there were a total systems failure, I would have to
> reinstall the old OS enough to get to the tape drive, and then worry
> about disk partitions (we have a RAID system, though). But otherwise,
> are there any other obvious dangers?
Yes.
Tape is very reliable, but not 100%. When you use products like
Edge or Lonetar, you learn that disturbing percentages of
backups have bit errors. Most of the time, of course, these
are in unimportant or easily recreated files, but sometimes they are
not.
What's "disturbing"? Even once or twice a year is enough to
scare me.
The Supertars are *very* inexpensive, even if you buy them at
full list (and that is hardly necessary). The peace of
mind they offer is an additional benefit to the joyous
ease they provide for upgrades, reinstallations and disaster
recovery.
I know that some support organizations make ownership of
one of these products an absolute requirement. Personally, I'm
not quite that tough with my clients, but I do strongly
and regularly encourage them to buy it, whether from me or anyone
else I do not care just so they *have* it.
This is not something you should even think about not having,
even if you use other methods for "normal" backups. That's
an opinion, of course, but an opinion forged from many
years of hard time in the trenches.
http://www.microlite.com
http://www.cactus.com
Do it TODAY :-)
--
Tony Lawrence (to...@aplawrence.com)
SCO ACE
SCO articles, help, book reviews: http://www.aplawrence.com
Neal Rhodes
They wouldn't believe me when I told them it was out of spec and
the vendor should replace.
With this cable installed, they could make a database backup, and
read that backup with the normal cpio commands.
Only problem was when they restored with it, the database wasn't
the same size.
Many trials, errors, a saturday trip to the computer store, and
a decent cable, and all was well.
A product like Edge would have spotted it on the first verify.
Tony Lawrence wrote:
>
> Tape is very reliable, but not 100%. When you use products like
> Edge or Lonetar, you learn that disturbing percentages of
> backups have bit errors. Most of the time, of course, these
> are in unimportant or easily recreated files, but sometimes they are
> not.
>
> What's "disturbing"? Even once or twice a year is enough to
> scare me.
>
> --
> Tony Lawrence (to...@aplawrence.com)
> SCO ACE
> SCO articles, help, book reviews: http://www.aplawrence.com
--
------------------------------------------------------------------------------
Neal Rhodes MNOP Ltd (770)-
972-5430
President Lilburn (atlanta) GA 30247 Fax:
978-4741
ne...@mnopltd.com
http://www.mnopltd.com/
Mark A. Davis
>
> Mark A. Davis wrote:
>
> > I hate to ask the question again, but won't a
> >
> > cd /
> > find . -mount -print > !tape_all
> > find ./stand -print >> !tape_all
> > cat !tape_all | cpio -oHcrc -C 10240 > /dev/rct0
> >
> > suffice as a complete system backup?
> >
> > I know that if there were a total systems failure, I would have to
> > reinstall the old OS enough to get to the tape drive, and then worry
> > about disk partitions (we have a RAID system, though). But otherwise,
> > are there any other obvious dangers?
>
> Yes.
>
> Edge or Lonetar, you learn that disturbing percentages of
> backups have bit errors. Most of the time, of course, these
> are in unimportant or easily recreated files, but sometimes they are
> not.
Hmm. I never thought of that as a potential problem on high-end drives,
such as the HP DAT 24/6. With tape error correction and read-back and
quality tapes that are rotated out of service regularly, regular
cleanings, regular read-back for verification, etc. The only thing I
can imagine that would cause an error would be some kind of highly
unusual SCSI hardware and/or driver problem.
> What's "disturbing"? Even once or twice a year is enough to
> scare me.
>
> The Supertars are *very* inexpensive, even if you buy them at
> full list (and that is hardly necessary). The peace of
> mind they offer is an additional benefit to the joyous
> ease they provide for upgrades, reinstallations and disaster
> recovery.
For us, the expense is not the issue. I have no problem spending
money. But I like using standard tools that create readable tapes on
any machine using standard syntax and command-line interface for cron.
I don't remember all the issues, but it didn't seem like the backup
products fit my bill, at the time. I don't need compression (hardware
is much better), I don't need GUI interfaces, I am already running at
full tape speed, I don't need to backup raw partitions, I have no
problems with tapes being overwriten (robot drive), I don't need remote
site interfaces, I don't want to have to upgrade yet something else all
the time, I believe my emergency root/boot disks will allow full
recovery (although I do need to test them again).
> This is not something you should even think about not having,
> even if you use other methods for "normal" backups. That's
> an opinion, of course, but an opinion forged from many
> years of hard time in the trenches.
I do appreciate the information. Perhaps it is worth having it for just
occasional "emergency" backups, if nothing else.
Logan McLeod
>For us, the expense is not the issue. I have no problem spending
>money. But I like using standard tools that create readable tapes on
>any machine using standard syntax and command-line interface for cron.
The Supertars are just suped up tars... I can happily read my BackupEdge
created tapes with any standard tar command or utility.
>I don't remember all the issues, but it didn't seem like the backup
>products fit my bill, at the time. I don't need compression (hardware
>is much better) I don't need GUI interfaces, I am already running at
>full tape speed, I don't need to backup raw partitions, I have no
>problems with tapes being overwriten (robot drive), I don't need remote
>site interfaces, I don't want to have to upgrade yet something else all
>the time, I believe my emergency root/boot disks will allow full
>recovery (although I do need to test them again).
Ah...but wouldn't a little piece of mind be nice for the measly $300 or so
it costs?
:)
Logan
Bill Vermillion
Mark A. Davis <ma...@yylaketaylor.org> wrote:
>Tony Lawrence wrote:
>> Tape is very reliable, but not 100%. When you use products like
>> Edge or Lonetar, you learn that disturbing percentages of
>> backups have bit errors. Most of the time, of course, these
>> are in unimportant or easily recreated files, but sometimes they are
>> not.
>Hmm. I never thought of that as a potential problem on high-end
>drives, such as the HP DAT 24/6. With tape error correction and
>read-back and quality tapes that are rotated out of service
>regularly, regular cleanings, regular read-back for verification,
>etc. The only thing I can imagine that would cause an error would
>be some kind of highly unusual SCSI hardware and/or driver problem.
I've never seen any type of tape that hasn't failed for me at least
once - but I've been working around magnetic media longer than I
have computers.
However the current high-end devices with read after write, etc.,
can ensure that the data on the head is the same as the data the
tape received. HOWEVER there is no assurance that the data being
sent by the computer is the same as that received by the drive.
This is rare but it has been known to happen. To really be safe
after a system restore from tape, you should verify the HD against
the tape, presuming of course you verified the tape against the HD
on the orlginal copy.
I can imagine other scenarios not involving SCSI hardware or
drivers, and some that would involve the hardware.
If you are running in a 100% isolated environment - eg using
a motor/generator to totally isolate your system (yes that is done
in critical environments) you are at the mercy of Mother Nature,
the maintenance staff, or any plain old "WOOPS!".
So if you ran into one of these problems and had to restore and
found that your 2GB plus data file was corrupt, and it had been
regularly updated the previous day by 40 users. If you had to go
back one day prior, and have the corrupted data re-entered, you
have just lost 320 hours of production time.
You may say that is very unlikely to happen. A house fire is quite
unlikely too - but most people have insurance.
>For us, the expense is not the issue. I have no problem spending
>money. But I like using standard tools that create readable tapes on
>any machine using standard syntax and command-line interface for cron.
>I don't remember all the issues, but it didn't seem like the backup
>products fit my bill, at the time. I don't need compression (hardware
>is much better), I don't need GUI interfaces, I am already running at
>full tape speed, I don't need to backup raw partitions, I have no
>problems with tapes being overwriten (robot drive), I don't need remote
>site interfaces, I don't want to have to upgrade yet something else all
>the time, I believe my emergency root/boot disks will allow full
>recovery (although I do need to test them again).
The only real thing you are missing in the above scenario, or maybe
it's just an oversight, is a verify procedure to ensure that data
on tape matches that which resides on the disk.
As to a robot drive, anything mechanical can fail.
If you are against the commercial products, you might look at the
checktar script (which I've mentioned before) that was on
alt.sources.misc in the fall of 1990/91. It should give you enough
guts to roll your own and modify it as neccesary.
--
Bill Vermillion bv @ wjv.com
Bill Vermillion
Logan McLeod <mcleod...@dynamex.ca> wrote:
>>For us, the expense is not the issue. I have no problem spending
>>money. But I like using standard tools that create readable tapes on
>>any machine using standard syntax and command-line interface for cron.
>The Supertars are just suped up tars... I can happily read my
>BackupEdge created tapes with any standard tar command or utility.
Just be aware that all the super-tars, while doing the same job,
handle things internally the same way.
This is particularly true when using internal compression.
The anomoly I've seen (admittedly not on an iNTEL platform, but one
of the OS'es supported by at least one of the super-tar vendors),
is that compressed files are store differently. Some store
the compress files with a .Z extension, while others store the
file with their real name so they look like a normal file.
The surprise is that you'll get all sorts of 'fun' errors when you
try to execute what you thought was executeable.
It's an easy workaround, just force uncompress on the files in
question, or rename all the compressed files. Use the 'file'
command to identify the extension-less compressed files and take
your choice from there - pipe to uncompress or rename with a .Z
extension.
I've only experienced this one time and it was backing up
cross-platform. Since I don't have one each of the SCO supertars
I can't say if this behaviour exists in these.
Forewarned is forearmed. Just like posting the notice on the wall
where the fire-extinguisher is located.
Bill Vermillion
Gary L. Burnore <gbur...@databasix.com> wrote:
>On Tue, 24 Nov 1998 13:48:55 GMT, in article
>>The Supertars are just suped up tars... I can happily read my
>>BackupEdge created tapes with any standard tar command or utility.
>Unless you used compression in which case, you'll see the data but
>not be able to correctly restore it.
Wrong. See my other post - but I know of no super-tar that creates
files that can't be read back in - one way or another - to permit
manipulation of the data once it is retreived.
The Supertars didn't gain their reputation by using proprietary
methods of data storage.
Bill Vermillion
internet before I saw the error. Two missing words and the meaning
changes. Sorry.
In article <F2xnH...@bilver.magicnet.netREMOVETHIS>,
Bill Vermillion <bi...@bilver.magicnet.netREMOVETHIS> wrote:
>Just be aware that all the super-tars, while doing the same job,
>handle things internally the same way.
Make that read "while doing the same job, DO NOT handle things
internally in the same way".
According to usenet law the original will always be flamed, and the
correct post will be ignored.
Bill
Logan McLeod
Gary L. Burnore wrote in message <365ac981...@nntpd.databasix.com>...
>On Tue, 24 Nov 1998 13:48:55 GMT, in article
><bPy62.94$3C4.8...@news20.bellglobal.com>, "Logan McLeod"
><mcleod...@dynamex.ca> wrote:
>
>>
>>The Supertars are just suped up tars... I can happily read my BackupEdge
>>created tapes with any standard tar command or utility.
>
>Unless you used compression in which case, you'll see the data but not be
able
>to correctly restore it.
>
Uh...No, that's not true. tar is tar is tar is tar. I can use tar to
safely restore files on tapes made by backupedge with no problem whatsoever,
regardless of whether compression is on or off. You should actually >try<
it rather than giving inaccurate information.
L
Tony Lawrence
> I've never seen any type of tape that hasn't failed for me at least
> once - but I've been working around magnetic media longer than I
> have computers.
>
> However the current high-end devices with read after write, etc.,
> can ensure that the data on the head is the same as the data the
> tape received. HOWEVER there is no assurance that the data being
> sent by the computer is the same as that received by the drive.
Nor is there any assurance that some process hasn't changed the
disk data deliberately or otherwise. That little side effect
is another unexpected benefit of bit verifies: it shows you
that someone or something was messing with your data while
you backed it up. Sometimes that alone is reason to
scrap the tape as useless.
Mark A. Davis
>
> Bill Vermillion wrote:
>
> > I've never seen any type of tape that hasn't failed for me at least
> > once - but I've been working around magnetic media longer than I
> > have computers.
> >
> > However the current high-end devices with read after write, etc.,
> > can ensure that the data on the head is the same as the data the
> > tape received. HOWEVER there is no assurance that the data being
> > sent by the computer is the same as that received by the drive.
>
> Nor is there any assurance that some process hasn't changed the
> disk data deliberately or otherwise. That little side effect
> is another unexpected benefit of bit verifies: it shows you
> that someone or something was messing with your data while
> you backed it up. Sometimes that alone is reason to
> scrap the tape as useless.
Oh yes, that reminds me- that was another reason I was unwilling to use
a super-tar with bit verify- I do not want to take the system into
single-user mode to do a backup (certainly not nightly; talk about
asking for trouble). So, a bit verification is likely to fail on many
files during the hour backup, I would assume...
Taking this system down for such things is simply not an option, I don't
even view it as feasible for an "occasional perfect backup". Otherwise,
I do appreciate all the information that people provided for the group-
most useful and interesting!
Bill Vermillion
>>
>> Bill Vermillion wrote:
>> > I've never seen any type of tape that hasn't failed for me at least
>> > once - but I've been working around magnetic media longer than I
>> > have computers.
>> > However the current high-end devices with read after write, etc.,
>> > can ensure that the data on the head is the same as the data the
>> > tape received. HOWEVER there is no assurance that the data being
>> > sent by the computer is the same as that received by the drive.
>> Nor is there any assurance that some process hasn't changed the
>> disk data deliberately or otherwise. That little side effect
>> is another unexpected benefit of bit verifies: it shows you
>> that someone or something was messing with your data while
>> you backed it up. Sometimes that alone is reason to
>> scrap the tape as useless.
>Oh yes, that reminds me- that was another reason I was unwilling to use
>a super-tar with bit verify- I do not want to take the system into
>single-user mode to do a backup (certainly not nightly; talk about
>asking for trouble). So, a bit verification is likely to fail on many
>files during the hour backup, I would assume...
No one said take it to single user mode. The programs also do not
'fail' but generate a list of files that have changed in some way
from the backup to the compare. You can also list a set of files
to exclude from the verify if you wish.
However having the list to peruse, you could find that some file
changed that should not have changed.
>Taking this system down for such things is simply not an option,
>I don't even view it as feasible for an "occasional perfect
>backup". Otherwise, I do appreciate all the information that
>people provided for the group- most useful and interesting! --
Taking most systems down is not an option in today's world.
Schedule the backup/verify for the time when the system is least
busy.
Jean-Pierre Radley
| Oh yes, that reminds me- that was another reason I was unwilling to use
| a super-tar with bit verify- I do not want to take the system into
| single-user mode to do a backup (certainly not nightly; talk about
| asking for trouble). So, a bit verification is likely to fail on many
| files during the hour backup, I would assume...
What utter nonsense!!! The LAST thing you want to when making a backup
is to go into single-user mode. All of your secondary hard drives are
then unmounted, invisible, and not backed up.
None of the "super-tar" products even begin to suggest such a silly
procedure.
Mark, your head is stuck somewhere. Unstick it, grab one of the demos,
and try it!
Bob Stockler
>
> Oh yes, that reminds me- that was another reason I was unwilling to use
> a super-tar with bit verify- I do not want to take the system into
> single-user mode to do a backup (certainly not nightly; talk about
> asking for trouble). So, a bit verification is likely to fail on many
> files during the hour backup, I would assume...
Just to clear up a point about the bit-level verification the
"Super-Tar" programs do.
There's no need to go into single-user mode while the backup
or verification is done.
Files expected to change can be excluded from verification if
you so choose.
Other than that, the list of files that failed verification
shows why (differing date-time stamps, difference in sizes,
difference in some byte, etc.).
Some of those you don't care about.
But it would be nice to know, before restoring from a tape
that - for instance - "/stand/unix" on the tape to be used
to restore from didn't pass a bit-level verification with
what was on the hard disk at the time it was saved.
And equally important is a bit-level verification of the
restored file on the hard disk -vs- its previously verified
version on the tape.
--
Bob Stockler - b...@trebor.iglou.com - Sysop CompuServe SCOForum [75162,1612]
Tony Lawrence
>
> Tony Lawrence wrote:
> > Nor is there any assurance that some process hasn't changed the
> > disk data deliberately or otherwise. That little side effect
> > is another unexpected benefit of bit verifies: it shows you
> > that someone or something was messing with your data while
> > you backed it up. Sometimes that alone is reason to
> > scrap the tape as useless.
>
> Oh yes, that reminds me- that was another reason I was unwilling to use
> a super-tar with bit verify- I do not want to take the system into
> single-user mode to do a backup (certainly not nightly; talk about
> asking for trouble). So, a bit verification is likely to fail on many
> files during the hour backup, I would assume...
You misunderstand how these products work. None of them
expect that your system will be in single user mode.
Failure to verify a particular file, or even failure of a number
of files, does not mean the backup failed. It simply means
that the files that do not compare are listed for your informed
review. *You* are the person who decides if the backup "failed".
Further, if you KNOW that certain files will have changed, you
could exclude them from verification.
Tony Lawrence
>
> Mark A. Davis averred (on Fri, Nov 27, 1998 at 04:12:50PM -0500):
> | Oh yes, that reminds me- that was another reason I was unwilling to use
> | a super-tar with bit verify- I do not want to take the system into
> | single-user mode to do a backup (certainly not nightly; talk about
> | asking for trouble). So, a bit verification is likely to fail on many
> | files during the hour backup, I would assume...
>
> is to go into single-user mode. All of your secondary hard drives are
> then unmounted, invisible, and not backed up.
While not disagreeing as to the intent, I would point out that
"mountall" will mount the filesystems if that were the only
reason not to be single-user.
However, as JP and others have implied, it is silly and unnecessary.
Darrell Tschakert
Mark A. Davis wrote:
> Jean-Pierre Radley wrote:
> >
> > kevi...@my-dejanews.com averred (on Fri, Nov 20, 1998 at 03:00:10AM +0000):
> > | Looks like I will have to do full backups, make use of your
> > | savefiles script and pray. :-)
> >
> > Uh, no matter *which* method(s) you use to move to a morre recent
> > release of *any* operating system, I do not see how you can avoid
> > making full backups. That's just part of the job, you tell one of Bru,
> > LoneTar, Ctar or BackupEDGE to make and verify at least two archives
> > before you make any major changes at all.
>
> I hate to ask the question again, but won't a
>
> cd /
> find . -mount -print > !tape_all
> find ./stand -print >> !tape_all
> cat !tape_all | cpio -oHcrc -C 10240 > /dev/rct0
>
> suffice as a complete system backup?
Yes. We regularly clone systems from a master. We make
a cpio backup and reload it onto a new system. Once loaded
we make minor changes to account for the differences in
hardware. The master has a /unix and an /etc/conf directory
for each different SCSI controller that we use on our various
systems. I have done this for years and have yet to see a failure.
It seems that if a file like sed, cpio, or unix itself was off by a
bit or two, it would fail. They NEVER fail unless I have a bad
tape or drive and then I know about it at load time.
>
>
> I know that if there were a total systems failure, I would have to
> reinstall the old OS enough to get to the tape drive, and then worry
> about disk partitions (we have a RAID system, though). But otherwise,
> are there any other obvious dangers?
To reinstall from a cpio tape, we just have a short shell script
on a boot floppy that
runs fdisk and divvy, creates lost+found's and adds the slots,
and then asks for the tape. I think that the backup programs from
LoneStar etc do the same thing. I have bought and used the
commercial backup programs but just never fell in love with
them. I seldom have time for the bit error check anyway. If
I had a little more time and a little more money, then perhaps
I would make the switch, but then real programmers ... ... .
Don't email me next week when you discover that the tape
you made with cpio failed to load properly. :-)
--
NOTE: REMOVE THE "NoNo" IN MY ADDRESS BEFORE SENDING
>> Darrell Tschakert >> Phone: (202) 663-4436 >>
>> Equal Employment Opportunity Commission >> Sent from Work. >>
>> 1801 L Street N.W. >> Email to: >>
>> Washington D.C. 20507 >> dtsc...@NoNoerols.com >>
NOTE: REMOVE THE "NoNo" IN MY ADDRESS BEFORE SENDING
Bill Vermillion
Darrell Tschakert <dtsc...@erols.com> wrote:
>Mark A. Davis wrote:
>> Jean-Pierre Radley wrote:
>> > Uh, no matter *which* method(s) you use to move to a morre
>> > recent release of *any* operating system, I do not see how you
>> > can avoid making full backups. That's just part of the job, you
>> > tell one of Bru, LoneTar, Ctar or BackupEDGE to make and verify
>> > at least two archives before you make any major changes at all.
>> I hate to ask the question again, but won't a
>> cd /
>> find . -mount -print > !tape_all
>> find ./stand -print >> !tape_all
>> cat !tape_all | cpio -oHcrc -C 10240 > /dev/rct0
>> suffice as a complete system backup?
Most of the times yes. However what about those fairly rare times
when only a bit-level verify will show corruption.
It's rare, but it can and does happen. If data becomes corrupted
after it is read by the drive, and before it is written to tape,
then you could have the possibility that checksum will be
calculated against the corrupt data. In this case a verify of the
tape will be ok, however it's not going to be correct.
Having two backups will reduce things a bit, as you just need to be
lucky enough to choose the one tape that is good out of two
availaalbe.
This is rare occurance. However there's something else that is
rare, which will only be caught with a verifying utility.
That is on a restore, when the data becomes corruprt after reading
from the verified master on it's way to being re-written.
>Yes. We regularly clone systems from a master. We make
>a cpio backup and reload it onto a new system. Once loaded
>we make minor changes to account for the differences in
>hardware. The master has a /unix and an /etc/conf directory
>for each different SCSI controller that we use on our various
>systems. I have done this for years and have yet to see a failure.
The day it will fail will be the day you can least afford to have
it fail.
>It seems that if a file like sed, cpio, or unix itself was off by a
>bit or two, it would fail. They NEVER fail unless I have a bad
>tape or drive and then I know about it at load time.
>> I know that if there were a total systems failure, I would have
>> to reinstall the old OS enough to get to the tape drive, and then
>> worry about disk partitions (we have a RAID system, though). But
>> otherwise, are there any other obvious dangers?
Isn't that though enough to make it worth while. It's typically 2
or 3 minutes from a Super-Tar backup until you are loading the OS
if you've replaced a dead-drive with a verified drive.
>I seldom have time for the bit error check anyway. If
>I had a little more time and a little more money, then perhaps
>I would make the switch, but then real programmers ... ... .
The nice thing about the super products is they can save you time
if used properly.
The problems with not having enough money is similar to saving
money by not buying fire insurance. I've seen data tapes fail
more often at client sites, than I have seen fires at the sites.
One was spectalar however - as 9600V xformer/feed to the building
blew up and left melted 2" AC conduit. Some companies could fail
with no backup.
Tom Parsons
| In article <368A4F97...@erols.com>,
| Darrell Tschakert <dtsc...@erols.com> wrote:
|
| >Mark A. Davis wrote:
|
| >> Jean-Pierre Radley wrote:
|
| >> > Uh, no matter *which* method(s) you use to move to a morre
| >> > recent release of *any* operating system, I do not see how you
| >> > can avoid making full backups. That's just part of the job, you
| >> > tell one of Bru, LoneTar, Ctar or BackupEDGE to make and verify
| >> > at least two archives before you make any major changes at all.
|
| >> I hate to ask the question again, but won't a
|
| >> cd /
| >> find . -mount -print > !tape_all
| >> find ./stand -print >> !tape_all
| >> cat !tape_all | cpio -oHcrc -C 10240 > /dev/rct0
|
| >> suffice as a complete system backup?
|
| Most of the times yes. However what about those fairly rare times
| when only a bit-level verify will show corruption.
Those rare times are becoming more common as processor/bus speeds
increase and people continue to insist on using poor quality scsi
bus cables/construction.
--
==========================================================================
Tom Parsons t...@tegan.com Sysop, SCOForum-CompuServe
==========================================================================
Bill Vermillion
Tom Parsons <c...@tegan.com> wrote:
>Bill Vermillion enscribed:
>| In article <368A4F97...@erols.com>,
>| Darrell Tschakert <dtsc...@erols.com> wrote:
>|
>| >Mark A. Davis wrote:
>|
>| >> Jean-Pierre Radley wrote:
>|
>| >> > Uh, no matter *which* method(s) you use to move to a morre
>| >> > recent release of *any* operating system, I do not see how you
>| >> > can avoid making full backups. That's just part of the job, you
>| >> > tell one of Bru, LoneTar, Ctar or BackupEDGE to make and verify
>| >> > at least two archives before you make any major changes at all.
>|
>| >> I hate to ask the question again, but won't a
>|
>| >> cd /
>| >> find . -mount -print > !tape_all
>| >> find ./stand -print >> !tape_all
>| >> cat !tape_all | cpio -oHcrc -C 10240 > /dev/rct0
>|
>| >> suffice as a complete system backup?
>|
>| Most of the times yes. However what about those fairly rare times
>| when only a bit-level verify will show corruption.
>Those rare times are becoming more common as processor/bus speeds
>increase and people continue to insist on using poor quality scsi
>bus cables/construction.
Thankfully my customers follow my recommendations. Have you priced
wide scsi telfon coated silver cables lately. Ain't cheap, but
they are really good. The higher end SCSI manufacturers recommend
them. A really need once you have more than a couple of SCSI
targets on a SCSI2 Wide/fast controller.
Whats really going to show borderline material up are the new
drives that can now sustain 22MBytes/second. If they get much
faster you could hear them on an FM radio set :-)
Darrell Tschakert
>
> In article <368A4F97...@erols.com>,
> Darrell Tschakert <dtsc...@erols.com> wrote:
>
> >Mark A. Davis wrote:
>
> >> Jean-Pierre Radley wrote:
> Most of the times yes. However what about those fairly rare times
> when only a bit-level verify will show corruption.
>
> It's rare, but it can and does happen. If data becomes corrupted
> after it is read by the drive, and before it is written to tape,
> then you could have the possibility that checksum will be
> calculated against the corrupt data. In this case a verify of the
> tape will be ok, however it's not going to be correct.
It just seems that if this sort of thing is happening then
it will be noticed. The executable files will not run, the
limited checksums built into cpio, the parity checks on the
SCSI buss, and so on would tell me if I am having an integrity
problem. Granted, I would not want to gamble the Earth on
it but I would certainly gamble a lot of money on it. There
are a number of considerations involved here. How much extra
time is it going to take me? is it an important data base or
is it a full system backup of programs? I, often, don't have
the time when I am cloning a machine. If I am backing up a data
base and I start the backup just before I leave for home, it
costs me nothing to do a verify since it is going to be done
when I return the next day whether I do a verify or not.
One of the people who I work with does a check sum on every
file in the backup - before the files go onto tape.
He then does another check sum after the
files are loaded onto the clone machine. He does this all with
a shell script. Given a little bit of thought, I think that
the integrity of a tape could be assured with programs found
on a SCO system or in a free GNU library.
>
> Having two backups will reduce things a bit, as you just need to be
> lucky enough to choose the one tape that is good out of two
> availaalbe.
>
> This is rare occurance. However there's something else that is
> rare, which will only be caught with a verifying utility.
>
> That is on a restore, when the data becomes corruprt after reading
> from the verified master on it's way to being re-written.
>
I don't quite understand here. Does the verify program do a
verify on the file after it is written to disk? If so couldn't
I just do a check sum on the file after it is written.
> >Yes. We regularly clone systems from a master. We make
> >a cpio backup and reload it onto a new system. Once loaded
> >we make minor changes to account for the differences in
> >hardware. The master has a /unix and an /etc/conf directory
> >for each different SCSI controller that we use on our various
> >systems. I have done this for years and have yet to see a failure.
>
> The day it will fail will be the day you can least afford to have
> it fail.
If it fails, I'll go make another tape. We just don't have
the extra time. If I would see a problem caused by a lack of
verify, then perhaps I would reconsider.
I have the person who does the daily and weekly backups read the
tape with this command "cpio -itcv < /dev/rct0" after she creates
them, just to make certain that I have no gross problems.
>
> >It seems that if a file like sed, cpio, or unix itself was off by a
> >bit or two, it would fail. They NEVER fail unless I have a bad
> >tape or drive and then I know about it at load time.
>
> >> I know that if there were a total systems failure, I would have
> >> to reinstall the old OS enough to get to the tape drive, and then
> >> worry about disk partitions (we have a RAID system, though). But
> >> otherwise, are there any other obvious dangers?
>
> Isn't that though enough to make it worth while. It's typically 2
> or 3 minutes from a Super-Tar backup until you are loading the OS
> if you've replaced a dead-drive with a verified drive.
>
> >I seldom have time for the bit error check anyway. If
> >I had a little more time and a little more money, then perhaps
> >I would make the switch, but then real programmers ... ... .
>
> The nice thing about the super products is they can save you time
> if used properly.
>
> The problems with not having enough money is similar to saving
> money by not buying fire insurance. I've seen data tapes fail
> more often at client sites, than I have seen fires at the sites.
If you are just worried about the tape failing in a gross sort of
way then read the tape with "cpio -itcv < /dev/rct0". Once you have
done that it is just as likely that your tape will fail as mine.
>
> One was spectalar however - as 9600V xformer/feed to the building
> blew up and left melted 2" AC conduit. Some companies could fail
> with no backup.
Certainly, no one would argue with this. I think that we have
drifted considerably from the original posting. The poster
wanted to know why not use find/cpio/rct0. It really depends
what the tape will contain. I can't afford the extra time
for verification of a tape that I am using for cloning. Nor
can I afford the time and money to do bit level verifies on
full system backups of our development machines. I do, however
ask the backup person to read newly created tapes with
"cpio -itcv < /dev/rct0". And, from time to time, I also in
a large data file from the backup tape and do a checksum
on it just to see if our backups are any good.
Darrell Tschakert
>
> --
> Bill Vermillion bv @ wjv.com
--
******* PLEASE REMOVE THE "NoNo" FROM MY URL ******
<< Darrell Tschakert || email dtsc...@NoNoErols.com>>
<< EEOC || National Data Base, Chief >>
<< 1801 L St. NW || Phone: (202) 663-4436> >>
<< Washington, DC 20507 || Sent from my home computer. >>
Bill Vermillion
>> In article <368A4F97...@erols.com>,
>> Darrell Tschakert <dtsc...@erols.com> wrote:
>> >Mark A. Davis wrote:
>> >> Jean-Pierre Radley wrote:
> <snip>
>> Most of the times yes. However what about those fairly rare times
>> when only a bit-level verify will show corruption.
>> It's rare, but it can and does happen. If data becomes corrupted
>> after it is read by the drive, and before it is written to tape,
>> then you could have the possibility that checksum will be
>> calculated against the corrupt data. In this case a verify of the
>> tape will be ok, however it's not going to be correct.
>It just seems that if this sort of thing is happening then
>it will be noticed. The executable files will not run, the
>limited checksums built into cpio, the parity checks on the
>SCSI buss, and so on would tell me if I am having an integrity
>problem.
I've seen errors on bit-level verify - not more than twice -
when the data on tape did not check with the HD and the HD had not
been modified. Somewhere between the HD read head and the tape
write head, somthing went away. That's why you CAN NOT trust
checksums to verify a tape backup. Those will be computed on the
data received, and all this will do is tell you if you have a
verifiable tape.
> Granted, I would not want to gamble the Earth on
>it but I would certainly gamble a lot of money on it.
$100,000.00 - more ? It all depends on your company and the useage
they have/need. Remember that almost anything is possible in a
computer. It could be that your executeable would run, but just
not run correctly. For example, suppose you had a backup that ran
only at night. At 4PM the next day someone ran a program that 'was
not exactly right' - that had come from a backup that was not
verified in BOTH directions - HD against tape in storage, and tape
against HD in restore.
Now further suppose that this errant program trashed the data base.
Now you can restore from the previous days backup with no problem.
But what about today's data. A small company could probably
re-enter most of it in a few days. Now consider just a medium
sized company, with 40 users modifying the database. Many of these
are taking order over the phone too. Quite often no paper exists.
So you've lost 40 users work for 8 hours - 320 hours. Thats' the
equivalent of giving one person 2 months vacation with pay.
But what about the orders you taken that day. What about anything
that was shipped that day. You are now with an out-of-date
inventory, you have customers calling about their 'lost' order.
The worst thing you can do is blame it on a computer problem.
That was a good excuse in the old days, but to use that now can
indicated to many customers that you don't know what you're doing
when it comes to computers, and by extension, they could come to
think that maybe you have other hidden things to watch out for.
That's an extreme scenario, but stranger things have happened.
>There are a number of considerations involved here. How much extra
>time is it going to take me?
Wrong question. How much is it going to COST the company if you
have a problem.
>is it an important data base or is it a full system backup of
>programs? I, often, don't have the time when I am cloning a
>machine.
Hm. Let me ask you just one question. "If you don't have the time
to do it right the first time, when are you going to find time to
fix it later?"
>If I am backing up a data base and I start the backup just before
>I leave for home, it costs me nothing to do a verify since it is
>going to be done when I return the next day whether I do a verify
>or not.
With the easy of the SuperTars to backup full systems, and the cost
of downtime can be pretty bad. By backing up a the full system
with one of the SuperTars - and their emergency recovery disks,
if you lose an HD - and SCSI drives don't need formatting - you can
be loading the entire OS and database from tape within 5 minutes
after rebooting from the disk. The fastest I've seen an OSR5
install was about 30 minutes. Get's longer the slower the machine.
One took about 5 hours. I started it and went home. After the
install you might need to configure the tape if not useing the
defaults. Then you can reload. However, you also need a system
backup from somewhere in the past, otherwise you have the 'joy' of
installing all the apps, and adding all the users. And if you have
a system backup you MUST MUST MUST make a new tape EVERYTIME you
change users, add users, etc., or you get the 'fun' of trying to
remember exactly what you did change.
>One of the people who I work with does a check sum on every file in
>the backup - before the files go onto tape. He then does another
>check sum after the files are loaded onto the clone machine. He
>does this all with a shell script.
That would work, but I prefer a commercial supported solution for
my customers. If you are really comfortable at that level, with
checksums computed prior to save, and after restore, in addition to
the checksum on the tape, then you should be pretty safe.
Are checksums 100% infallible. I've heard/read that under certain
circustances you could compute the same checksums for two different
data input. However - that may only be urban myth. I'll leave
that to the real programmers to answer.
Somewhere in the above description you neglected to indicate
comparing the checksums computed on the target with those on the
source. I assume it is there.
>Given a little bit of thought, I think that
>the integrity of a tape could be assured with programs found
>on a SCO system or in a free GNU library.
Yes. Given thought and work, and looking at all the eventualities.
It is highly dependant on your expertise, the cost of lost time,
etc. For many people the cost of a commerical program, fully
supported, is more economical than having a (probably) overworked
staff conjur a backup schema.
If you want to do bit-level verifies you can roll one yourself.
You might want to see if you can find the 'checktar' program from
alt.sources.misc - fall of 1990 or 1991. It appeared to be the
groundwork for at least one of the super-tar products I've seen.
>> This is rare occurance. However there's something else that is
>> rare, which will only be caught with a verifying utility.
>> That is on a restore, when the data becomes corruprt after reading
>> from the verified master on it's way to being re-written.
>I don't quite understand here. Does the verify program do a
>verify on the file after it is written to disk? If so couldn't
>I just do a check sum on the file after it is written.
Here is what can happen. Date is read from the head on the HD.
The HD will perform a checksum verification. Data is then sent to
the tape drive. Somewhere before arriving at the circuits in the
tape drive before the tape checksum is written, a bit or bits will
become corrupt. The tape checksum now will verify that the
incorrect data is correct. It has computed a checksum on invalid
data. Thus the false sense of security by just doing a 'tape
verify' command, which only assures you that the tape is readable,
not that your data contents are accurate.
Running a checksum after it is written will give you data that you
can compare with checksums generated as it is read. This gives you
more assurance,but it is still not the same as a bit-level verify,
as I'm sure that Tom (Microlite), Jeff (LoneTar), Steve (Unitrends)
and (darn forgot his name from BRU) can explain to you.
>> The day it will fail will be the day you can least afford to have
>> it fail.
>If it fails, I'll go make another tape. We just don't have
>the extra time. If I would see a problem caused by a lack of
>verify, then perhaps I would reconsider.
See my above comment. Taken to the EXTREME. What happens when the
backup to the clone fails, and then you go to make another and that
machine fails. Far fetched?
Not really. I had an SGI Challenge S server that did a nightly
verified backup onto an SGI Indy - using Lone-Tar. (BRU was also
available for that platform but I didn't know the SGI port was so
old).
We had a problem, programmer error, that needed the OS to be
reloaded and the tape to be reloaded. (they don't have emergency
boot disks for most non-SCO environmets, and floppy disks are a
rarity there too.).
After spending the afternoon reloading the OS, doing other
configuration, etc., I left about 9PM and went home expeciting the
backup/restore to finish about 3 to 4 am. (backups over a network
do take longer).
I came in and found that the system was still down HARD. The tape
drive PHYSICALLY FAILED during the restore. So while waiting for a
new tape drive to arrive, I reloaded and reconfigured most of the
OS. Thankfully the data was on a second hard drive so it only
required reloading the bare Netscape server - as all the web data
was fine.
As I said before, anything that can go wrong, will go wrong.
>I have the person who does the daily and weekly backups read the
>tape with this command "cpio -itcv < /dev/rct0" after she creates
>them, just to make certain that I have no gross problems.
Which will not correct corrupt data descirbed in the above
scenario. And why do you have the person do this? A short script
would be much better. I have SEEN people get the < and > confused,
or just missed, as they are adjacent. In the above scenario it
would destroy your backup. I also saw a person redirect the
inittab into some program ?? (I didn't teach them how to use it!).
They mis-typed the < and hit >. Now the inittab had to be
reloaded from the last backup, and the changes made that day had to
be entered again.
Anything that can go wrong, will go wrong. It bears repeating.
>> >It seems that if a file like sed, cpio, or unix itself was off by a
>> >bit or two, it would fail. They NEVER fail unless I have a bad
>> >tape or drive and then I know about it at load time.
Nope. See above.
>> The problems with not having enough money is similar to saving
>> money by not buying fire insurance. I've seen data tapes fail
>> more often at client sites, than I have seen fires at the sites.
>If you are just worried about the tape failing in a gross sort of
>way then read the tape with "cpio -itcv < /dev/rct0". Once you have
>done that it is just as likely that your tape will fail as mine.
Nope - see above as to why a tape drive verifies via a checksum
which may be wrong.
See my above scenario on tape failure relating only to your clone
efforts.
As to the fire. I had a client whose tape drive died. He thought
they were too expensive. I told him I had an idea whereby he could
get a new tape drive and have some extra money too. "How?" he
said.
"Cancel your fire insurance and take the premiums and buy a new
tape drive, as your system will fail before you have a fire".
Now the odds are in my favor on that, but there is nothing to say
that he couldn't have a fire the instant he hung up the phone, or
worse, right after he canceled his insurance.
>Certainly, no one would argue with this. I think that we have
>drifted considerably from the original posting. The poster
>wanted to know why not use find/cpio/rct0. It really depends
>what the tape will contain.
No it does not really depend on what the tape contains. It depends
entirely on how much you trust your data and how much you rely on
it.
>I can't afford the extra time
>for verification of a tape that I am using for cloning. Nor
>can I afford the time and money to do bit level verifies on
>full system backups of our development machines.
With the super-tar products your development systems can be backed
up automatically, rewound and bit level verified. Then the program
will send you mail and/or use the system printer to tell you the
success or failure.
Why does having a machine do this overnight take time? I would
think that it would save time.
>I do, however ask the backup person to read newly created tapes
>with "cpio -itcv < /dev/rct0". And, from time to time, I also in a
>large data file from the backup tape and do a checksum on it just
>to see if our backups are any good.
False sense of security if you aren't comparing those checksums to
the original checksums before saved from tape.
One more horror story. This was back before things were as good
and convenient as they are now.
I had a holiday time-frame in which to transfer apps and data from
a 68000 Unix platform into a WE23010 platform. (one was little
Endian and the other was Big Endian - which meant that the data
files had to converted to otherwise the index pointers were
backwards. This conversion was done during the transfer).
The machine arrived Wedneday moring, December 24. About 1 PM I
found that the base machine had a 35MB drive instead of the 70MB,
and that the expansion drive/bay - 70 MB was okay.
We called the distibutor. Close early for Christmas. I did some
system configuration, adding users, etc., and backed up everything.
I ran the 'verify' program that was on the OS backup menu. Tape
verified just fine.
Friday AM we call. Somebody had accidentally swapped packages on
the loading dock. They'd air freight one out for Saturday
delivery. I then did more more work. Friday night another backup
followed by a verify that said it was good.
Saturday AM, more work. A backup about 1PM as the new machine
arrived. Machine gets set up, things get configured, OS gets
installed from floppies (aint that fun), and I'm ready to pickup
where I left off.
First tape would not read. That was this mornings tape. Second
tape. Would not read. That was last nights. That only left the
original tape. The dealer suggested I use that. I declined as I
had tried 2 of 3 backups and the tape drive was integral to the
machine, so I did not know if I had made good tapes, and the new
drive was destroying them, or if the original ones were bad.
It was after 11PM, so about 1.5 hours later we arrived at the HW
dealers office in Orlando and tried out the tapes. They were bad
on this one too.
Since the verify routine was hidden behind menus, I did not know
the exact method it was using. However in the end it appeared that
the tape drive wrote nothing to the tape, and upon verify it also
verified that nothing was written to the tape. It was probably an
extremely out of alignment tape head - but by that time the old
unit was boxed up - and far too much time had been lost then.
As I said before, anything that can go wrong will go wrong.
Prepare for ALL eventualites. When I was a recording engineer we'd
make safety copies of masters before shipping them to the pressing
plant. In one rush situation someone said "we don't have time to
make a safety, it has to be at UPS/FEDEX/whomever in the next 30
minutes".
Tape never arrived. Not even in the late afternoon delivery. So
we got the master tapes, spent 8 to 10 hours remixing it (small
session, fairly easy mix), squencing it, editing, putting on the
tones, and making a safety - just in case.
And about 4 months later - the tapes showed up at our door, as we
had the studio's name and address physically on the reels.
Somewhere in the jouncing and bouncing the tapes put enough stress
on the package side to come out of their packaging and fall to
1) floor of the truck 2) floor of the transport 3) warehouse
transfer point floor 4) somewhere else entirely!. Pick 1 of the 4.
Saving time cost a lot more that time. If we could be sure of
everthing we would only buy car insurance the day we were going to
be in a wreck, health insurance the day before we got sick, and
fire insurance the day before the neighborhood kids set your house
on fire playing with matches.
Before I was a recording engineer I worked in broadcast. I've been
working with magnetic tape longer than many of the posters to this
group have been alive. (that's literally not figuratively).
I've seen tape get better and better. But still things fail.
I had a 2" reel in the studio start having drop-outs in the last
minutes - centered on the middle tracks of a 24 track tape.
Found a bump in the tape. Unwrapped it further, and found that spider
had gotten trapped during the spooling process at 3M's plant.
Distorted the tape enough so it would not contact the heads
properly at that point.
The only time I've had from any manufacturer was when I used so
little of one brand I never had enough use for any meaninful data.
Only you can determine the real costs of a failure along the line.
In very small systems, it's not a big deal. Before we upgrade to a
faster DAT at a client it was about 1.5 hours to write, and another
to verify. However, having to restore will take much longer, it
was about 4 hours on restoration as files had to be written.
The newer drive gets all 8GB data in under 40 minutes. When I do
ANY work on that - always after hours - I unmount the data drive,
and then do a full system backup on the OS - time is under 10
minutes. Had to use that when new serial drivers totally made
things unworkable.
Have I said, if anything can go wrong it will go wrong?
Lots of luck. And may all your bits be verified.
Bill
(Gosh that got to be along post. I hope it helps save someone from
disaster)
Kevin Smith
>...snip...
>Are checksums 100% infallible. I've heard/read that under certain
>circustances you could compute the same checksums for two different
>data input. However - that may only be urban myth. I'll leave
>that to the real programmers to answer.
Well, a 1k block can have 2^8192 states and a 32 bit checksum can have
2^32 states, therefore, for every checksum value there are 2^256 states
of a 1k block that can generate that value.
The value of a good checksum algorithm is a kind of "chaotic" behavior
where small changes (i.e. 1 bit) in the input always result in
changes in the checksum.
--
Do two rights make | Kevin Smith, ShadeTree Software, Philadelpha, PA, USA
a libertarian | 001-215-487-3811 shady.com,kevin bbs.cpcn.com,sysop
Bill Vermillion
Kevin Smith <kbs=cu...@shady.com> wrote:
>In article <F5Auo...@bilver.magicnet.netREMOVETHIS> bi...@bilver.magicnet.netREMOVETHIS (Bill Vermillion) writes:
>>...snip...
>>circustances you could compute the same checksums for two different
>>data input. However - that may only be urban myth. I'll leave
>>that to the real programmers to answer.
>
>Well, a 1k block can have 2^8192 states and a 32 bit checksum can have
>2^32 states, therefore, for every checksum value there are 2^256 states
>of a 1k block that can generate that value.
>
>The value of a good checksum algorithm is a kind of "chaotic" behavior
>where small changes (i.e. 1 bit) in the input always result in
The 'always' was the part I've questioned. Maybe things are new
and improved now, but at least at one time is was thought that
there 'might' be a change somewhere - typically two changes - that
could compute the same checksums. Have things changed or should it
"almost always". Or changed that to read "small changes will
NEVER result in changes in the checksum". I tend to get concerned
with words like 'never' and 'always'. In a great many cases these
don't hold true.
Kevin Smith
>In article <77aoig$r...@shady.shady.com>,
>Kevin Smith <kbs=cu...@shady.com> wrote:
>>In article <F5Auo...@bilver.magicnet.netREMOVETHIS> bi...@bilver.magicnet.netREMOVETHIS (Bill Vermillion) writes:
>>>...snip...
>>>circustances you could compute the same checksums for two different
>>>data input. However - that may only be urban myth. I'll leave
>>>that to the real programmers to answer.
>>
>>Well, a 1k block can have 2^8192 states and a 32 bit checksum can have
>>2^32 states, therefore, for every checksum value there are 2^256 states
>>of a 1k block that can generate that value.
>>
>>The value of a good checksum algorithm is a kind of "chaotic" behavior
>>where small changes (i.e. 1 bit) in the input always result in
>>changes in the checksum. ^^^^^^
>
>The 'always' was the part I've questioned. Maybe things are new
>and improved now, but at least at one time is was thought that
>there 'might' be a change somewhere - typically two changes - that
>could compute the same checksums. Have things changed or should it
>"almost always". Or changed that to read "small changes will
>NEVER result in changes in the checksum". I tend to get concerned
>with words like 'never' and 'always'. In a great many cases these
>don't hold true.
This is where we need some input from the mathmaticians.
Error correction algorithms are a form of checksums that can guarantee
a level of performance. I know a three bit checksum on an eight bits
can correct one bit errors and detect two bit errors (ECC). On DAT
tapes something like 1/3'rd of the data is the error correction
checksums and they allow a great deal of correction and detection
(i.e. they can correct several hundred bytes in a 1k block).
Something as simple as xor'ing all the bytes (words, whatevers) will
always catch a single bit error. For that matter, if you are doing
bytes, it will 'always' detect any single byte change (or you could
say it will always detect any single bit change in any bit column).
XOR is, it would seem, not bad for detecting point errors but not so
good if errors will be distributed.
Bill Vermillion
Kevin Smith <kbs=cu...@shady.com> wrote:
>In article <F5CwA...@bilver.magicnet.netREMOVETHIS> bi...@bilver.magicnet.netREMOVETHIS (Bill Vermillion) writes:
>>>The value of a good checksum algorithm is a kind of "chaotic"
>>>behavior where small changes (i.e. 1 bit) in the input always
>>>result in changes in the checksum. ^^^^^^
>>The 'always' was the part I've questioned. Maybe things are new
>>and improved now, but at least at one time is was thought that
>>there 'might' be a change somewhere - typically two changes - that
>>could compute the same checksums. ...
>This is where we need some input from the mathmaticians.
>Error correction algorithms are a form of checksums that can guarantee
>a level of performance. I know a three bit checksum on an eight bits
>can correct one bit errors and detect two bit errors (ECC).
I have one machine that has ECC. So far I've seen three messges
over a 2 year period - in the system log - where there was an ECC
memory correction made at 0xnnnnnnn. Works well.
> On DAT tapes something like 1/3'rd of the data is the error
>correction checksums and they allow a great deal of correction and
>detection (i.e. they can correct several hundred bytes in a 1k
>block).
Generically they are called DAT's but the correct name for data is
DDS. The difference is that in the original DAT, there was a level
of error correction, and if that failed there was error
concealment. When my audio-engineer friends think their audio
devices are superior, I bring up the extra level of correction with
this story.
If I'm doing something like processing check, with the first check
being $1.00, data for the second check missing, and the 3rd
$1000.00, I can not assume the missing check was for $500. They
get the picture.
Years ago when Ampex was a viable company and they were
experimenting with digital audio, they had the signals scattered
across and along the tape so that you could take a 1/4" hole punch
to the tape and still get 100% recovery.
Back in the days when floppies were the only means (or only
remotely affordable means) of backup for small systems, I ran
across a product that did error correction to the n'th degree (or
at least n-1).
After a backup you could take a floppy and punch a hole in it.
Put it in and recover everything.
ECC is good. As to checksums? I'm dubious. (Hello Mr Dubious.
How is Mrs Dubious and all the kids). [for the trivia quiz of the
day who has any idea where that came from? Just a generic answer
will do]
makes me a little dubious.
jrich...@my-dejanews.com
In article <F5GFt...@bilver.magicnet.netREMOVETHIS>,
> --
> Bill Vermillion bv @ wjv.com
>
-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own
Darrell Tschakert
Bill Vermillion wrote:
I think that the original poster asked something about whether or
not a find ... ... | cpio ... ... would do. The answer is still "yes"
it has always worked just fine for me. There are lots of checks
put in along the way even in cpio. But the answer to whether or
not it can be done is that we have done it for hundreds of machines
and we have yet to see ANY indication that we have a problem.
They have a financial interest in convincing me. I am just answering
the question the was posted earlier.
>
> >> The day it will fail will be the day you can least afford to have
> >> it fail.
>
> >If it fails, I'll go make another tape. We just don't have
> >the extra time. If I would see a problem caused by a lack of
> >verify, then perhaps I would reconsider.
>
> See my above comment. Taken to the EXTREME. What happens when the
> backup to the clone fails, and then you go to make another and that
> machine fails. Far fetched?
Yes, very far fetched. I want to see one problem before I start spending
an extra hour or two on every tape that I make to clone a machine.
>
<snip>
>
> >I have the person who does the daily and weekly backups read the
> >tape with this command "cpio -itcv < /dev/rct0" after she creates
> >them, just to make certain that I have no gross problems.
>
> Which will not correct corrupt data descirbed in the above
> scenario. And why do you have the person do this? A short script
> would be much better. I have SEEN people get the < and > confused,
This person logs in and enter bkup.sh which contains the command.
You are taking things too literal.Anything that can go wrong, will go
wrong. It bears repeating.
>
<snip>
>
> >> >It seems that if a file like sed, cpio, or unix itself was off by a
> >> >bit or two, it would fail. They NEVER fail unless I have a bad
> >> >tape or drive and then I know about it at load time.
>
> Nope. See above.
Again, yup.
>
>
> >> The problems with not having enough money is similar to saving
> >> money by not buying fire insurance. I've seen data tapes fail
> >> more often at client sites, than I have seen fires at the sites.
>
> >If you are just worried about the tape failing in a gross sort of
> >way then read the tape with "cpio -itcv < /dev/rct0". Once you have
> >done that it is just as likely that your tape will fail as mine.
>
> Nope - see above as to why a tape drive verifies via a checksum
> which may be wrong.
>
Again, Yup. Just because you can think of some 10 to the minus
14 sincerio does not mean the I am going for go through life
fearing it.
I am still waiting for the first problem.
>
> See my above scenario on tape failure relating only to your clone
> efforts.
>
> As to the fire. I had a client whose tape drive died. He thought
> they were too expensive. I told him I had an idea whereby he could
> get a new tape drive and have some extra money too. "How?" he
> said.
>
> "Cancel your fire insurance and take the premiums and buy a new
> tape drive, as your system will fail before you have a fire".
>
> Now the odds are in my favor on that, but there is nothing to say
> that he couldn't have a fire the instant he hung up the phone, or
> worse, right after he canceled his insurance.
There is nothing to say that a meteor will not fall on me on the
way home from work. Maybe I should stay inside to avoid this.
Certainly my life is worth more than some data. And the odds
of something falling on me are about as greate as us haveing a
problem that never becomes know to us.
>
>
> >Certainly, no one would argue with this. I think that we have
> >drifted considerably from the original posting. The poster
> >wanted to know why not use find/cpio/rct0. It really depends
> >what the tape will contain.
>
> No it does not really depend on what the tape contains. It depends
> entirely on how much you trust your data and how much you rely on
> it.
Oh. I thought it depended on what was on the tape. Well now I
know.
>
>
> >I can't afford the extra time
> >for verification of a tape that I am using for cloning. Nor
> >can I afford the time and money to do bit level verifies on
> >full system backups of our development machines.
>
> With the super-tar products your development systems can be backed
> up automatically, rewound and bit level verified. Then the program
> will send you mail and/or use the system printer to tell you the
> success or failure.
Send me email ... ?????. I'm running around getting the shipping docs
ready.
When I come back that machine had better be on tape and ready
to place in the clone. Then get out of the way because I have to
go call Joe Blow who's machine is coughing blood while the
clone read the new tape. Then get out of the way because
the packer has to get the machine down to the FedEx station
by 5:00 PM and we didn't start putting this thing together until
3:00 PM.
(I hope no one who Really know us is reading this. :-)
>
>
> Why does having a machine do this overnight take time? I would
> think that it would save time.
Don't understand. the question. I must have deleted something here.
>
>
> >I do, however ask the backup person to read newly created tapes
> >with "cpio -itcv < /dev/rct0". And, from time to time, I also in a
> >large data file from the backup tape and do a checksum on it just
> >to see if our backups are any good.
>
> False sense of security if you aren't comparing those checksums to
> the original checksums before saved from tape.
>
And what about the meteor? Do I dare leave work tonight?
>
> One more horror story. This was back before things were as good
> and convenient as they are now.
>
Not another story. I have work to do.
<snip>
>
> Saving time cost a lot more that time. If we could be sure of
> everthing we would only buy car insurance the day we were going to
> be in a wreck, health insurance the day before we got sick, and
> fire insurance the day before the neighborhood kids set your house
> on fire playing with matches.
>
By the way. I do not buy collision insurance on my car. I buy big
liability packages but no collision. The reason for this is that when
I am all done with life I should have a little extra money in my pocket.
The extra money will be about that same amount as what the
insurance company would have made if I had purchased collision
insurance.
>
> The newer drive gets all 8GB data in under 40 minutes. When I do
> ANY work on that - always after hours - I unmount the data drive,
> and then do a full system backup on the OS - time is under 10
> minutes. Had to use that when new serial drivers totally made
> things unworkable.
>
> Have I said, if anything can go wrong it will go wrong?
>
Have I asked you if I dare leave work tonight for fear of falling
objects?
>
> Lots of luck. And may all your bits be verified.
>
> Bill
>
> (Gosh that got to be along post. I hope it helps save someone from
> disaster)
> --
> Bill Vermillion bv @ wjv.com
--
Ken Andrews
> ECC is good. As to checksums? I'm dubious. (Hello Mr Dubious.
> How is Mrs Dubious and all the kids). [for the trivia quiz of the
> day who has any idea where that came from? Just a generic answer
> will do]
Riff-Raff: "... so say goodbye to all this and Hello, Oblivion!"
Audience: "Hi, Oblivion! How's the wife and kids?"
Bill Vermillion
It actually dates back to the 1920s vaudeville. I think it was
either Jones & Hare, (the Happiness boys) or Moran and Mack (The
Two Black Crows).