backup/restore using disk instead of tape
Brian K. White
floppy see more than one disks using a btld floppy and a generic
unix.install kernel.
suppose a raid card has a few logical drives configured and that I
will know how to configure them from scratch in a new cards bios.
logical 0 - ha=0 bus=0 target=0 lun=0
this one will get restored to. it will need to get filesystems set up
and a ctar backup restored onto it, including /stand / /u etc...
logical 1 - ha=0 bus=0 target=0 lun=1
another drive that can be dealt with easily after / is restored,
doesn't *have* to be done during initial restore.
logical 2 - ha=0 bus=0 target=0 lun=2
this is a drive that has been used in place of a tape. a ctar (or
other supertar) has written a tar directly to it (no filesystem)
they are all on the same "amird" controller, and if you were restoring
to new hardware after a crash, you can assume another single raid
controller, possibly not the same model or manufacturer, but assume a
btld disk is ready.
I want to be able to boot a floppy and give boot: commands like
boot: restart link=amird Sdsk=amird(0,0,0,0)
except I think I need more than one Sdsk argument for the 0,0,0,2 disk
how do I do that? and what would the /dev nodes need to be created on
the floppy? I could copy them from the running system, but what if on
the new hardware the "tar" disk ends up being some other
bus/target/lun like 0,1,0,0 instead of 0,0,0,2, I'd need to know how
to mknod the right numbers manually while booted to the floppy. assume
that the tar and root disksare always on the same controller, whatever
it is, and that the root is always 0,0,0,0, but that the tar disk
could be some other bus/target/lun. I would only ever need to know how
to create the /dev node for the whole raw disk, such as /dev/dsk/2s0
the idea is to use an sca2 hot-swap drive bay as admittedly delicate
but fast and big and flexable removable backup devices.
for the size of this server, we considered the equivalent tape
solution, and decided we are ok making the leap to just having to
treat the hd's with more care than tapes. we also understand the
limited (approx 500 cycles) mechanical life of the sca2 connectors
themselves and are prepared/ok with that.
I just need to work up some directions for the on-site guy to follow
if he ever needed to retore to the same, or possibly new hardware and
I wasn't available (I'm not local to them anyways, even if I was
reached by phone at the time.)
They have ctar & airbag now, and that has always been fine, using
tapes, I've restored boxes to completely new and un-similar hardware
using it myself several times. but the tape device was never odd like
this. it was either the same, relative to the controller, or it was
target 0 on it's own card, either way the boot: command line was
simple.
this however is a new situation.
Brian K. White
> what would be the boot: command line to allow a airbag (or other) boot
> floppy see more than one disks using a btld floppy and a generic
> unix.install kernel.
>
>
I successfully wrote a partial backup to a raw disk device on a live
machine, and then sucessfully read the disk on a foreign machine using
only a airbag boot floppy that had a generin unix.install kernel on it
and the btld floppy for the foreign raid card. the "restore" machine
(the same customers old server, now a backup server) has a different
make of raid card, and the address of the "tar" disk is perforce
different than it is on the live machine. I had the on-site tech type
in the following at the boot: prompt, booting from the ctar airbag
boot floppy:
boot: defbootstr link=dpti5 Sdsk=dpti(0,0,0,0) Sdsk=dpti(0,0,4,0)
Sdsk=dpti(0,0,6,0)
only the last Sdsk argument "took", but that was the most important
for this experiment. on the "new" hardware (really the old hardware,
but we are pretending to restore the live machine onto strange new
hardware, to prove that this backup strategy will be effective in the
worst case, so the old server is safe to "restore" onto and
potentially damage the data. so the old server is the pretend "new"
hardware). on the "new" hardware the "tar" disk is driver dpti,
adapter 0, bus 0, target 6, lun 0. this is completely unlike the
originating box where the "tar" disk is driver amird, adapter 0, bus
0, target 0, lun 2.
since I happened to put the 0,0,6,0 Sdsk argument last, that was the
only one that was used, and so although I couldn't really test
restoring, I was able to test the most important part that I really
needed to verify that I could do, which was be able to read a tar from
a raw disk, without any "magic help" from the airbag saving the the
settings from the originating box. that much worked. I captured a ls
-l listing of /dev/*/*s0 on both the old and new boxes after the "tar"
disk had been added, and used that to manually run mknod while booted
from the airbag floppy to create a device and then read a tar from it.
so now all I need is the proper syntax to allow me to specify two or
more Sdsk on the boot: command line.
the bootstring man page offeres no clue. should I maybe just say
Sdsk=dpti(0,0,0,0) Stp=dpti(0,0,6,0) even though it's not really a
tape?
or Sdsk="dpti(0,0,0,0) dpti(0,0,6,0)" ??
Bela Lubkin
> so now all I need is the proper syntax to allow me to specify two or
> more Sdsk on the boot: command line.
>
> the bootstring man page offeres no clue. should I maybe just say
> Sdsk=dpti(0,0,0,0) Stp=dpti(0,0,6,0) even though it's not really a
> tape?
> or Sdsk="dpti(0,0,0,0) dpti(0,0,6,0)" ??
Those won't work; and in fact there is no syntax for what you're trying
to do. At most a single "Sdsk=" argument is parsed.
You can:
- use commercial crash recovery software after all, or
- construct a boot kernel with the necessary host adapter driver(s)
and mscsi entries, or
- attempt to write a script that uses sconf(ADM) to add additional
drives to the running kernel (this is not well supported)
>Bela<
Brian K. White
You insult me! I am using a commercial crash-recovery software thank
you very much, and am perfectly willing to use any other one if it
could do this. All I am not using is a *tape*. In this case it's ctar
+ airbag, I did't select them, but neither have I yet seen a need to
tell my boss he has to switch to something else when he's been using
this for years before I came along and it does the job as well as
anything else as far as I can tell. I've had to restore a few times,
to new/bigger hardware even, and it was fairly easy and worked. what
else do I need? features like being able to fast-seek through a tape
to get a particular file out fast, or putting more than one backup on
a single tape using filemarks, or burning bootable cd's or dvd's are
"neat" and all, but of no real world value to me.
sconf looks like it might be the ticket, thanks.
other questions would be:
* perhaps I could use Sdsk=... Sflp=... and pretend the tar disk is a
floptical?
would the floptical driver work on a hard disk?
* what happens when you have a kernel with a particular scsi driver
and a few disks configured in, and then you boot up on new hardware
with btld for the new scsi card, but don't put any Sdsk= on the boot:
command line?
do the disk devices automatically try to use whatever ha is found? Is
this maybe exactlty what the "auto" keyword, in place of an "ha"
driver name in the mscsi file is for? could I build a kernel that had
several "likely" scsi disks configured, (adapter 0, bus 0, target 0-6,
lun 0-6), with auto in place of the adapter name in mscsi, and then
boot using a btld for the adapter, not specifying any Sdsk on the
boot: line, or, specifying just the root (0,0,0,0) disk, and then be
able to use any disk that fell within my "likely" range of addresses
on the new hardware? (assuming I manually ran mknod to create the /dev
nodes, or had them built into the floppy already.)
I can, if all else fails, just "cheat" and build a boot floppy with
the currently running kernel, but this means that in order to restore,
I need the same model of raid card on the new machine. fine for
restoring to the same box, and fine if the restore happens within a
year or so, but I think a proper backup solution must allow you to
restore onto a completely new box, in case the original hardware is no
longer manufactured. And I have not been able to load btld drivers
using the fully configured kernel on either the new or old boxes, only
using the unix.install kernel. (the airbag utility lets you specify
the kernel that will be built into the floppy)
thanks again for the reminder of sconf. that's probably my answer.
Bill Vermillion
Brian K. White <br...@aljex.com> wrote:
>Bela Lubkin <be...@caldera.com> wrote in message news:<2002111622...@mammoth.ca.caldera.com>...
>> Brian K. White wrote:
>>
>> > so now all I need is the proper syntax to allow me to specify two or
>> > more Sdsk on the boot: command line.
>> >
>> > the bootstring man page offeres no clue. should I maybe just say
>> > Sdsk=dpti(0,0,0,0) Stp=dpti(0,0,6,0) even though it's not really a
>> > tape?
>> > or Sdsk="dpti(0,0,0,0) dpti(0,0,6,0)" ??
>>
>> Those won't work; and in fact there is no syntax for what you're trying
>> to do. At most a single "Sdsk=" argument is parsed.
>>
>> You can:
>>
>> - use commercial crash recovery software after all, or
>>
>> - construct a boot kernel with the necessary host adapter driver(s)
>> and mscsi entries, or
>>
>> - attempt to write a script that uses sconf(ADM) to add additional
>> drives to the running kernel (this is not well supported)
>>
>> >Bela<
>You insult me! I am using a commercial crash-recovery software thank
>you very much, and am perfectly willing to use any other one if it
>could do this. All I am not using is a *tape*.
The problem with not using tape is that it's harder to make
off-site backups. If you use removeable drives they are more
fragile than tape. Try dropping one of each from about four feet.
I had a client who started getting tapes off site ONLY when he
re-read his business interuption insurance policy that said it
would be void for any losses relating to computer >>IF HE DID NOT
HAVE OFF SITE BACKUPS<<.
>In this case it's ctar
>+ airbag, I did't select them, but neither have I yet seen a need to
>tell my boss he has to switch to something else when he's been using
>this for years before I came along and it does the job as well as
>anything else as far as I can tell.
And what about the above scenario. Too often the "we've done it
this way for years" fails badly when something unforseen comes up.
You protect against the obvious - but many things aren't apparent
at first glance.
>I've had to restore a few times,
>to new/bigger hardware even, and it was fairly easy and worked.
And in case of fire where the computer is destroyed how do you load
a new system. Recently there was a story in the trade mags about
an internet monitoring company that had a disatrous fire that
destoyred the two story structure.
The had been making backup on small tape - I got the impression
they were DATs. They had been converting over to AIT for speed
and greater density. The fire struck, and took out their AIT
device and any tapes. The off site tapes were about 1 month old.
Their only luck in the whole operation was that one drive that was
in a machine not totally destroyed was able to have the data
on that recovered by a data retreival company. And that doesn't
come cheap when they remove platters and place them in other
mechanism.
Just something for everyone on the list to think about if they
think having tape backup and not having it off site. Fireproof
safes are designed to protect paper documents but tapes can be
damaged at about 1/2 the temperature the renders paper useless.
And don't forget water damage from the fire department.
Bill
--
Bill Vermillion - bv @ wjv . com
Mike Brown
With Backup Edge you can define a Device "other" and specify
the dev entry to use. You would have to run mkdev hd to
get it into the kernel first, then build the recovery disks.
In single user mode or from a boot floppy you can dd the entire
raw UNIX drive to an identical backup drive, which is then a
complete bootable solution.
You can also use a HW raid controller to produce a image, then
replace the drive with a spare. Most RAID controllers will
rebuild the mirror automatically, and the removed drive may
be a bootable full backup. Always test this assumption.
Given the reliability and recovery capability of a tape
solution what are you trying to save?
Mike
--
Michael Brown
The Kingsway Group
Bela Lubkin
> You insult me! I am using a commercial crash-recovery software thank
> you very much, and am perfectly willing to use any other one if it
> could do this. All I am not using is a *tape*. In this case it's ctar
> + airbag, I did't select them, but neither have I yet seen a need to
> tell my boss he has to switch to something else when he's been using
> this for years before I came along and it does the job as well as
> anything else as far as I can tell. I've had to restore a few times,
> to new/bigger hardware even, and it was fairly easy and worked. what
> else do I need? features like being able to fast-seek through a tape
> to get a particular file out fast, or putting more than one backup on
> a single tape using filemarks, or burning bootable cd's or dvd's are
> "neat" and all, but of no real world value to me.
Ok... I forget your exact scenario, but before we get all weird with
sconf and stuff, maybe you can do something else...
I'm making this up without looking back up-thread. Suppose you have an
old machine with a small drive, and you want to move to a new machine
with a big drive (or even to the same machine with a new big drive as
root). You could:
1. add the new large drive as a secondary drive
2. partition it "whole disk for Unix"
3. divvy it with one division large enough to hold your backup, and
physically at the end of the partition
4. backup to that division (as a raw device)
5. put the drive into the new (or old) machine at the bootable
position (IDE primary/master, or SCSI ID 0, or whatever you can do
with BIOS)
6. use the crash recovery software to restore from that end-of-disk
division onto a set of newly created divisions
For this to work, the new drive has to be twice as big as the old (well,
twice as big as the old data, or big enough for the restored old data +
a compressed copy of it in the bridge division). This shouldn't be a
big deal since you're probably going from a 10GB drive to 180GB or
something like that...
This would still be tricky because you have to convince the crash
recovery software to let you add new divisions to the drive without
trashing the old one. Or, I suppose you could do all the dividing under
the old root -- then you just have to convince the crash recovery
program to use the premade divisions.
> sconf looks like it might be the ticket, thanks.
>
> other questions would be:
>
> * perhaps I could use Sdsk=... Sflp=... and pretend the tar disk is a
> floptical?
> would the floptical driver work on a hard disk?
There's a chance that it would, actually. Not a very good chance, but
worth a try. You can try it without all the weird booting stuff, just
run `mkdev flopti` and point it at the SCSI coordinates of a trashable
hard disk; reboot; try to access it.
Oddly enough, you might also be able to lie that it's a SCSI CD-ROM,
with "Srom=". The Srom driver doesn't support writing, but that would
be OK for your purposes.
In both cases I suspect you'll be foiled by the respective driver
complaining "hey, that's not a SCSI [floptical | CD-ROM]". Chances are
pretty good that if you disabled only that self-check, the drivers would
actually work well enough to access data (probably not well enough for
live use). Such disabling would be a simple patch in the Driver.o --
simple for someone who's into hacking such things, that is.
> * what happens when you have a kernel with a particular scsi driver
> and a few disks configured in, and then you boot up on new hardware
> with btld for the new scsi card, but don't put any Sdsk= on the boot:
> command line?
> do the disk devices automatically try to use whatever ha is found? Is
> this maybe exactlty what the "auto" keyword, in place of an "ha"
> driver name in the mscsi file is for? could I build a kernel that had
> several "likely" scsi disks configured, (adapter 0, bus 0, target 0-6,
> lun 0-6), with auto in place of the adapter name in mscsi, and then
> boot using a btld for the adapter, not specifying any Sdsk on the
> boot: line, or, specifying just the root (0,0,0,0) disk, and then be
> able to use any disk that fell within my "likely" range of addresses
> on the new hardware? (assuming I manually ran mknod to create the /dev
> nodes, or had them built into the floppy already.)
Hmmm, I can't see anything wrong with this idea. "auto" gets filled in
with the first detected host adapter, so you get limited to just one HA
(until you're restored enough to start working on the _real_ kernel
configuration) -- probably not a fatal limitation.
In machines with multiple HA hardware, you can use "disable=fooha,barha"
to disable any drivers that are being detected before the one you need.
If the machine has two adapters using the same driver, there's no way
to prevent one from being recognized; in that case you're forced to
use whichever is recognized first. (You might be able to affect the
recognition order by fooling around in BIOS setup, host adapter BIOS
setup, or by moving cards around in slots.)
The kernel on the install disks we ship has a couple of "auto" host
adapter disks, at least one tape and one CD-ROM. I think it actually
has those on "standard" IDs like 0, 1 for disk, 2 for tape, 5 for
CD-ROM, and actually _does_ use `sconf` to reconfigure them on the fly.
You could do that, or you could preconfigure a lot of likely
coordinates. I'm pretty sure you can safely configure two peripherals
to the same coordinates, e.g.
auto Sdsk 0 2 0 0
auto Srom 0 2 0 0
(/etc/conf/cf.d/mscsi entries) indicating both a disk and a CD-ROM at ID
2, LUN 0 of adapter 0 -- as long as you do not attempt to _access_ both
of them during one system uptime.
> I can, if all else fails, just "cheat" and build a boot floppy with
> the currently running kernel, but this means that in order to restore,
> I need the same model of raid card on the new machine. fine for
> restoring to the same box, and fine if the restore happens within a
> year or so, but I think a proper backup solution must allow you to
> restore onto a completely new box, in case the original hardware is no
> longer manufactured. And I have not been able to load btld drivers
> using the fully configured kernel on either the new or old boxes, only
> using the unix.install kernel. (the airbag utility lets you specify
> the kernel that will be built into the floppy)
>
> thanks again for the reminder of sconf. that's probably my answer.
It's one of my top two choices, the other being the Swiss Army Kernel
with every HBA turned on and a whole bunch of "auto" entries in mscsi...
>Bela<
Brian K. White
they are getting several drives and hot swapping them like tapes.
the sca2 connector is only rated at about 500 cycles, so the back
plane they plug in to will only last about 2 or 3 years assuming we
devote 2 drive bays to the backup drives. this is fine.
It is a leap deciding to give up the ruggedness of tapes, but, they
uderstand the extra care the drives require and are fine with that.
They transport the drives in a padded holder like a camera case, and
will have one or more extra drives besides the number that will be in
live rotation. We actually did not come to this idea in a vacuum.
another $6000 "commercial backup solution" basically only offered this
as the answer for getting the backups off site. That thing didn't end
up working out, but, once they had accepted the idea of removing
drives and all that implies, it occured to me that we might as well
just write backups directly to the disk like it was a tape. I'd done
it using plain tar before in various odd situations where it was the
easiest way to move a lot of data between two boxes or os's, and I
knew ctar like plain tar would try and write to any device you want.
>
> >In this case it's ctar
> >+ airbag, I did't select them, but neither have I yet seen a need to
> >tell my boss he has to switch to something else when he's been using
> >this for years before I came along and it does the job as well as
> >anything else as far as I can tell.
>
> And what about the above scenario. Too often the "we've done it
> this way for years" fails badly when something unforseen comes up.
> You protect against the obvious - but many things aren't apparent
> at first glance.
we've never used disks as a backup solution before. we've always used
ctar is what I said. I was trying to head off all the "Why aren't you
using Zbackup2000?" unless they had some real-world benefit.
There is an advantage to not changing things that do not need to be
changed. Do not refuse to progress, but change that doesn't offer
enough benefit to be worth the hassle... is not worth the hassle. Any
of us in my company can talk any of our customers through almost any
situation relating to backups, because the customers all have the same
backup software installed. _That Is Valuable_ to the customers and to
us.
>
> >I've had to restore a few times,
> >to new/bigger hardware even, and it was fairly easy and worked.
>
> And in case of fire where the computer is destroyed how do you load
> a new system. Recently there was a story in the trade mags about
> an internet monitoring company that had a disatrous fire that
> destoyred the two story structure.
>
> The had been making backup on small tape - I got the impression
> they were DATs. They had been converting over to AIT for speed
> and greater density. The fire struck, and took out their AIT
> device and any tapes. The off site tapes were about 1 month old.
>
> Their only luck in the whole operation was that one drive that was
> in a machine not totally destroyed was able to have the data
> on that recovered by a data retreival company. And that doesn't
> come cheap when they remove platters and place them in other
> mechanism.
>
> Just something for everyone on the list to think about if they
> think having tape backup and not having it off site. Fireproof
> safes are designed to protect paper documents but tapes can be
> damaged at about 1/2 the temperature the renders paper useless.
> And don't forget water damage from the fire department.
yeah, they are taking the disks off site every night.
Just for the record. the disk thing wasn't really my idea. They bought
an expensive network backup product that only offered removable disks
as the means of taking the backups off site. As usual, I was not
cosulted beforehand, but I am sent in to make the best of the mess. :)
I decided the idea of using ordinary supertar to raw disk was
acceptable overall, given that this customer has a full time IT guy
who is very conscientious and understands the delicacy of a hard
drive, who was already all set to do this anyways. By doing this, they
gain a speed and flexability that tape does not offer. Their current
(logical) hard drive space is 140 gig. a compressed backup still only
takes about 40 gig at the moment, but they produce data steadily,
mostly in the form of already-compressed png scanned documents. They
want to have the whole thing backed up at once, every night. This puts
them into the realm of DLT or LTO 100/200 gig tape drives. $5000 for
the drive, $80 - $100 per tape. That's fine, and they would do that if
they had to, but they would not be able to add more hd space without
having to buy a whole new bigger, even more expensive tape drive plus
labor for me to install it. because most of their data won't actually
compress so they will not actually get 200gigs on a 100/200 drive.
Using disks, they can just buy bigger disks. Even now, disks are
available almost twice as large as all their raid arrays put together.
Bill Vermillion
Brian K. White <br...@aljex.com> wrote:
>b...@wjv.comREMOVE (Bill Vermillion) wrote in message news:<H5qvw...@wjv.com>...
>> In article <60bd4c6b.02111...@posting.google.com>,
{Lucretia Deletia yields here might blade here - wnv}
>> The problem with not using tape is that it's harder to make
>> off-site backups. If you use removeable drives they are more
>> fragile than tape. Try dropping one of each from about four feet.
>they are getting several drives and hot swapping them like tapes.
>the sca2 connector is only rated at about 500 cycles, so the back
>plane they plug in to will only last about 2 or 3 years assuming we
>devote 2 drive bays to the backup drives. this is fine.
You certainly have an ambitious plan. I have customers have
at least one tape for each of Mon-Thu - and four Friday tapes, and
optionally a month end tape. You can usually get back to data four
weeks old if someone accidentally deleted items. That would
be a minimu of 8 HDs on that schedule. You have a storage problem
too.
>It is a leap deciding to give up the ruggedness of tapes, but, they
>uderstand the extra care the drives require and are fine with that.
>They transport the drives in a padded holder like a camera case,
It needs more than padding. Cameras tend to be more rugged than HD
devices. Be sure to check the specs on the HD drives you use
for static G loads. If they really want to do this, then
the best thing would be to get the 2.5" drives designated for
lap-tops as most of them will take a 100G shock when turned off.
That is a lot considering the human body will only survive 20Gs.
>and will have one or more extra drives besides the number that
>will be in live rotation.
What will the total number of drives be.
>We actually did not come to this idea in a vacuum. another $6000
>"commercial backup solution" basically only offered this as the
>answer for getting the backups off site.
That's pretty bizarre IMO.
> .... I'd done it using plain tar before in various odd
>situations where it was the easiest way to move a lot of data
>between two boxes or os's, and I knew ctar like plain tar would
>try and write to any device you want.
Tape data is far more transportable than HD data. As long as
you are aware of that and will never need to have the data moved to
another platform that works.
>> And what about the above scenario. Too often the "we've done
>> it this way for years" fails badly when something unforseen
>> comes up. You protect against the obvious - but many things
>> aren't apparent at first glance.
>we've never used disks as a backup solution before. we've always
>used ctar is what I said. I was trying to head off all the "Why
>aren't you using Zbackup2000?" unless they had some real-world
>benefit.
>There is an advantage to not changing things that do not need to
>be changed.
Agree.
>yeah, they are taking the disks off site every night.
>Just for the record. the disk thing wasn't really my idea. They
>bought an expensive network backup product that only offered
>removable disks as the means of taking the backups off site. As
>usual, I was not cosulted beforehand, but I am sent in to make
>the best of the mess. :)
Hm. I guess that depends on what is meant by 'current network
backup'. If it's a plain IP style - eg NAS/NFS then I can see the
limitation as those really address online drive storage. I'd
personally not consider that a backup method - but I'm opinionated.
There are other non-internal methods. Those will be depend upon
the the availability of controller cards for your system. iSCSI
is still not sorted out. There is fibre channel and there is
FireWire. All are fast and all will address extrenal tape drives
or tape changers.
Nothing says you can't have a dedicated link on those but a switch
[and the latest enterprise targets ones can handle the IP
protocols as well as the the storage ones so you can elminiate a
lot fo wiring - but those are expensive]. So you don't have to give
up a disk bay [which you mention below].
>Their current (logical) hard drive space is 140 gig. a
>compressed backup still only takes about 40 gig at the
>moment, but they produce data steadily, mostly in the form of
>already-compressed png scanned documents. They want to have the
>whole thing backed up at once, every night. This puts them into
>the realm of DLT or LTO 100/200 gig tape drives. $5000 for the
>drive, $80 - $100 per tape.
And why not the AIT or VXA solutions. Those are cheaper but offer
a lot of storage. VXA-1 comes in SCSI, Firewire and ATAPI. The
VXA-2 is 80GB native/uncompressed. Ultra SCSI only an lists for
$999. It's basically a DATish concept but with packet writing to
tape.
AIT-3 is 100GB native. That's the latest version. AIT-2 is 50GB.
You mentioned compression and that many are. You will often find
that hardware compression will outperform SW compression. The
only way to be sure is test with your data to see if it offers
enough gain. Tapes are $66 each [Pricewatch price}. Drives are in
the low $3K range. It's a good target when DAT is too small
and LTO is more than you need.
>That's fine, and they would do that if they had to, but they
>would not be able to add more hd space without having to buy
>a whole new bigger, even more expensive tape drive plus labor
>for me to install it. because most of their data won't actually
>compress so they will not actually get 200gigs on a 100/200
>drive.
As above - HW compression in tape drives is usually higher than
than SW in a computer. Labor installation time should not be
that much when you figure it in to the overall scheme of things.
How big and how many disks are in the machine natively.
>Using disks, they can just buy bigger disks. Even now, disks are
>available almost twice as large as all their raid arrays put
>together.
And how large is their raid array?
Brian K. White
> It needs more than padding. Cameras tend to be more rugged than HD
> devices. Be sure to check the specs on the HD drives you use
> for static G loads. If they really want to do this, then
> the best thing would be to get the 2.5" drives designated for
> lap-tops as most of them will take a 100G shock when turned off.
> That is a lot considering the human body will only survive 20Gs.
wow that is a *great* idea. do they make 150 gig scsi laptop drives?
I'd happlily make up sca2 adapters for them if they are even
available.
ok that sounds facetious, but I actually do like the laptop drive idea
because I hadn't thought of it and it would help a lot with
portability and ruggedness.
actually, ... this may be an even better answer than using the scsi
hot swap bays. there are 3 unused ordinary 5 1/4 inch bays and the
motherboard IDE only being used by the cdrom. I could put the other
ide channel on a standard removable ide 5 1/4 inch dirve bay, and buy
extra bays to get the removable shuttle part to mount 2.5 inch drives
in. I could "mount" the 2.5 drives inside the 3.5 inch shuttles by
just packing them with foam on all sides, and the user could just
carry the bare shuttles without any extra padding necessary.
it's dirt cheap to replace the drive bay or a shuttle when the
connector starts to wear out too, unlike the sca2 bays where one side
of the connector is an integral part of a several hundred dollar
drive, and the other connector is part of a back plane that may not be
able to be replaced except by buying a whole new server case. (5u rack
mount w/ dual power and I don't even know how many fans...). Probably
the drive side connector will never wear out because that is not the
spring-loaded side, but still...
maybe I can play the "drives will get bigger" game and get 60 gig
drives now, banking on the hope that by the time the actual compressed
full backup reaches that size, new 2.5 inch drives will be available
at that time in higher capacities. Worse case scenario: they are not
and we switch to putting ordinary ide drives in the shuttles and
padding them externally for transport. which is still a little better
than what I'm contemplating now, because the drives would be cheap
enough that we can have lots more in live rotation, and the wear &
tear on the hot-swap connector is made unimportant by the ease with
which it can simply be replaced when it wears out.
I dislike the idea of using ide though. I don't know how much it would
impact the servers performance. I guess theoretically it shouldn't
matter since the backup should be running mostly by itself anyways.
> >and will have one or more extra drives besides the number that
> >will be in live rotation.
>
> What will the total number of drives be.
unsure as yet. maybe just two at first. (that's all the other device
offered, as-shipped, though they would sell you more, and you had to
get them from them, because they needed to be specially
formatted...raising the cost even higher) we are currently rsyncing
the data to 2 other servers, one is 10 or 15 miles away. probably they
will get 5 for live rotation, and if one is damaged they will just use
the remaining 4 while the replacement is ordered. I don't see a lot of
necessity to have so many copies. For a tape it's a good idea because
tapes are kind of like floppies in that you shouldn't *really* depend
on the media being perfect, because it does suffer wear and loss
sometimes. the drives are fragile, but aside from dropping or zapping
with static, that data is basically 100% effective. (else servers
would be crashing and dying left and right on an hourly basis, and
they are not)
>
> >We actually did not come to this idea in a vacuum. another $6000
> >"commercial backup solution" basically only offered this as the
> >answer for getting the backups off site.
>
> That's pretty bizarre IMO.
yeah, I said as much to, slightly less politely :)
(yet after a while here I am attempting to do about the same thing,
just without the problematical whole extra server just to house the
drives)
>
> > .... I'd done it using plain tar before in various odd
> >situations where it was the easiest way to move a lot of data
> >between two boxes or os's, and I knew ctar like plain tar would
> >try and write to any device you want.
>
> Tape data is far more transportable than HD data. As long as
> you are aware of that and will never need to have the data moved to
> another platform that works.
it would only ever concevably be open server, unixware, linux,
freebsd, or in a extremely abstract parallel univers, solaris. no
problems there.
there is no filesystem or partitions, simply a disk of (super)tar
data.
only windows would be baffled by it.
> And why not the AIT or VXA solutions. Those are cheaper but offer
> a lot of storage. VXA-1 comes in SCSI, Firewire and ATAPI. The
> VXA-2 is 80GB native/uncompressed. Ultra SCSI only an lists for
> $999. It's basically a DATish concept but with packet writing to
> tape.
>
> AIT-3 is 100GB native. That's the latest version. AIT-2 is 50GB.
> You mentioned compression and that many are. You will often find
> that hardware compression will outperform SW compression. The
> only way to be sure is test with your data to see if it offers
> enough gain. Tapes are $66 each [Pricewatch price}. Drives are in
> the low $3K range. It's a good target when DAT is too small
> and LTO is more than you need.
mostly because I didn't know about them. I know dlt and lto are good
technologies, at least if the drive is bought from a good
manufacturer. I do not know as much about every type of tape out
there.
the lto claims to be able to do the whole 200g in 2 hours. that is a
consideration. this is not a 24hour operation, but considering they do
work into the late evening, and certain automated EDI/import/export
procedures start at 4:45 am, and that the users are scattered from
coast to coast, that all adds up to there being not very many
regulalrly dead hours at night. (between 8-9pm on west coast, and
4:45am east coast)
>
> How big and how many disks are in the machine natively.
>
> >Using disks, they can just buy bigger disks. Even now, disks are
> >available almost twice as large as all their raid arrays put
> >together.
>
> And how large is their raid array?
at the moment, a raid-10 (yes 10) array of 4 36 gig drives, yeilding
73 gigs of useable space, plus a raid-0 array of two 73 gig drives
yeilding another 73 gigs. the raid10 is for the / & /u (the OS &
database application) the raid0 is for the scanned documents. probably
the raid10 will not need to be increased for a few years, the raid0
may need to be increased in as little as a year, but probably more
like almost 2.
this is a new server so at the moment they are only using about 50% of
everything (about evenly split too) hmm... they are already at 60% of
the raid0 for scanned documents. may need to recalculate my timetable
after all :)
the plan was to get bigger drives after a year or so when they are
cheaper, and the 140 is big enough to get them that far easily.
getting 6 drives all double the current sizes is kind of expensive
considering they just bought a new dual-p4-xeon 2.2ghz, 66MHz 64bit
pci raid card, etc, the $6000 network backup unit, paid us a bunch of
on-site hours for me to do the install & migration, plus travel &
hotel etc... can't do *everything* you would like to all at once
sometimes, so the drive space is not spectacular at first, it's enough
to go until drives are cheaper and I don't even have to go on site to
migrate to the new drives when the day comes.
Brian K. White
good stuff, thanks bela.
when I have a working tested recipe I will post it back here.
As per the previous post, I may actually try to use 2.5 inch ide disks
mounted in ordinary 5 inch removable bays...
oh wait... I can't do that! what was I thinking before???
ide is not hot swappable just because it is in a removable bay!
plan a proceeds as planned :)
Bill Vermillion
>
>> It needs more than padding. Cameras tend to be more rugged than HD
>> devices. Be sure to check the specs on the HD drives you use
>> for static G loads. If they really want to do this, then
>> the best thing would be to get the 2.5" drives designated for
>> lap-tops as most of them will take a 100G shock when turned off.
>> That is a lot considering the human body will only survive 20Gs.
>wow that is a *great* idea. do they make 150 gig scsi laptop drives?
>I'd happlily make up sca2 adapters for them if they are even
>available.
Largest I've seen is an 80GB somewhere but the 60GB seem to be
available in many places.
>actually, ... this may be an even better answer than using the scsi
>hot swap bays. there are 3 unused ordinary 5 1/4 inch bays and the
>motherboard IDE only being used by the cdrom. I could put the other
>ide channel on a standard removable ide 5 1/4 inch dirve bay, and buy
>extra bays to get the removable shuttle part to mount 2.5 inch drives
>in. I could "mount" the 2.5 drives inside the 3.5 inch shuttles by
>just packing them with foam on all sides, and the user could just
>carry the bare shuttles without any extra padding necessary.
That sounds good because you aren't every going to screw up your
SCA connectors. And the IDE carriers are cheap enough so you would
wouldn't worry about wearing them out. I had the door come off on
one of mine after many insertions but at the cheap price that was
no biggie.
>maybe I can play the "drives will get bigger" game and get 60 gig
>drives now, banking on the hope that by the time the actual compressed
>full backup reaches that size, new 2.5 inch drives will be available
>at that time in higher capacities.
IBM is making some interesting advances [ and IBM seems to make
most of the advances and then get royalites from other
manufacturers]. It will be a year or two more before we get
2.5" drives that are in the 150GB range. [IBM's "microdrive"
uses 1" platter and form factor is the side of small pacakge of
matches. Designed to go inside such things as cameras. 1GB
capactity on their biggest]
>Worse case scenario: they are not
>and we switch to putting ordinary ide drives in the shuttles and
>padding them externally for transport. which is still a little better
>than what I'm contemplating now, because the drives would be cheap
>enough that we can have lots more in live rotation, and the wear &
>tear on the hot-swap connector is made unimportant by the ease with
>which it can simply be replaced when it wears out.
Good thinking.
>> And why not the AIT or VXA solutions. Those are cheaper but offer
>> a lot of storage. VXA-1 comes in SCSI, Firewire and ATAPI. The
>> VXA-2 is 80GB native/uncompressed. Ultra SCSI only an lists for
>> $999. It's basically a DATish concept but with packet writing to
>> tape.
>> AIT-3 is 100GB native. That's the latest version. AIT-2 is 50GB.
>> You mentioned compression and that many are. You will often find
>> that hardware compression will outperform SW compression. The
>> only way to be sure is test with your data to see if it offers
>> enough gain. Tapes are $66 each [Pricewatch price}. Drives are in
>> the low $3K range. It's a good target when DAT is too small
>> and LTO is more than you need.
>mostly because I didn't know about them.
Sony developed them. The cartridge is 'intelligent' in that it has
an embedded chip. The AIT-3 is 3rd generation. 8mm format. Now
there are several manufactures using them.
AIT and the DLT/LTO techologies are competing with each other
for capacity and speed of backup.
>> And how large is their raid array?
>at the moment, a raid-10 (yes 10) array of 4 36 gig drives, yeilding
>73 gigs of useable space, plus a raid-0 array of two 73 gig drives
>yeilding another 73 gigs. the raid10 is for the / & /u (the OS &
>database application) the raid0 is for the scanned documents. probably
>the raid10 will not need to be increased for a few years, the raid0
>may need to be increased in as little as a year, but probably more
>like almost 2.
You could mirror two of the 180GB Barcudda and get the same but
those are still about $1800 each. It seems like not too long ago
it took a roomfull of disk drives to get that capacity.
>getting 6 drives all double the current sizes is kind of expensive
>considering they just bought a new dual-p4-xeon 2.2ghz, 66MHz 64bit
>pci raid card, etc, the $6000 network backup unit, paid us a bunch of
>on-site hours for me to do the install & migration, plus travel &
>hotel etc... can't do *everything* you would like to all at once
>sometimes, so the drive space is not spectacular at first, it's enough
>to go until drives are cheaper and I don't even have to go on site to
>migrate to the new drives when the day comes.
As they need more space the newer technologies will become cheap
and you can then have external drives that are accessible by other
machines - not to be confused with the NAS devices.