Slow DLT tape access with Backup
Malcolm Dunnett
I've installed a DLT 2000 XT on a VMS 7.2 system ( directly connected
to the SCSI port). I seem to be having a problem with performance
when skipping files:
- if I put a bunch of backup savesets on the tape and try to find
one later it takes a long time ( eg with 9 files each 1-3GB in size,
starting at BOT and doing a $BACKUP/LIST of the 9th file takes
about 1/2 hour before the list starts).
- If I do a $SET MAGTAPE/SKIP=FILE=24 to skip over the first 8
savesets ( each saveset being 3 filemarks : 1 after the
HDR labels, 1 after the data and 1 after the EOF labels ) it
takes under 1 minute to position to the start of the 9th
saveset ( as determined by doing a $DUMP MKxxx: at that
point and seeing the HDR1 label for the saveset ).
- If I do the above via a series of $SET MAGT/SKIP=FILE=3
commands it takes about 8 minutes to locate the start of the
9th saveset ( ie approx 1 minute to position between each
saveset ). This is the performance I would expect from BACKUP.
It's obviously unreasonable to expect backup to know how
many savesets to skip to get to the one specified, but it
should be using a "skip by file" between each saveset, giving
an average seek time of about 1 minute/saveset.
SHO DEV/FU MKxxx: claims the drive supports fastskip, and the
SET MAGT/SKIP results appear to bear this out.
All but point 1 seem consistant with the specifications for the
drive. It appears that the BACKUP command is always skipping via
records though ( even if I do a SET MAGT/FILE_SKIP=ALWAYS ).
If I position the tape to the start of the saveset using the SET
MAGTAPE commands as described above BACKUP appears to ignore that
saveset and do a "skip files by records" to the next saveset.
Is it supposed to work this way? Is there some magic required to
get BACKUP to use the skip file operation? It's pretty frustrating
to have this fast tape drive and have to wait potentially hours
to get at the data.
--
=============================================================================
Malcolm Dunnett Malaspina University-College Email: dun...@mala.bc.ca
Information Systems Nanaimo, B.C. CANADA V9R 5S5 Tel: (250)755-8738
Andy Goldstein
>
> I've installed a DLT 2000 XT on a VMS 7.2 system ( directly connected
> to the SCSI port). I seem to be having a problem with performance
> when skipping files:
>
> - if I put a bunch of backup savesets on the tape and try to find
> one later it takes a long time ( eg with 9 files each 1-3GB in size,
> starting at BOT and doing a $BACKUP/LIST of the 9th file takes
> about 1/2 hour before the list starts).
[Summarizing: $SET MAGTAPE/SKIP=FILE=24 takes 1 minute from BOT to the
last file. $SET MAGTAPE/SKIP=FILE=3 8 times takes about 8 minutes.]
The problem is that BACKUP, while it is using the fast skip feature of
the drive, is not as smart as it could be about searching the tape. It
skips one file mark at a time and looks for a HDR1 label at each one.
This was done as a simple way of dealing with an unknown starting tape
position. Also, we were concerned that damage to an otherwise unrelated
part of the tape (i.e., a save set we're skipping over) might cause
BACKUP to miss a tape mark and therefore miss a label set if it was
skipping 3 tape marks at a time. Back when the algorithm was designed it
obviously didn't make any difference. However, faced with a drive that
takes 1 minute to skip to any number of tape marks, it might be a good
idea to revisit this.
> If I position the tape to the start of the saveset using the SET
> MAGTAPE commands as described above BACKUP appears to ignore that
> saveset and do a "skip files by records" to the next saveset.
BACKUP does a skip to the next tape mark before checking the label. If
you manually position the tape one tape mark short of the start of the
desired save set it should find the one you want.
Malcolm Dunnett
Andy Goldstein <Gold...@star.zko.dec.youknowwhere> writes:
>
> The problem is that BACKUP, while it is using the fast skip feature of
> the drive, is not as smart as it could be about searching the tape. It
> skips one file mark at a time and looks for a HDR1 label at each one.
ok, that explains some of what I saw.
>
> BACKUP does a skip to the next tape mark before checking the label. If
> you manually position the tape one tape mark short of the start of the
> desired save set it should find the one you want.
Thanks, that was the missing piece. If I skip 3*n-1 file marks and
then issue the backup command the listing starts right up.
I've cobbled together a bit of DCL to use the skip file function to
find the start of a saveset. Using my test tape I can find the 9th
saveset in about 8 minutes ( checking each header along the way ) or
in about 1 minute if I tell it to skip the first 8 files. The code is
attached in case anyone else is interested ( as-is, use at your own
risk, etc ).
What does backup do with a /VERIFY? Does it rewind the tape and
search from the beginning or does it skip backwards across the file?
(or, as I seem to recall from the 9 track days, does it depend on the
size of the saveset? )
----
$!
$! FIND_SAVESET: Skip to a specified saveset on a tape
$!
$! This procedure uses the skip by file function
$! to locate a specified saveset on a backup tape
$!
$! Parameters p1 Tape drive
$! p2 saveset to find
$! p3 [OPTIONAL] savesets to skip over before
$! starting search ( if known came improve
$! performance significantly )
$!
$
$!
$! Version Check
$!
$ VERSION = F$EXTRACT(1,3,F$GETSYI("VERSION"))
$ IF VERSION .LTS. "7.1"
$ THEN
$ WRITE SYS$OUTPUT "Procedure must run on VMS 7.1 or higher"
$ EXIT
$ ENDIF
$!
$! Get Parameters
$!
$ IF "''P1'" .EQS. "" THEN INQUIRE P1 "Tape drive"
$ IF "''P2'" .EQS. "" THEN INQUIRE P2 "Saveset to find"
$ P2 = F$EDIT(P2,"UPCASE,TRIM")
$
$ if "''P3'" .EQS. "" THEN P3 = 0
$
$ TAPE = "''p1'"
$ SET MAGT/REWIND 'TAPE' ! Make sure we're at BOT
$!
$! Enable fast skip
$! (use MKSET for 7.1, SET MAGTAPE command for 7.2 and up )
$!
$ IF VERSION .EQS. "7.1"
$ THEN
$ MKSET = "$SYS$ETC:MKSET.EXE"
$ MKSET/ALWAYS 'TAPE'
$ ELSE
$ SET MAGT/FAST=ALWAYS 'TAPE'
$ ENDIF
$
$ OPEN/READ TAPE 'TAPE'
$!
$! Get Volume label
$!
$ READ TAPE REC
$ IF F$EXTRACT(0,4,REC) .NES. "VOL1"
$ THEN
$ WRITE SYS$OUTPUT "VOL1 not found, tape is wrong format"
$ GOTO TAPE_DONE
$ ENDIF
$
$ WRITE SYS$OUTPUT "Volume Label: ", F$EDIT(F$EXTRACT(4,17,REC),"TRIM")
$!
$! Skip some savesets if requested
$!
$ IF P3 .GT. 0
$ THEN
$ skipcnt = p3*3
$ SET MAGT/SKIP=FILE='skipcnt' 'TAPE'
$ ENDIF
$!
$! Get a file header
$!
$GET_HDR1:
$
$ READ TAPE REC
$ IF F$EXTRACT(0,4,REC) .NES. "HDR1"
$ THEN
$ WRITE SYS$OUTPUT "HDR1 not found, tape is wrong format"
$ GOTO TAPE_DONE
$ ENDIF
$
$ FILE = F$EDIT(F$EXTRACT(4,17,REC),"TRIM,UPCASE")
$ WRITE SYS$OUTPUT "Found file: ", FILE
$
$ IF FILE .EQS. P2
$ THEN
$ SET MAGT/SKIP=FILE=-2 'TAPE' ! Position to record before HDR1
$ GOTO TAPE_DONE
$ ENDIF
$!
$! Position to start of next saveset
$!
$ SET MAGT/SKIP=FILE=3 'TAPE'
$ IF $STATUS .EQ. %X1001827A THEN GOTO TAPE_DONE ! RMS$_EOF
$
$ GOTO GET_HDR1
$
$TAPE_DONE:
$
$ CLOSE TAPE
$
djesys
Andy Goldstein wrote:
> If you manually position the tape one tape mark short of the start of the
> desired save set it should find the one you want.
Thanx for the confirmation! I discovered this using DUMP, and now exploit this in my
company's Disaster Recovery automation. Significant speed improvements were realized
(even on 8mm!) by using SET MAGTAPE/SKIP, COPY to temporary disk files and DCL READs
instead of using BACKUP to locate a specific saveset.
David J. Dachtera
dba DJE Systems
http://home.earthlink.net/~djesys/
Unofficial Affordable OpenVMS Home Page and Message Board:
http://home.earthlink.net/~djesys/vms/soho/
Andy Goldstein
>
> What does backup do with a /VERIFY? Does it rewind the tape and
> search from the beginning or does it skip backwards across the file?
> (or, as I seem to recall from the 9 track days, does it depend on the
> size of the saveset? )
It depends on the size of the saveset and whether you said /REWIND to
begin with. If the save set is smaller than 1000 blocks and you wrote it
/NOREWIND, BACKUP backspaces over the save set. Otherwise it rewinds and
searches forward. Again, this logic is unchanged since it was originally
designed.
I'm reluctant to mess with this logic. There is such diversity in the
performance characteristics of the various tape drives that trying to
optimize this could be a real swamp.
Glenn C. Everhart
never done unless the count of filemarks to skip is greater than
2. There is a considerable bit of extra processing necessary at the
end of each skip by filemarks which can make that method slower
than the older skip by records one. It was determined to leave
the processing to skip by filemarks only for counts of over 2
since most ANSI processing uses either +2, -2, or 32767 and
this meant that the ACP would not be disturbed, nor would timing,
for common MTAACP operations.
If you want to skip faster for one or two eof marks, you'll need
to resort to io$_diagnose, or patch sys$mkdriver.
The faster skip speedup is most dramatic with DLT...can be factor
of 100 faster...but the speedup exists with some other tapes as
well. I believe firmware in many 4mm tapes treats skips of 32000+
records as skip by filemarks, though, so these may not have any
visible difference. On some tape technologies skipping by filemark
can be slower because the system must check that the tape is
positioned as doc'd afterwards. Recall the docs say the tape
gets left between the last 2 EOF marks if there are such on
tape...
Malcolm Dunnett wrote:
>
> In article <3742A455...@star.zko.dec.youknowwhere>,
> Andy Goldstein <Gold...@star.zko.dec.youknowwhere> writes:
>
> >
> > The problem is that BACKUP, while it is using the fast skip feature of
> > the drive, is not as smart as it could be about searching the tape. It
> > skips one file mark at a time and looks for a HDR1 label at each one.
>
> ok, that explains some of what I saw.
>
> >
> > BACKUP does a skip to the next tape mark before checking the label. If
> > you manually position the tape one tape mark short of the start of the
> > desired save set it should find the one you want.
>
> Thanks, that was the missing piece. If I skip 3*n-1 file marks and
> then issue the backup command the listing starts right up.
>
> I've cobbled together a bit of DCL to use the skip file function to
> find the start of a saveset. Using my test tape I can find the 9th
> saveset in about 8 minutes ( checking each header along the way ) or
> in about 1 minute if I tell it to skip the first 8 files. The code is
> attached in case anyone else is interested ( as-is, use at your own
> risk, etc ).
>
> What does backup do with a /VERIFY? Does it rewind the tape and
> search from the beginning or does it skip backwards across the file?
> (or, as I seem to recall from the 9 track days, does it depend on the
> size of the saveset? )
>
Malcolm Dunnett
> Be aware too that the "faster" skip (using SCSI skip by files) is
> never done unless the count of filemarks to skip is greater than
> 2.
So then if ( as Andy says ) backup skips one filemark at at time
to check for labels it would never take advantage of the fileskip
function, right?
>
> If you want to skip faster for one or two eof marks, you'll need
> to resort to io$_diagnose, or patch sys$mkdriver.
>
I don't think I need this for my purposes, I want to skip 3 filemarks
at a time, except for the final repositioning at the saveset, which
should be a short movement anyway, since its just backing up over the
EOF labels.
> The faster skip speedup is most dramatic with DLT...can be factor
> of 100 faster...but the speedup exists with some other tapes as
> well. I believe firmware in many 4mm tapes treats skips of 32000+
> records as skip by filemarks, though, so these may not have any
> visible difference.
That could explain why I never saw this problem with the 4mm HP
drives we'd been using, the tape drive firmware was probably
automagically translating the commands for me.
Malcolm Dunnett
>
> It depends on the size of the saveset and whether you said /REWIND to
> begin with. If the save set is smaller than 1000 blocks and you wrote it
> /NOREWIND, BACKUP backspaces over the save set. Otherwise it rewinds and
> searches forward. Again, this logic is unchanged since it was originally
> designed.
>
backups first then do a compare pass in the same order - kind of like
we used to have to do with Exabyte 8200s - when the "skip file" operation
could take hours ( EXB 8200s skipped records at read/write speed :-( )