SW control of HW Data Compression for DAT Tape Drives
wd
A growing number of Linux users is asking for software control to en-
able and disable the hardware data compression that many SCSI tape
drives allow. Unfortunately, there is no single standard way to do
this: AFAIK there are 3 different "standard" ways for controlling HW
compression:
* the SDCA field in the Device Configuration Mode Page is used for
for QIC drives
* the Density Code in the Mode Select Header is used for Exabyte
drives
* the Data Compression Mode Page (0x0F) is used for DAT drives
I am running a DAT drive and implemented the neccessary patches to
the SCSI tape driver and th "mt" command that allow SW control of HW
DC for DAT's. The code for QIC and Exabyte drives has still to be
written - it's really easy, but I can't test it.
Instead of mailing my patches again and again I think I should post
them now.
Well, this is what I did for the DAT drives:
I added code to:
* return minimal and maximal block size with the MTIOCGET command
* enable/disable hardware compression
The first is simple and needs no further explanations.
The second may be device dependend. I have only access to one device
that is capable of hardware compression. This is a HP 35480A DAT
drive. The documentation I have for DDS DAT Drives is from HP and I
am not absolutely sure if the used MODE SELECT page is vendor
dependend or not. The SCSI-II standard does not list it (at least the
fairly old version I have access to), so probably this is not valid
for other drives. All I know for the HP drives is the following:
MODE SELECT Page 0x0F (Data Compression Characteristics Page):
Byte: bits:
| 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
0 | (0) | PageCode (0f) |
1 | pageLength (0Eh) |
2 | DCE | DCC | reserved |
3 | DDE | RED | reserved |
4 | compression Algorithm [20h] |
...
7 | ---"--- (LSB) |
8 | decompression Algorithm [0h] |
...
11 | ---"--- (LSB) |
12 | reserved |
...
15 | reserved |
DCE = DataCompressionEnable <--- 0 or 1 to switch compression OFF or ON
DCC = DataCompressionCapable <-- always 1
DDE = DataDecompressionEnable <--- always 1
RED = ReportExeptionOnDecompression <-- usually 0 (?)
So the data just looks like this:
Switch ON: 0F 0E C0 80 00 00 00 20 00 00 00 00 00 00 00 00
Switch OFF: 0F 0E 40 80 00 00 00 20 00 00 00 00 00 00 00 00
Since I wanted to be able to switch compression on and off by soft-
ware I implemented this, and it works fine for me.
Maybe one should extend my implementation a little bit. The things
that could/should be done include: check if the device is a SCSI-II
device since I use the SCSI-II Mode Select format; issue a Mode Sense
first to check if the drive really supports page 0xF;
The patch is against 1.1.80 kernel sources. (1.1.81 works, too.)
Then I changed some things to the mt command. First I changed it to
try to use the non-rewinding device no matter what the user supplies
as device name [I think it is really annoying to find after half an
hour that "mt -f /dev/rmt0 fsf 20" just leaves you at the beginning
of the tape...]. Then I changed the output format of the "status"
command to include more information. And finally - of cource - I
added the "setcompression" command to switch hardware compression on
or off.
The patch is against the source version that you can find in on
TSX-11 in ALPHA/scsi/mt-st-0.1.tar.gz (12633 Sep 21 1993).
---- Cut Here and feed the following to sh ----
#!/bin/sh
# This is a shell archive (produced by GNU sharutils 4.1).
# To extract the files from this archive, save it to some FILE, remove
# everything before the `!/bin/sh' line above, then type `sh FILE'.
#
# Made on 1995-01-14 15:27 MET by <wd@denx>.
# Source directory was `/tmp'.
#
# Existing files will *not* be overwritten unless `-c' is specified.
#
# This shar contains:
# length mode name
# ------ ---------- ------------------------------------------
# 3082 -rw-r--r-- PATCHES_1.1.80
# 5851 -rw-r--r-- PATCHES_MT
#
touch -am 1231235999 $$.touch >/dev/null 2>&1
if test ! -f 1231235999 && test -f $$.touch; then
shar_touch=touch
else
shar_touch=:
echo
echo 'WARNING: not restoring timestamps. Consider getting and'
echo "installing GNU \`touch', distributed in GNU File Utilities..."
echo
fi
rm -f 1231235999 $$.touch
#
# ============= PATCHES_1.1.80 ==============
if test -f 'PATCHES_1.1.80' && test X"$1" != X"-c"; then
echo 'x - skipping PATCHES_1.1.80 (file already exists)'
else
echo 'x - extracting PATCHES_1.1.80 (text)'
sed 's/^X//' << 'SHAR_EOF' > 'PATCHES_1.1.80' &&
*** linux/drivers/scsi/st.c.ORIG Sun Jan 1 14:44:03 1995
--- linux/drivers/scsi/st.c Fri Jan 13 21:19:12 1995
***************
*** 512,517 ****
--- 512,519 ----
X printk("st%d: Can't read block limits.\n", dev);
X #endif
X }
+ (STp->mt_status)->mt_minblk = STp->min_block;
+ (STp->mt_status)->mt_maxblk = STp->max_block;
X
X SCpnt->sense_buffer[0]=0;
X memset ((void *) &cmd[0], 0, 10);
***************
*** 1509,1514 ****
--- 1511,1538 ----
X ((STp->buffer)->b_data[2] >> 4) & 7);
X }
X #endif
+ break;
+ case MTSETCOMPRESSION:
+ #ifdef DEBUG
+ printk("st%d: Setting compression %s.\n", dev, arg ? "ON" : "OFF");
+ #endif
+ if (STp->dirty || (STp->buffer)->buffer_bytes != 0)
+ return (-EIO); /* Not allowed if data in buffer */
+ cmd[0] = MODE_SELECT; /* MODE SELECT command */
+ cmd[1] = 0x10; /* SCSI-II page format */
+ cmd[4] = datalen = 20; /* 4 header list, 16 page 0F */
+
+ memset((STp->buffer)->b_data, 0, 20);
+ (STp->buffer)->b_data[2] = STp->drv_buffer << 4;
+ (STp->buffer)->b_data[3] = 0; /* no block descriptor length */
+ (STp->buffer)->b_data[4] = 0x0F; /* page code 0F */
+ (STp->buffer)->b_data[5] = 0x0E; /* page length 0E */
+ (STp->buffer)->b_data[6] = 0x40; /* DCC on: Data Compression Capable */
+ if (arg)
+ (STp->buffer)->b_data[6] |= 0x80; /* DCE on: Data Compression Enable */
+ (STp->buffer)->b_data[7] = 0x80; /* DDE on: Data Decompression Enable */
+ /* RED off: Report Error on Decompression */
+ (STp->buffer)->b_data[11] = 0x20; /* compression algorithm = 0x20 */
X break;
X default:
X printk("st%d: Unknown st_ioctl command %x.\n", dev, cmd_in);
*** linux/include/linux/mtio.h.ORIG Thu Nov 3 15:44:04 1994
--- linux/include/linux/mtio.h Fri Jan 13 21:20:41 1995
***************
*** 53,58 ****
--- 53,59 ----
X #define MTFSS 25 /* space forward over setmarks */
X #define MTBSS 26 /* space backward over setmarks */
X #define MTWSM 27 /* write setmarks */
+ #define MTSETCOMPRESSION 28 /* set the drive compression mode */
X
X /* structure for MTIOCGET - mag tape get status command */
X
***************
*** 63,68 ****
--- 64,71 ----
X * number of files not skipped, or
X * number of records not skipped.
X */
+ long mt_minblk; /* minimal blocksize */
+ long mt_maxblk; /* maximal blocksize */
X /* the following registers are device dependent */
X long mt_dsreg; /* status register */
X long mt_gstat; /* generic (device independent) status */
*** linux/net/unix/proc.c.ORIG Mon May 23 07:17:54 1994
--- linux/net/unix/proc.c Fri Jan 13 21:05:57 1995
***************
*** 49,55 ****
X
X len += sprintf(buffer, "Num RefCount Protocol Flags Type St Path\n");
X
! for(i = 0; i < NSOCKETS; i++)
X {
X save_flags(flags);
X cli();
--- 49,55 ----
X
X len += sprintf(buffer, "Num RefCount Protocol Flags Type St Path\n");
X
! for(i = 0; i < NSOCKETS_UNIX; i++)
X {
X save_flags(flags);
X cli();
SHAR_EOF
$shar_touch -am 0114151995 'PATCHES_1.1.80' &&
chmod 0644 'PATCHES_1.1.80' ||
echo 'restore of PATCHES_1.1.80 failed'
shar_count="`wc -c < 'PATCHES_1.1.80'`"
test 3082 -eq "$shar_count" ||
echo "PATCHES_1.1.80: original size 3082, current size $shar_count"
fi
# ============= PATCHES_MT ==============
if test -f 'PATCHES_MT' && test X"$1" != X"-c"; then
echo 'x - skipping PATCHES_MT (file already exists)'
else
echo 'x - extracting PATCHES_MT (text)'
sed 's/^X//' << 'SHAR_EOF' > 'PATCHES_MT' &&
diff -cr mt-st-0.1/mt.1 mt-wd/mt.1
*** mt-st-0.1/mt.1 Sat Aug 7 15:50:44 1993
--- mt-wd/mt.1 Tue Nov 22 17:35:09 1994
***************
*** 90,95 ****
--- 90,103 ----
X The proper value for unbuffered operation is zero and "normal" buffered
X operation one. The meanings of other values can be found in the drive
X documentation or, in case of a SCSI-2 drive, from the SCSI-2 standard.
+ .IP setcompression
+ If
+ .I count
+ is zero, then switch hardware data compression OFF.
+ If
+ .I count
+ is 1 (or any other non-zero decimal number),
+ then switch hardware data compression ON.
X .IP seek
X Seek to the
X .I count
diff -cr mt-st-0.1/mt.c mt-wd/mt.c
*** mt-st-0.1/mt.c Sat Aug 7 15:50:44 1993
--- mt-wd/mt.c Mon Nov 7 20:41:04 1994
***************
*** 47,56 ****
--- 47,59 ----
X * magnetic tape manipulation program
X */
X #include <sys/types.h>
+ #include <sys/stat.h>
X #include <sys/ioctl.h>
X #include <sys/mtio.h>
X #include <fcntl.h>
X #include <stdio.h>
+ #include <string.h>
+ #include <stdlib.h>
X #include <ctype.h>
X
X #define equal(s1,s2) (strcmp(s1, s2) == 0)
***************
*** 81,86 ****
--- 84,92 ----
X { "setblk", MTSETBLK, 1},
X { "setdensity", MTSETDENSITY, 1},
X { "drvbuffer", MTSETDRVBUFFER, 1},
+ #ifdef MTSETCOMPRESSION
+ { "setcompression", MTSETCOMPRESSION, 1},
+ #endif /* MTSETCOMPRESSION */
X { 0 }
X };
X
***************
*** 93,99 ****
X main(argc, argv)
X char **argv;
X {
! char line[80], *getenv();
X register char *cp;
X register struct commands *comp;
X
--- 99,105 ----
X main(argc, argv)
X char **argv;
X {
! char line[80], *getenv(), *nrdev();
X register char *cp;
X register struct commands *comp;
X
***************
*** 116,121 ****
--- 122,131 ----
X fprintf(stderr, "mt: don't grok \"%s\"\n", cp);
X exit(1);
X }
+ if ((tape = nrdev(tape)) == NULL) {
+ fprintf(stderr, "mt: can't find non-rewinding device\n");
+ exit (1);
+ }
X if ((mtfd = open(tape, comp->c_ronly ? O_RDONLY : O_RDWR)) < 0) {
X perror(tape);
X exit(1);
***************
*** 191,196 ****
--- 201,207 ----
X #endif
X #ifdef linux
X { MT_ISSCSI1, "SCSI 1", 0, 0 },
+ { MT_ISSCSI2, "SCSI 2", 0, 0 },
X #endif
X { 0 }
X };
***************
*** 202,221 ****
X register struct mtget *bp;
X {
X register struct tape_desc *mt;
X
X for (mt = tapes; mt->t_type; mt++)
X if (mt->t_type == bp->mt_type)
X break;
X if (mt->t_type == 0) {
! printf("unknown tape drive type (%d)\n", bp->mt_type);
X return;
X }
X printf("%s tape drive, residual=%d\n", mt->t_name, bp->mt_resid);
X printreg("ds", bp->mt_dsreg, mt->t_dsbits);
X printreg("\ner", bp->mt_erreg, mt->t_erbits);
X putchar('\n');
X }
X
X /*
X * Print a register a la the %b format of the kernel's printf
X */
--- 213,273 ----
X register struct mtget *bp;
X {
X register struct tape_desc *mt;
+ #ifdef linux
+ long blocksize;
+ int dens;
+ char *density;
+ char buf[32];
+ #endif
X
X for (mt = tapes; mt->t_type; mt++)
X if (mt->t_type == bp->mt_type)
X break;
X if (mt->t_type == 0) {
! printf("unknown tape drive type (0x%02x)\n", bp->mt_type);
X return;
X }
+ #ifdef linux
+ blocksize = (bp->mt_dsreg & MT_ST_BLKSIZE_MASK) >> MT_ST_BLKSIZE_SHIFT;
+ dens = (bp->mt_dsreg & MT_ST_DENSITY_MASK) >> MT_ST_DENSITY_SHIFT;
+ switch (dens) {
+ case 0x24: density = "DAT DDS2";break;
+ case 0x13: density = "DAT DDS1";break;
+ case 0x11: density = "320/525"; break;
+ case 0x10: density = "150"; break;
+ case 0x0f: density = "120"; break;
+ case 0x05: density = "45/60"; break;
+ case 0x00: density = "default"; break;
+ default : sprintf (buf, "unknown [0x%02x]", dens);
+ density = buf; break;
+ }
+ printf ("%s tape drive, %s, residual=%d, at block %d.\n",
+ mt->t_name,
+ bp->mt_gstat ? "write protected" : "write enabled",
+ bp->mt_resid,
+ bp->mt_blkno);
+ #ifdef MTSETCOMPRESSION
+ printf ("Blocksize = %d (Min = %d, Max = %d), Density = %s, Error register = 0x%x\n",
+ blocksize,
+ bp->mt_minblk,
+ bp->mt_maxblk,
+ density,
+ bp->mt_erreg);
+ #else
+ printf ("Blocksize = %d, Density = %s, Error register = 0x%x\n",
+ blocksize,
+ density,
+ bp->mt_erreg);
+ #endif /* MTSETCOMPRESSION */
+ #else /* ! linux */
X printf("%s tape drive, residual=%d\n", mt->t_name, bp->mt_resid);
X printreg("ds", bp->mt_dsreg, mt->t_dsbits);
X printreg("\ner", bp->mt_erreg, mt->t_erbits);
X putchar('\n');
+ #endif /* linux */
X }
X
+ #ifndef linux
X /*
X * Print a register a la the %b format of the kernel's printf
X */
***************
*** 247,250 ****
--- 299,362 ----
X }
X putchar('>');
X }
+ }
+ #endif /* linux */
+
+ /*
+ * nrdev returns a pointer to a device name for a non-rewinding tape drive
+ * If this pointer is not NULL it was malloc()ed.
+ *
+ * Input is a tape drive device name. If this is a rewinding device
+ * nrdev() tries to generate a name for the corresponding non-rewin-
+ * ding device
+ *
+ * In case of problems NULL is returned
+ */
+
+ #define MT_NONREW 0x80
+
+ char *
+ nrdev (const char *const devname)
+ {
+ char *nrdev, *n, *s, *t;
+ struct stat stb;
+
+ int l = strlen (devname);
+
+ if (stat (devname, &stb))
+ return (NULL);
+
+ if (stb.st_rdev & MT_NONREW) {
+ if ((nrdev = malloc (l+1)) == NULL) {
+ return (NULL);
+ }
+ strcpy (nrdev, devname);
+ return (nrdev);
+ }
+
+ if ((nrdev = malloc (l+2)) == NULL) {
+ return (NULL);
+ }
+
+ /* copy device name until and including last '/' */
+
+ n = (char *)devname;
+ s = strrchr (n, '/');
+ t = nrdev;
+
+ while (s && n <= s)
+ *t++ = *n++;
+
+ /* insert 'n' to get non-rewinding device */
+ *t = 'n';
+ /* copy rest of file name */
+ while ((*++t = *n++) != '\0')
+ /* empty */ ;
+
+ if (stat (nrdev, &stb) || ((stb.st_rdev & MT_NONREW) == 0)) {
+ free (nrdev);
+ return (NULL);
+ }
+
+ return (nrdev);
X }
SHAR_EOF
$shar_touch -am 0114151895 'PATCHES_MT' &&
chmod 0644 'PATCHES_MT' ||
echo 'restore of PATCHES_MT failed'
shar_count="`wc -c < 'PATCHES_MT'`"
test 5851 -eq "$shar_count" ||
echo "PATCHES_MT: original size 5851, current size $shar_count"
fi
exit 0
Wolfgang
Private: (+49)-89-952275 w...@denx.muc.de
Office: (+49)-89-722-41782 w...@uebemc.siemens.de
"I've seen it. It's rubbish." - Marvin the Paranoid Android