Cdrecord data integrity II -- the case for DAO
Christopher R. Carlen
The following transcripts show clearly that using -dao with cdrecord (on
this particular system) is the only way to make it possible to match the
burned image with the source image.
Explanations will be read with interest. Those who can explain in
detail the cause of this receive a chocolate fish.
Explanations involving write speed will be rejected. I can perform
these operations *hundreds* of times at the speeds shown, as I already
have, and the results will always be the same. Also, cdrecord will
always choose a lower speed than what you specify on the command line if
it determines that the disk's labelled maximum speed is less that what
you tell it with the speed= option.
------------------------------------------------------------
11-22-02
I made an .iso image. Notice that the mkisofs command pads the output:
mkisofs -o ~/cd-rw/full.backup.iso -R -v -graft-points -path-list
~/backup.scripts/full.backup.dir.list
[edited]
Total translation table size: 0
Total rockridge attributes bytes: 390241
Total directory bytes: 1173504
Path table size(bytes): 6692
Done with: The File(s) Block(s) 116787
Writing: Ending pad block Start Block 117391
Done with: Ending pad block Block(s) 17
Max brk space used 213c84
117408 extents written (229 Mb)
Then I mounted the .iso twice and diffed the two mount points.
mango:/home/crcarle # mount -o loop cd-rw/full.backup.iso mnt.iso
mango:/home/crcarle # mount -o loop cd-rw/full.backup.iso mnt.iso2
mango:/home/crcarle # diff -r mnt.iso mnt.iso2
Then I burned to a CD-R:
crcarle@mango:~> cdrecord dev=0,0,0 -v speed=24 ~/cd-rw/full.backup.iso
Cdrecord 1.11a05 (i686-suse-linux) Copyright (C) 1995-2001 Jörg Schilling
TOC Type: 1 = CD-ROM
scsidev: '0,0,0'
scsibus: 0 target: 0 lun: 0
Linux sg driver version: 3.2.0
Using libscg version 'schily-0.5'
atapi: 1
Device type : Removable CD-ROM
Version : 0
Response Format: 2
Capabilities :
Vendor_info : 'LITE-ON '
Identifikation : 'LTR-40125S '
Revision : 'ZS0A'
Device seems to be: Generic mmc CD-RW.
Using generic SCSI-3/mmc CD-R driver (mmc_cdr).
Driver flags : SWABAUDIO BURNFREE
Supported modes: TAO PACKET SAO SAO/R96P SAO/R96R RAW/R16 RAW/R96P RAW/R96R
Drive buf size : 1966272 = 1920 KB
FIFO size : 4194304 = 4096 KB
Track 01: data 229 MB
Total size: 263 MB (26:05.46) = 117410 sectors
Lout start: 263 MB (26:07/35) = 117410 sectors
Current Secsize: 2048
ATIP info from disk:
Indicated writing power: 6
Is not unrestricted
Is not erasable
Disk sub type: Medium Type C, low Beta category (C-) (6)
ATIP start of lead in: -11231 (97:32/19)
ATIP start of lead out: 359849 (79:59/74)
Disk type: Short strategy type (Phthalocyanine or similar)
Manuf. index: 27
Manufacturer: Prodisc Technology Inc.
Blocks total: 359849 Blocks current: 359849 Blocks remaining: 242439
Starting to write CD/DVD at speed 24 in write mode for single session.
Last chance to quit, starting real write in 0 seconds. Operation starts.
Waiting for reader process to fill input buffer ... input buffer ready.
BURN-Free is ON.
Performing OPC...
Starting new track at sector: 0
Track 01: 229 of 229 MB written (fifo 100%).
Track 01: Total bytes read/written: 240451584/240451584 (117408 sectors).
Writing time: 78.921s
Fixating...
Fixating time: 17.685s
cdrecord: fifo had 3788 puts and 3788 gets.
cdrecord: fifo was 0 times empty and 1727 times full, min fill was 21%.
crcarle@mango:~> cat /dev/cdrecorder|md5sum
cat: /dev/cdrecorder: Input/output error
5398175e9c8f4a6bf88df1c77bd6bd3c -
crcarle@mango:~> cat cd-rw/full.backup.iso|md5sum
0bc35f013f6743270e8f627440d1879a -
The md5sums don't match. Let's try readcd:
mango:/home/crcarle # readcd dev=0,0,0 f=cd-rw/readback.iso
Capacity: 117410 Blocks = 234820 kBytes = 229 MBytes = 240 prMB
Sectorsize: 2048 Bytes
Copy from SCSI (0,0,0) disk to file 'cd-rw/readback.iso'
end: 117410
readcd: Input/output error. read_g1: scsi sendcmd: no error
CDB: 28 00 00 01 CA 80 00 00 22 00
status: 0x2 (CHECK CONDITION)
Sense Bytes: 70 00 03 00 01 CA A0 0A 00 00 00 00 11 00 00 00
Sense Key: 0x3 Medium Error, Segment 0
Sense Code: 0x11 Qual 0x00 (unrecovered read error) Fru 0x0
Sense flags: Blk 117408 (not valid)
cmd finished after 5.956s timeout 40s
readcd: Input/output error. Cannot read source disk
readcd: Retrying from sector 117376.
.................................~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~+~~~-~~~
readcd: Input/output error. Error on sector 117408 not corrected. Total
of 1 errors.
Time total: 501.077sec
Read 234752.00 kB at 468.5 kB/sec.
Interesting. Readcd has problems. Let's mount the CD and diff against
the source image:
mango:/home/crcarle # mount /dev/cdrecorder mnt.cd
mount: block device /dev/cdrecorder is write-protected, mounting read-only
mango:/home/crcarle # mount -o loop cd-rw/full.backup.iso mnt.iso
mango:/home/crcarle # diff -r mnt.cd mnt.iso
diff: mnt.cd/eagle3/mylib.old/eagle-lbr: No such file or directory
diff: mnt.iso/eagle3/mylib.old/eagle-lbr: No such file or directory
[other missing symlink target errors deleted]
mango:/home/crcarle #
Thus, all the files are OK.
Finally, the cmp -l command fails:
crcarle@mango:~> cmp -l /dev/cdrecorder cd-rw/full.backup.iso
cmp: /dev/cdrecorder: Input/output error
Notice that the sector where readcd chokes is 117408, which is the first
sector after the last one padded by mkisofs. Yet when mkisofs starts it
says the disk capacity is 117410 blocks. The man page for mkisofs
mentions read-ahead bugs in Linux. All the disk device reads have
failed with IO errors, and they *always* do when I don't use -dao, and
the man page seems to predict this. It says that -dao will avoid this.
Thus it might actually be more peculiar why one would succeed at
reading a CD burned in TAO mode, than why one might fail.
From man mkisofs:
-pad Pad the end of the ISO9660 by 16 sectors (32kB). If
the total size then is not a multiple of 16 sec
tors, the needed number of sectors is added. If
the option -B is used, then there is a second
padding at the end of the boot partitions.
The padding is needed as many operating systems
(e.g. Linux) implement read ahead bugs in their
filesystem I/O. These bugs result in read errors on
one or more files that are located at the end of a
track. They are usually present when the CD is
written in Track at Once mode or when the disk is
written as mixed mode CD where an audio track fol
lows the data track.
To avoid problems with I/O error on the last file
on the filesystem, the -pad option has been made
the default.
Now let's do it with -dao (on a CD-RW):
crcarle@mango:~> cdrecord dev=0,0,0 -v blank=fast speed=10 -dao
~/cd-rw/full.backup.iso
Cdrecord 1.11a05 (i686-suse-linux) Copyright (C) 1995-2001 Jörg Schilling
TOC Type: 1 = CD-ROM
scsidev: '0,0,0'
scsibus: 0 target: 0 lun: 0
Linux sg driver version: 3.2.0
Using libscg version 'schily-0.5'
atapi: 1
Device type : Removable CD-ROM
Version : 0
Response Format: 2
Capabilities :
Vendor_info : 'LITE-ON '
Identifikation : 'LTR-40125S '
Revision : 'ZS0A'
Device seems to be: Generic mmc CD-RW.
Using generic SCSI-3/mmc CD-R driver (mmc_cdr).
Driver flags : SWABAUDIO BURNFREE
Supported modes: TAO PACKET SAO SAO/R96P SAO/R96R RAW/R16 RAW/R96P RAW/R96R
Drive buf size : 1966272 = 1920 KB
FIFO size : 4194304 = 4096 KB
Track 01: data 229 MB
Total size: 263 MB (26:05.44) = 117408 sectors
Lout start: 263 MB (26:07/33) = 117408 sectors
Current Secsize: 2048
ATIP info from disk:
Indicated writing power: 2
Reference speed: 6
Is not unrestricted
Is erasable
Disk sub type: High speed Rewritable (CAV) media (1)
ATIP start of lead in: -11077 (97:34/23)
ATIP start of lead out: 336075 (74:43/00)
speed low: 4 speed high: 8
power mult factor: 2 6
recommended erase/write power: 5
A2 values: 14 A4 4A
Disk type: Phase change
Manuf. index: 11
Manufacturer: Mitsubishi Chemical Corporation
Trying to clear drive status.
cdrecord: Drive needs to reload the media to return to proper status.
Blocks total: 1166730 Blocks current: 1166730 Blocks remaining: 1049322
Starting to write CD/DVD at speed 10 in write mode for single session.
Last chance to quit, starting real write in 0 seconds. Operation starts.
Waiting for reader process to fill input buffer ... input buffer ready.
Performing OPC...
Blanking PMA, TOC, pregap
Blanking time: 18.557s
BURN-Free is ON.
Performing OPC...
Sending CUE sheet...
Writing pregap for track 1 at -150
Starting new track at sector: 0
Track 01: 229 of 229 MB written (fifo 100%).
Track 01: Total bytes read/written: 240451584/240451584 (117408 sectors).
Writing time: 172.155s
Fixating...
Fixating time: 10.088s
cdrecord: fifo had 3788 puts and 3788 gets.
cdrecord: fifo was 0 times empty and 3704 times full, min fill was 87%.
crcarle@mango:~> cat /dev/cdrecorder|md5sum
0bc35f013f6743270e8f627440d1879a -
crcarle@mango:~> cat cd-rw/full.backup.iso|md5sum
0bc35f013f6743270e8f627440d1879a -
mango:/home/crcarle # readcd dev=0,0,0 f=cd-rw/readback-dao.iso
Capacity: 117408 Blocks = 234816 kBytes = 229 MBytes = 240 prMB
Sectorsize: 2048 Bytes
Copy from SCSI (0,0,0) disk to file 'cd-rw/readback-dao.iso'
end: 117408
addr: 117408 cnt: 328
Time total: 87.349sec
Read 234816.00 kB at 2688.3 kB/sec.
Finally:
crcarle@mango:~> ll cd-rw/full.backup.iso
-rw-r--r-- 1 crcarle users 240451584 2002-11-22 12:04
cd-rw/full.backup.iso
crcarle@mango:~> ll cd-rw/readback.iso
-rw-r--r-- 1 root root 240386048 2002-11-22 12:47
cd-rw/readback.iso
crcarle@mango:~> ll cd-rw/readback-dao.iso
-rw-r--r-- 1 root root 240451584 2002-11-22 15:05
cd-rw/readback-dao.iso
crcarle@mango:~> diff cd-rw/full.backup.iso cd-rw/readback-dao.iso
crcarle@mango:~>
Now who wants to tell me that -dao is not needed? Ok, I suppose it
isn't because my files were OK without it. But what is the most logical
way? I'd be very hard to convince that the logical way is the one that
produces IO errors and an inability to read the same image back as the
source.
Note, there is a great likelyhood here that there are some subtle things
going on in different versions of cdrecord, and perhaps mkisofs, and
perhaps even between builds, due to configuration options.
My next experiments will focus on compiling cdrecord myself, and seeing
what those options might be, and what happens.
Thanks for your input.
Good day.
--
____________________________________
Christopher R. Carlen
Principal Laser/Optical Technologist
Sandia National Laboratories CA USA
crc...@sandia.gov
Neal P. Murphy
> Hi:
>
> The following transcripts show clearly that using -dao with cdrecord (on
> this particular system) is the only way to make it possible to match the
> burned image with the source image.
>
> Explanations will be read with interest. Those who can explain in
> detail the cause of this receive a chocolate fish.
>
It would appear that the actual burned size is 32 2KB sectors short.
Just for grins, try:
cp ~/cd-rw/full.backup.iso test.iso
dd if=/dev/zero bs=2048 count=32 >> test.iso
and burn test.iso onto a CD without using -dao.
After it is written, read it back:
dd if=/dev/sr0 bs=2048 count=117408 >test.iso2
where 117408 is the size of your original ISO image.
Then do a compare of both:
diff ~/cd-rw/full.backup.iso test.iso2
If they are the same, you now have *one* strong clue.
In the interest of saving time, you could try making a few short ISO
images, say 128KB, 1MB, 5MB and 12MB and see if the burned size is
consistently 32 2KB sectors short.
With only the data you have presented, it almost appears that without -dao,
cdrecord is not recording the last 32 2K blocks. If so, you should have
a fair idea where to look for the problem.
Fest3er
Adam
Whenever I use readcd to get an ISO image from a CD, it always screws up
towards the end. Here's an example.
I put a bunch of files of data in a directory along with an md5sum file for
all the files. Then I used mkisofs to created n image called test.iso and
used cdrecord to burn it onto a CD. When I mount the CD, the checksum file
is identical to the one on my hard drive and md5sum -c sums says they are
all OK.
So I want to use readcd to get an ISO image of the CD and compare it to the
one I made the CD from.
/test/RUN/NEW $ readcd dev=0,0,0 f=test.test.iso
Capacity: 58754 Blocks = 117508 kBytes = 114 MBytes = 120 prMB
Sectorsize: 2048 Bytes
Copy from SCSI (0,0,0) disk to file 'test.test.iso'
end: 58754
readcd: Input/output error. read_g1: scsi sendcmd: no error
CDB: 28 00 00 00 E5 80 00 00 02 00
status: 0x2 (CHECK CONDITION)
Sense Bytes: 70 00 03 00 00 E5 80 0A 00 00 00 00 11 00 00 00
Sense Key: 0x3 Medium Error, Segment 0
Sense Code: 0x11 Qual 0x00 (unrecovered read error) Fru 0x0
Sense flags: Blk 58752 (not valid)
cmd finished after 4.047s timeout 40s
readcd: Input/output error. Cannot read source disk
readcd: Retrying from sector 58752.
..
Every time I use readcd I get this sort of thing. I've left it alone and
found, 5 minutes later, a bunch of . and a few error messages. In this case
I killed the readcd process from another terminal window. But lo and
behold, the ISO image obtained is correct!
/test/RUN/NEW $ md5sum *.iso
a0dbbb0c75f1efa2d69be4cbf2830cf7 test.iso
a0dbbb0c75f1efa2d69be4cbf2830cf7 test.test.iso
/test/RUN/NEW $ diff *.iso
/test/RUN/NEW $
So why does readcd report errors at the end?
-- Adam
Douglas Mayne
> (condensed)
> Hi:
>
> The following transcripts show clearly that using -dao with cdrecord (on
>
> :
> :
>
> Then I burned to a CD-R:
>
> crcarle@mango:~> cdrecord dev=0,0,0 -v speed=24 ~/cd-rw/full.backup.iso
> Thanks for your input.
>
> Good day.
>
>
the CD file tools needed to be updated. From that update, my cdrecord
reports:
Cdrecord 1.11a23 (i686-pc-linux-gnu) Copyright (C) 1995-2002 Jörg
Schilling
The Freshmeat homepage for CDRecord:
http://freshmeat.net/projects/cdrecord/ has a table with the most recent
version at 1.11a40 which claims "major bug fixes." Anyway, just a thought.
Also, while checking this out, I noticed xcdroast 0.98alpha11 has just
been released. This package requires cdrecord 1.11a34 (minimum), with
1.11a40 (recommended).
Juergen Pfann
>
> 3. cdrecord -v -dao speed=20 bak20021128.iso
> (I'd never used -dao before because I didn't know of any reason to do so)
>
There are plenty; see below.
> (...)
> /shared $ readcd dev=0,0,0 f=bak20021128.test.iso
> (...)
> Time total: 205.061sec
> Read 545152.00 kB at 2658.5 kB/sec.
>
> Success!
>
> 6. bak20021128.iso and bak20021128.test.iso have the same md5sum;
>
> 7. just to be sure "diff bak20021128.iso bak20021128.test.iso" says they
> are identical.
>
> Conclusion: without -dao on cdrecord, readcd does not work.
> _But_ without -dao the files were always OK anyway.
>
Well, one might say it is a matter of point of view, whether
you say "Isn't this some sort of bug in readcd?"
My POV is different: "normal" audio and data CDs _should_
always be written in DAO mode, in order to be fully RedBook
compliant ;-)). (Well if your burner does support DAO, that
is. But that should be true for 95% of modern burners; some
older models, like my "secondary" Teac CD-R55S, don't for ex.)
I'm no expert on this I admit, but the cause for this "error"
using readcd on TAO disks is not an issue of readcd itself,
but more one of the Linux generic SCSI driver if I understand it
correctly.
The most important point for me, however, is: if I want to make
a 1:1 copy of a DAO-written original CD, I'll simply miss this
goal if I burn the copy in TAO - as you noticed, too.
As a conclusion, I only _omit_ the -dao option in very few
special cases, i.e. multisession CDs, or copies of disks
that were burnt in TAO mode before...
Juergen
Adam
> Well, one might say it is a matter of point of view, whether
> you say "Isn't this some sort of bug in readcd?"
>
> My POV is different: "normal" audio and data CDs _should_
> always be written in DAO mode, in order to be fully RedBook
> compliant ;-)). (Well if your burner does support DAO, that
> is. But that should be true for 95% of modern burners; some
> older models, like my "secondary" Teac CD-R55S, don't for ex.)
>
> I'm no expert on this I admit, but the cause for this "error"
> using readcd on TAO disks is not an issue of readcd itself,
> but more one of the Linux generic SCSI driver if I understand it
> correctly.
>
> The most important point for me, however, is: if I want to make
> a 1:1 copy of a DAO-written original CD, I'll simply miss this
> goal if I burn the copy in TAO - as you noticed, too.
>
> As a conclusion, I only _omit_ the -dao option in very few
> special cases, i.e. multisession CDs, or copies of disks
> that were burnt in TAO mode before...
OK, I'm convinced. I'll use -dao from now on.
What method do you use for copying CDs that were previously burnt in TAO
mode? (e.g. the ones I burnt until now)
Thanks,
Adam
Adam
> The following transcripts show clearly that using -dao with cdrecord (on
> this particular system) is the only way to make it possible to match the
> burned image with the source image.
> Now who wants to tell me that -dao is not needed? Ok, I suppose it
> isn't because my files were OK without it. But what is the most logical
> way? I'd be very hard to convince that the logical way is the one that
> produces IO errors and an inability to read the same image back as the
> source.
Based on my experience and Juergen Pfann's comments (in this thread), I
agree that -dao is "a good thing" and I'm going to use it from now on.
The CD-Writing HOWTO doesn't mention that -dao is needed in order to pull
the ISO image off the CD later. Maybe it should?
It does explain that using readcd is better than using cat or dd directly
on /dev/cdrw, however, because the non-readcd methods do not correct for
dust, scratches, etc.
-- Adam
Juergen Pfann
>
> On Friday 29 November 2002 11:50, Juergen Pfann wrote:
>
>
> OK, I'm convinced. I'll use -dao from now on.
>
> What method do you use for copying CDs that were previously burnt in TAO
> mode? (e.g. the ones I burnt until now)
>
Well, it just depends (TM). I also thought about this / learnt
this, after having burnt several data and audio CDs in TAO, too.
As for the ("old") data CDs, I very seldomly need to copy them.
Let's take an example here: I've got an "archive" CD with
the contents of all my old (HW driver) floppies. As this is
only ~450 MB IIRC, I'd rather integrate the (few) floppies
I got after the first burning. This would be no copy in strict
sense, but a new personal collection. I'd write this in DAO
then, of course. In such a case, I simply mount the CD and
copy the files to HD, add the new file(s), and do a new
"mkisofs".
As for copying audio CDs that already are copies made in TAO,
I usually don't copy them image-wise either, but re-grab
the tracks - I still prefer cdda2wav to cdparanoia -, and burn
them again, this time in DAO.
Of course, this is not a 1:1 copy, too. Hence, in such a case,
freedb regards the TAO-written original and a DAO-written copy as
differing, unfortunately. In an ideal world, all freedb entries
would describe DAO disks - as all commercial audio CDs are
DAO -, BUT many many users already entered their own TAO
copies.
Alternatively, you could just use 'cdrdao copy ...'.
As the program name implies, cdrdao defaults to writing
in DAO mode anyway.
There are the options --tao-source and --tao-source-adjust
to cope with the runout sectors problem.
(Perhaps I should use cdrdao more often, but I'm still
more used to cdrecord, anyway.)
Juergen