need help extracting files from Acorn BBC image

[retrogear]

In the interest of studying gsx80 I have been trying to extract the z80 version
of graphplan from a .dsd image here:

http://www.g7jjf.com/acornz80_disc_images.htm

I contacted Jon Welch about the disk parameters and he sent me a pdf which
contained the format spec:

CP/M Disc Format

Acorn CP/M uses the following double sided disc format:-

80 tracks / surface
10 sectors / track
256 bytes / sector

A double sided disc is regarded by CP/M as a single logical disc with 160 tracks numbered from 1 to 159.
In order to obtain the best disc performance the following logical to physical track mapping is performed.

Logical CP/M Physical
track track
0-79 0-79 (top surface)
80-159 79-0 (bottom surface)

The first 3 tracks on the top surface are reserved for the CP/M system.

The CP/M directory starts at track 3 sector 0 and uses 4K bytes to allow up to 128 directory entries/disc.
This leaves 388K bytes/disc available for user programs and data.

Acorn CP/M uses deblocking to allow the physical disc sector size to be larger than the logical CP/M record
size of 128 bytes. Although a 256 byte sector size is used the effective sector size is 512 bytes as all disc
operations read or write 2 sectors at a time using an appropriate sector skew. The following table defines
the logical record to physical sector relationship.

Logical CP/M Logical disc Physical disc
record (128 bytes) sector (512 bytes) sector (256 bytes)
0 0 0
1 0 0
2 0 1
3 0 1
4 1 4
5 1 4
6 1 5
7 1 5
8 2 8
9 2 8
10 2 9
11 2 9
12 3 2
13 3 2
14 3 3
15 3 3
16 4 6
17 4 6
18 4 7
19 4 7


So it reads the top surface then the bottom. I remember reading somewhere in this newsgroup that another drive does that - Northstar? The image appears to be binary and I am able to read the directory and extract files with this
cpmtools definition:

diskdef bbc
seclen 256
tracks 160
sectrk 20
blocksize 4096
maxdir 128
skew 0
boottrk 3
os 2.2
end

However, the files extracted have alternating blank sectors inside so it must
have something to do with the double sided sequencing of sectors?
Has anyone successfully read an image like this? Is there a cpmtools parameter
that would work? I've tried some utilites specific to Acorn images but it
seems to treat the entire top surface of the disk as one file ?

Larry G

[David Schmidt]

On 2/9/2015 11:22 AM, retrogear wrote:
> Although a 256 byte sector size is used the effective sector size is 512 bytes as all disc
> operations read or write 2 sectors at a time using an appropriate sector skew.

Ok. First red flag - you need a skew defined. From your table
(omitted, above) you likely need something like this:
skewtab 0,1,4,5,8,9,2,3,6,7

> So it reads the top surface then the bottom. I remember reading somewhere in this newsgroup that another drive does that - Northstar? The image appears to be binary and I am able to read the directory and extract files with this
> cpmtools definition:
>
> diskdef bbc
> seclen 256
> tracks 160
> sectrk 20
> blocksize 4096
> maxdir 128
> skew 0
> boottrk 3
> os 2.2
> end

Sector length of 256 bytes times 20 sectors per track times 160 tracks
is twice the data you actually have. :-) You don't really have 20
sectors per track - you have 10. And I'm not convinced your blocksize
is 4k, either. It sure would help if you used a disk image with some
text so you can check if things are looking reasonable or not. A pile
of overlays and .com files are hard to judge if they're sane
(alternating blank sectors notwithstanding).

I played for 0.5 minutes with this:

diskdef bbc
seclen 256
tracks 160

sectrk 10
blocksize 2048
maxdir 128
skewtab 0,1,4,5,8,9,2,3,6,7
boottrk 6
os 2.2
end

Still not correct, but it's closer...

[Udo Munk]

I had a brief look at a dump of the disk and searched for strings like GSX and SYS.
Not included anywhere, this is not a GSX-80 application, no GSX RSX attached,
probably it is using the system graphics native and would run on the Acorn machine
only.

[retrogear]

Yes I believe you're right. I was curious because gsx and driver are on the utilities disk. I would still like to crack the image and see more gsx software running in the the BeebEm emulator. Did any Acorn software run gsx?

Larry

[dottor Piergiorgio M. d' Errico]

Il 09/02/2015 19:20, David Schmidt ha scritto:
> Sector length of 256 bytes times 20 sectors per track times 160 tracks
> is twice the data you actually have. :-) You don't really have 20
> sectors per track - you have 10. And I'm not convinced your blocksize
> is 4k, either. It sure would help if you used a disk image with some
> text so you can check if things are looking reasonable or not. A pile
> of overlays and .com files are hard to judge if they're sane
> (alternating blank sectors notwithstanding).

if one care to notice, one of the diskimage (the very last in the list)
has a very well known (and long enough) text base, the 350-pts Crowther
& Woods Colossal cave, I strongly suspect that should be enough for
figuring very quickly the skewing needed, I suppose.

Best regards from Italy,
dott.Piergiorgio.

[Axel Berger]

retrogear wrote on Mon, 15-02-09 17:22:

> In order to obtain the best disc performance the following logical to
> physical track mapping is performed.

> 0-79 0-79 (top surface)
> 80-159 79-0 (bottom surface)

Why? Everybody else goes by cylinder, i.e. first reads all the tracks
immediately accessible without stepping, then step. A head switch is
electronic and thus immediate, stepping takes time.
Or is it that slow systems need that time anyway and the drive is not
necessarily the speed limiting element?

[Udo Munk]

They implemented an elevator algorithm to minimize stepping distance,
because the next block needed might be on another track anyway. Files
allocating adjanced blocks is initially only, after a while deleting
and creating files they usually get fragmented all over the disk.

[retrogear]

I found the utility cpmfiler.exe on mdfs.net which copies files in and out of Acorn cp/m images. The graphplan image first has to be converted with DSDtoSSD.exe

Graphplan Z80 does not utilize gsx and is proprietary to Acorn hardware

At least now I can try gsx stuff inside the Acorn BBC Z80

Larry

[Johny B Good]

No sane designer since the early 80s would utilise the "Use up side 0
1st, followed by side 1, 2nd" algorithm. However, since the earliest
floppy disks were single sided, I expect such an algorithm would have
simply been a consequence of maintaining backwards compatability of
some sort.

With double sided floppy drives and the first Hard Disk Drives based
on a glorified FDD stepper seeking mechanism using constant sectors
per track where the bits per inch density increased with track count,
the performance issue (electronic switching of heads versus stepping
to the next track) wasn't the only reason for filling the media
cylinder by cylinder from track zero _inwards_.

Track zero (always the outermost track of a 35/40/80 track formatted
floppy disk) being the one guaranteed locatable origin for data
retrieval was chosen for critical FS metadata storage because that was
the most reliable portion of any such disk as a consequence of having
the lowest bits per inch density, least effected by imperfections in
the medium.

It's always a good idea to start using up the most reliable portions
of the disk medium first before being forced to resort to the least
reliable portion (innermost track 34 or 39 or 79 of a floppy disk).

Short of using a set of floppies to store a large data archive, with
any luck, you might never ever have to trouble the innermost sectors
of your software / user data / distribution media thus neatly avoiding
potential problems of corrupted data trying to squeeze your quart pot
of data into a pint pot sized floppy disk.

Filling one side completely before being allowed to start using the
most reliable tracks on the other side of a floppy is not exactly
playing to the media's strengths (more accurately, avoiding the
media's weakness).

The same problem afflicted the early HDDs until a couple of decades
ago when zone bit recording was applied to trade off track capacity
(and data transfer rate performance) on the innermost tracks for
improved reliability by reducing the sectors per track to approximate
a fixed bits per inch density across the whole medium.

The only obvious sign of this compromise being that the SDTR falls
gradually to about half its maximum at the outermost tracks by the
time you reach the final zone on the innermost tracks.

Since the old fixed sectors per track HDD storage system was limited
by a maximum sectors per track requirement to achieve a minimum level
of reliability on the innermost tracks, zone bit recording allowed
more efficient use of the physically longer outermost tracks to effect
an increase in capacity on the same quality of media compared to the
old system of fixed sectors per track.

Such complexities were never applied to the humble floppy, not even
the later 3 1/2 inch HD 1440KB capacity[1] floppy disk drives
popularized by the ubiquitous IBM PC and its clones. The quality of
the media was simply upgraded to render the innermost track storage
more reliable than even that on the outermost tracks of a single sided
8 inch floppy disk using the then best quality media and the
'over-kill' improvement on the rest of the tracks on a 1.4MB floppy
was simply accepted as the price to retain simplicity and still
achieve good reliability from first to last track.

[1] Ignoring other short lived oddities such as the 120MiB floptical,
the only other format option, the 2.88MB EHD, never gained enough
market traction to oust the tried and trusted 1.44MB HD format. That,
like the 120MiB floptical, was simply the victim of "too little too
late".
--
J B Good

[glen herrmannsfeldt]

Johny B Good <johnny...@invalid.ntlworld.com> wrote:
> On Tue, 10 Feb 2015 00:48:00 +0100, Axel_...@b.maus.de (Axel
> Berger) wrote:

>>retrogear wrote on Mon, 15-02-09 17:22:
>>> In order to obtain the best disc performance the following logical to
>>> physical track mapping is performed.
>>> 0-79 0-79 (top surface)
>>> 80-159 79-0 (bottom surface)

>>Why? Everybody else goes by cylinder, i.e. first reads all the tracks
>>immediately accessible without stepping, then step.

> No sane designer since the early 80s would utilise the "Use up side 0
> 1st, followed by side 1, 2nd" algorithm. However, since the earliest
> floppy disks were single sided, I expect such an algorithm would have
> simply been a consequence of maintaining backwards compatability of
> some sort.

It is convenient to allow either single or double side, without any
special tests. As long as there are no tracks on the second side,
everything works out right.

(snip)

> Track zero (always the outermost track of a 35/40/80 track formatted
> floppy disk) being the one guaranteed locatable origin for data
> retrieval was chosen for critical FS metadata storage because that was
> the most reliable portion of any such disk as a consequence of having
> the lowest bits per inch density, least effected by imperfections in
> the medium.

I believe that there are file systems that store the most important
metadata in the center. If you have to access it often, it decreases
the average seek time.

(nsip)

-- glen

[Steve Nickolas]

On Tue, 10 Feb 2015, glen herrmannsfeldt wrote:

> I believe that there are file systems that store the most important
> metadata in the center. If you have to access it often, it decreases
> the average seek time.

Apple ][ DOS 3.3 does this.

-uso.

[Steven Hirsch]

On 02/10/2015 04:02 PM, glen herrmannsfeldt wrote:

>> No sane designer since the early 80s would utilise the "Use up side 0
>> 1st, followed by side 1, 2nd" algorithm. However, since the earliest
>> floppy disks were single sided, I expect such an algorithm would have
>> simply been a consequence of maintaining backwards compatability of
>> some sort.

> It is convenient to allow either single or double side, without any
> special tests. As long as there are no tracks on the second side,
> everything works out right.

Exactly. Magnolia Microsystem CP/M for Heath H-89 works in exactly that
manner and for exactly that reason.

[glen herrmannsfeldt]

Steve Nickolas <usot...@buric.co> wrote:

(snip, I wrote)

>> I believe that there are file systems that store the most important
>> metadata in the center. If you have to access it often, it decreases
>> the average seek time.

> Apple ][ DOS 3.3 does this.

The IBM OS/360 disk format allows one to specify the tracks for
the VTOC when the disk is initialized. I don't know if it is/was
more common to put it at the beginning or middle.

-- glen

[ldkr...@gmail.com]

I think some of your definition terms for CPMTOOLS are incorrect. Also,
if you have cpmtools built for libdsk, you also need the libdsk definition updated for ACORN.

New CPMTOOLS definitions require the following information:
diskdef ACORN
seclen 256 #= Sectors xx,256
tracks 160 #= (Cylinders * Sides) = 80*2 = 160
sides outback #= Order of Cylinders
sectrk 10 #= Sectors 10,xxx
blocksize 2048 #= (128*(BLM+1)) = 2048
maxdir 128 #= (DRM+1) = 128
datarate DD #= LOW Density
FM YES #= FM Format
skew 0 #= may be 0 thru 6, or so... try 3 & 0
boottrk 3 #= OFS = 3
os 2.2 #= 2.2, or 2, or 3
end

You may have to try a SKEW of 0, 3, or others to get it perfect. You may also
have to adjust the "Order of Cylinders" to a different value than outback.
Just refer to the cpmtools documentation. This should be a lot closer for the
definition.


Larry

[Jack Strangio]

Axel_...@b.maus.de (Axel Berger) writes:
> retrogear wrote on Mon, 15-02-09 17:22:
> > In order to obtain the best disc performance the following logical to
> > physical track mapping is performed.
> > 0-79 0-79 (top surface)
> > 80-159 79-0 (bottom surface)
>
> Why? Everybody else goes by cylinder, i.e. first reads all the tracks
> immediately accessible without stepping, then step. A head switch is
> electronic and thus immediate, stepping takes time.

North Star used this kind of system also. Its advantage is that the
software doesn't have to be different for single-sided and double-
sided disks or drives. You merely reduce the number of tracks for
single-sided.

Jack.
--
"I'm a home-loving girl. And that's where I wish I was."
"At home ..."
"Loving."
- Laugh-In, 1968

[ldkr...@gmail.com]

.libdskrc updated file for libdsk:

[acn3]
description = Sloger DDCPM on Acorn Z80 second processor
cylinders = 160
sides=outout
heads = 2
sectors = 10
secbase = 0
secsize = 512
datarate = DD
rwgap = 12
fmtgap = 23
fm = N
multitrack = N
skipdeleted = Y

diskdefs updated file for cpmtools:

diskdef acn3
seclen 512 #= Sectors xx,512

tracks 160 #= (Cylinders * Sides) = 80*2 = 160
sides outback #= Order of Cylinders
sectrk 10 #= Sectors 10,xxx

blocksize 4096 #= (128*(BLM+1)) = 2048
maxdir 256 #= (DRM+1) = 128
skewtab 0,2,4,6,8,1,3,5,7,9

datarate DD #= LOW Density

FM NO #= FM Format

skew 0 #= may be 0 thru 6, or so... try 3 & 0
boottrk 3 #= OFS = 3
os 2.2

end

Does this appear to be a valid Directory Listing:
larry@debian:~/Downloads/cpmtools$ cpmls -f acn3 -T raw,acn3 -d cpmutils.img
UNLIST COM : CONVERT COM : CRC COM : CRCKLIST CRC
COPY COM : FORMAT COM : VERIFY COM : TERM COM
STAR COM : DDT COM : STAT COM : ED COM
XSUB COM : ASM COM : SUBMIT COM : DDBBC0 PRL
SYSGEN COM : PIP COM : DUMP COM : LOAD COM
DIP COM : DDBBC1 PRL : ASSIGN SYS : GSX SYS
DDFXHR8 PRL : DDFXLR8 PRL : DDFXLR7 PRL : GENGRAF COM
DDOKI84 PRL : MOVCPM COM
larry@debian:~/Downloads/cpmtools$


Larry

[retrogear]

It's getting real close but not quite. I sent you a pm with the actual binaries for comparison. The utility I found extracts files but doesn't write files into the image. My goal is to be able to write files into the image to test gsx80 on the BeebEm Acorn emulator. So far I have not found an emulator that can run DRI's DrDraw and display on-screen. I'm using Windows.

It was suggested to me that maybe JOYCE will ?

Larry G

[John Elliott]

Not only can JOYCE run DR Draw with the original PCW drivers (720x256
mono), it can also run it with the JOYCE native GSX driver (800x600 colour).

--
John Elliott

Thinks: This is what a nice clean life leads to. Hmm, why did I ever lead one?
-- Bluebottle, in the Goon Show

[retrogear]

Good to know. I'll give it a try.

Larry G

[j...@mdfs.net]

retrogear wrote:
> In the interest of studying gsx80 I have been trying to extract the z80
> version of graphplan from a .dsd image here:
> http://www.g7jjf.com/acornz80_disc_images.htm

(...)

I've been remiss in dipping into co.cpm recently, or I'd have replied
to this earlier. I have the individual files from the Acorn CP/M disks
here: http://mdfs.net/Mirror/Archive/AcornCPM

as well as disk images here: http://mdfs.net/Mirror/Image/AcornCPM

You may also want to dip into StarDot, we've been discussing Acorn Z80
systems recently in reference to the new Matchbox CoProcessor and
getting the DataCentre RAM disks to work with it:
http://www.stardot.org.uk/forums

jgh

[retrogear]

Thanks. During the past month I obtained the Acorn graphplan binaries and have been told and agree with the fact graphplan z80 does not utilize gsx and is heavily modified to run on Acorn bios / hardware so cannot be ported to other emulators without more knowledge than I have. Wiley Moore now has a utility for his zemu emulator that can map foreign cp/m disk images to a drive letter and r/w files. It can move files in/out of the Acorn cp/m images now. Kind of like cpmtools on steroids :)

The stardot forum looks interesting and very active. Thanks for the link.

Larry G

[j...@mdfs.net]

ldkr wrote:
> .libdskrc updated file for libdsk:
> [acn3]
> description = Sloger DDCPM on Acorn Z80 second processor
> cylinders = 160
> sides=outout

If 'sides=outback' means go up side 0, then down side 1, then Slogger
also needs to get sides=outback, as the only difference between
SloggerCPM and AcornCPM is 20 physical sectors of 256 byters per
physical track, and a block of 4096 bytes instead of 2048.

See http://mdfs.net/Docs/Comp/Disk/Format/AcornCPM where I added a
note a few days ago wondering how to specify this.

jgh