FASTCIRC--High-speed circles for HGR graphics

[Michael J. Mahon]

If you've ever wanted to create HGR graphics containing circles of
various sizes, you’ve probably been disappointed with the slow drawing
speed. The straightforward method using sines and cosines can take
several seconds per circle.

I’ve just added a short (374 bytes) subroutine, FASTCIRC, to my website
that draws circles in from 22 to 75 milliseconds, depending on radius.
It’s self-relocating, so you can BLOAD it anywhere convenient.

To use FASTCIRC, the center (Xlo, Xhi, Y) and the radius are POKEd into
locations 6, 7, 8, and 9 and FASTCIRC is CALLed at its load address.

My website has more complete documentation and a FASTCIRC.DSK image
containing FASTCIRC, its Merlin source, and several demo programs to
illustrate its use.

Time to have fun with circles!
--

-michael

NadaNet 3.1 for Apple II parallel computing!
Home page: http://michaeljmahon.com

"The wastebasket is our most important design
tool--and it's seriously underused."

[fadden]

On Friday, October 21, 2022 at 12:01:49 AM UTC-7, Michael J. Mahon wrote:
> If you've ever wanted to create HGR graphics containing circles of
> various sizes, you’ve probably been disappointed with the slow drawing
> speed.

It's nice to know I'm not the only one. :-)

The self-relocation routine has that fun "living on the edge" feel to it. Very nice.

[TRS-90]

> Time to have fun with circles!

> -michael
> Home page: http://michaeljmahon.com

Hi Michael, can I just say a big "thank you" for keeping your website http so folks like me (on a IIgs) can read your post and then go check out the website without switching to modern hardware? It adds so much atmosphere and fun to be able to do that. I even downloaded the .dsk via the IIgs, however I haven't yet figured out how to access .dsk, .po, etc... once downloaded without switching to moden hardware and making an image with CiderPress. But that's besides the point. Great website and it works on "vintage" (haha) hardware!

Matt

[Michael J. Mahon]

Thanks! I’m pretty happy with my “key byte(s)” relocation scheme. I use
it on most programs of few pages, since in a short chunk of code it’s
extremely unlikely that all 256 byte values appear (and impossible in less
than one page of code!).

I actually use a short Applesoft program to histogram the byte values in
the target program and print a list of all the unused values. It’s typical
that there are several runs of consecutive “missing” values, so it’s not
difficult to choose unique values to be used as “keys” to flag addresses
requiring relocation. Setting the assembly ORG then completes the job. ;-)

I haven’t seen this approach used by others, but I’d be surprised if it
hasn’t been.

Of course, if code can be written so that it’s location-independent, that’s
great, but if you need internal subroutines or data, relocation is
necessary.

--
-michael - NadaNet 3.1 and AppleCrate II: http://michaeljmahon.com

[Oliver Schmidt]

Hi Matt,

> [...] however I haven't yet figured out how to access .dsk, .po, etc...

> once downloaded without switching to moden hardware and making an image with CiderPress.

There are (at least) these options:

- http://www.sheppyware.net/apple-ii-software/diskmaker.html

- https://archive.org/details/diskmaker8

-
https://mirrors.apple2.org.za/ground.icaen.uiowa.edu/apple8/Utils/dsk2file.info

Or bypass the image file step altogether with
https://github.com/cc65/ip65/wiki/Wget65

Regards,
Oliver

[Andrew Roughan]

Oliver Schmidt <ol...@web.de> wrote:
> Hi Matt,
>
>> [...] however I haven't yet figured out how to access .dsk, .po, etc...
>> once downloaded without switching to moden hardware and making an image with CiderPress.
>
> There are (at least) these options:

Have you tried MountIt ?
https://www.brutaldeluxe.fr/products/apple2gs/mountit.html

[TRS-90]

"Oliver Schmidt" wrote:

> Hi Matt,

> There are (at least) these options:
> - http://www.sheppyware.net/apple-ii-software/diskmaker.html
> - https://archive.org/details/diskmaker8

> https://mirrors.apple2.org.za/ground.icaen.uiowa.edu/apple8/Utils/dsk2fil

> e.info
>
> Or bypass the image file step altogether with
> https://github.com/cc65/ip65/wiki/Wget65

Thank you, man, Will do. I hope comp.sys.apple2 sticks around for another 30 years. I learn so much here. Hahaha any hope for a favoites page in the 8-bit Contiki browser?

** Proudly posted from an Apple IIgs **

[Oliver Schmidt]

Hi Matt,

> Thank you, man, Will do.

You're welcome :-)

> I hope comp.sys.apple2 sticks around for another 30 years. I learn so much here.

Me too!

> Hahaha any hope for a favoites page in the 8-bit Contiki browser?

Not much memory left for any feature whatsoever. Maybe a prefilled start
URL (aka home page) set to http://frogfind.com

Regards,
Oliver

[Kent Dickey]

In article <SPecnXQkNMtF3s_-...@giganews.com>,

Michael J. Mahon <mjm...@aol.com> wrote:
>If you've ever wanted to create HGR graphics containing circles of
>various sizes, you’ve probably been disappointed with the slow drawing
>speed. The straightforward method using sines and cosines can take
>several seconds per circle.
>
>I’ve just added a short (374 bytes) subroutine, FASTCIRC, to my website
>that draws circles in from 22 to 75 milliseconds, depending on radius.
>It’s self-relocating, so you can BLOAD it anywhere convenient.
>
>To use FASTCIRC, the center (Xlo, Xhi, Y) and the radius are POKEd into
>locations 6, 7, 8, and 9 and FASTCIRC is CALLed at its load address.
>
>My website has more complete documentation and a FASTCIRC.DSK image
>containing FASTCIRC, its Merlin source, and several demo programs to
>illustrate its use.
>
>Time to have fun with circles!
>--
>
>-michael

Very cool little program, it draws fast circles.

The FASTCIRC.dsk image is actually in ProDOS order. Could you rename
the image to FASTCIRC.po?

Kent

[Michael J. Mahon]

Hi, Kent!

I’m glad you like it—it was fun to write!

Interesting that you should ask about the disk image. There are so many
disk images “out there”, many made before .po existed, that almost all
emulators disregard the image name and just examine the first sectors to
determine how to treat it. As a result, I’ve adopted the convention of
naming all 5.25” images .dsk!

I’m fact, I think all the disk images on my site are ProDOS images, but all
are named .dsk—and (so far) you’re the only one to mention it! ;-)

I’m willing to rename it, but then it would be the “inconsistent” one, or I
could change them all (which would be a pain since I refer to them by name
in several places in the text). I figure that if I name one image .po
people will assume that I consider the suffix meaningful, and all the other
images are *not* ProDOS! ;-)

Frankly, I don’t think the Apple II world will ever get the naming of all
the images consistent with their content. (And we haven’t even gotten to
disks that are neither DOS nor ProDOS.) I don’t think the rampant
inconsistency is a source of much confusion, either. Certainly you weren’t
confused. ;-)

Put another way, any of us who deal with disk images or write programs that
deal with them, have already adapted to “misnamed” images and learned to
cope with them, automatically or manually. That’s a useful skill in the
Apple II world. And, frankly, having a single suffix about which no
assumptions should be made is more consistent than trying to retroactively
impose a new standard which will never apply universally.

I originally tried posting .sdk ShrinkIt images, which are universal for
recreating physical disks, but received several inquiries about what they
were and why didn’t they work with emulators! So I switched several years
ago to simply posting disk (.dsk) images. Maybe .woz images will finally
become universal.

“The great thing about standards is that there are so many to choose from.”
And our “standard” solution is to create yet another “to rule them all”.

If my reasoning is faulty, please straighten me out. ;-)

[Oliver Schmidt]

Hi,

> Interesting that you should ask about the disk image. [...]

If there is any "standard" at all, then it's this:

.do: DOS3.3 order
.po: ProDOS8 order
.dsk: Go figure

Regards,
Oliver

[Michael J. Mahon]

Fair enough, though the Apple II world seems to have lots of other image
types, many of which aim(ed) to be more “universal”.

But I stand corrected, and will review my site and adopt the “standard” as
I have the time.

[Oliver Schmidt]

Hi Michael,

>> If there is any "standard" at all, then it's this:
>>
>> .do: DOS3.3 order
>> .po: ProDOS8 order
>> .dsk: Go figure

> But I stand corrected, and will review my site and adopt the “standard” as
> I have the time.

There seems to be a misunderstanding. I do NOT see an issue with .dsk files
having ProDOS ordering!

.do and .po both imply a statement regarding their ordering but .dsk does
NOT imply any ordering - and therefore a .dsk can per definition never have
a wrong ordering.

At least that is my POV which I wanted to express with my last posting.

Regards,
Oliver

[Kent Dickey]

The two main problems with .dsk being "Go figure" is:

1) It makes emulators harder to write. An Apple II emulator is pretty easy
to get started with, and my guess is there are at least 20 freely
available. But: needing to auto-detect .dsk can be quite complex to do well,
and it's one of many features that emulators seem to need that raises the bar
to get new emulators.

2) It's actually not possible to detect PO or DO for an arbitrary .dsk
image. Let me see your code you're using to auto-detect, and I can
create an image which it will fail on. And many games have no OS, so there's
no great way to detect PO or DO of these images.

I think AppleWin's detection algorithm for .dsk is:

- Detect .DO order first. See if the link pointers on Track $11 look
like a DOS 3.3 catalog track for all sectors 0-14, looking at byte
+$02 in each sector. If it is, it's .DO. Then look to see if
blocks 2,3,4,5 on track $00 look like they have the ProDOS links
in the first 4 bytes of each block look valid for DOS 3.3 order.
If so, it's .DO. Otherwise, it's MAYBE .DO, which will be the
choice if nothing else matches.

- Detect .PO order next. Checks DOS 3.3 catalog track first, but only checks
some of the sectors (I'm not sure why, probably a trick of sector
numbers). If it looks valid, it's .PO. Then check blocks
2,3,4,5 for the ProDOS links in the first 4 bytes. If it
matches, it's PO order.

- Anything not detected defaults to .DO order

Note, this can be confused by a file on a ProDOS image in ProDOS order
that is on track $11 (blocks 136-151) and matches the DOS 3.3 pattern it's
looking for. It would be better if AppleWin checked ProDOS directory patterns
first when considering ProDOS order, to help avoid this confusion. And I'm
not picking on AppleWin, it just is the easiest source for me to look at.

Kent

[D Finnigan]

Kent Dickey wrote:
>
> 2) It's actually not possible to detect PO or DO for an arbitrary .dsk
> image. Let me see your code you're using to auto-detect, and I can
> create an image which it will fail on.

But that's not the criteria is it? We don't need to write code that can
detect any arbitrary image which you create solely for the purpose of
failing said code.

The set of all Apple II disks in existence is finite.

--
]DF$
The New Apple II User's Guide:
https://macgui.com/newa2guide/

[D Finnigan]

D Finnigan wrote:
> Kent Dickey wrote:
>>
>> 2) It's actually not possible to detect PO or DO for an arbitrary .dsk
>> image. Let me see your code you're using to auto-detect, and I can
>> create an image which it will fail on.
>
> But that's not the criteria is it? We don't need to write code that can
> detect any arbitrary image which you create solely for the purpose of
> failing said code.
>
> The set of all Apple II disks in existence is finite.
>

When I wrote and tested my Apple II disk format detection code for the Mac
GUI Vault web site, I tested it against however many thousand disk images I
had in the Apple II collection.

A finite set.

[fadden]

On Friday, October 28, 2022 at 12:21:40 PM UTC-7, D Finnigan wrote:
> But that's not the criteria is it? We don't need to write code that can
> detect any arbitrary image which you create solely for the purpose of
> failing said code.
>
> The set of all Apple II disks in existence is finite.

Even if your auto-detection succeeds on 100% of all known DOS and ProDOS (and Pascal and CP/M) disks, you will be unable to auto-detect the layout of disks with a customized operating system, of which there are many. I believe most ".dsk" images are DOS order, which means the odds of success are > 50% on a blind guess... but why guess at all, when giving the image the correct filename extension removes the ambiguity?

[Michael 'AppleWin Debugger Dev']

On Friday, October 28, 2022 at 10:42:32 AM UTC-7, Kent Dickey wrote:

> I think AppleWin's detection algorithm for .dsk is:
> - Detect .DO order first.

> - Detect .PO order next.

> - Anything not detected defaults to .DO order

Yes, that's correct. CDiskImageHelper::Detect() runs through the supported disk types (initialized in the CDiskImageHelper::CDiskImageHelper() ctor) calling each Disk Type's Detect() function.
i.e.
CDoImage::Detect()
CPoImage::Detect()
etc.

> Note, this can be confused by a file on a ProDOS image in ProDOS order
> that is on track $11 (blocks 136-151) and matches the DOS 3.3 pattern it's
> looking for. It would be better if AppleWin checked ProDOS directory patterns
> first when considering ProDOS order, to help avoid this confusion. And I'm
> not picking on AppleWin, it just is the easiest source for me to look at.

Tom (or 4am or maybe qkumba) would know better then me but I don't believe defaulting to ProDOS sector order is a good idea for the (historical) reasons Andy mentioned.

Ideally, someone would do an in-depth analysis of the thousands of images and verify this assumption. In the 80's the number of DOS formatted disks dwarfed the ProDOS format. My gut feeling is that ProDOS didn't really take off in popularity until well into the 2000's when emulators made it trivial to switch. I would also say qkumba converting games to ProDOS also had an impact. And probably John Brooks excellent work that allowed newer version of ProDOS to run on 6502s.

Given the fact that it is not possible to write a 100% fool proof system there will always be edge cases that fail. Whoever is making the initial disk image "should" be properly tagging the disk format. But alas not everyone takes the same pride so we have some (many?) disks that aren't properly tagged and as such need to guess.

Regardless, please open an AppleWin issue so the rest of the team can mull this over and see if it is time to auto-detect a ProDOS volume directory first. Thanks.

Michael

[Kent Dickey]

In article <dog_cow-1...@macgui.com>,

Can you share some of this information? Do you know roughly what is the
percentage of .dsk images that are in ProDOS order? I do know the biggest
irony is DOS 3.3 Master.dsk on Asimov was in ProDOS order.

I think what you're saying is, if someone wants to throw together an Apple II
emulator (say, written in whatever new language is hot this year), before
releasing it they need to:

- Write .dsk auto-detect code. As far as I know, my description of AppleWin's
algorithm is the first public documentation of .dsk sector order
detection. Otherwise, they would need to do a bunch of research to
decide what to do. Note: other disk formats other than DOS 3.3 and
ProDOS are not covered, such as Pascal.

- Download every Apple II .dsk image available on the web.
How should I go about doing this? I am unable to see how to download
all Apple II files from macgui.com, can you provide some help?

- Test their auto-detect code and make sure they don't make any mistakes.

This seems like a pretty high bar, and I'm not sure any existing emulator
meets this requirement.

Kent

[fadden]

On Saturday, October 29, 2022 at 10:17:38 AM UTC-7, Michael 'AppleWin Debugger Dev' wrote:
> My gut feeling is that ProDOS didn't really take off in popularity until well into the 2000's when emulators made it trivial to switch.

Early emulators assumed all non-nibble disk images were in DOS order, because that's what the disk transfer utilities available back then used.

I suspect that what propelled ProDOS-ordered images was the use of ShrinkIt to compress disks, and utilities like NuLib that would unpack them into a ProDOS-ordered file. When using that approach, you'd need to use a second utility to reorder the sectors to DOS order so the image would be accepted by the emulator. Support for both DOS and ProDOS ordering in emulators came later, and I don't think any emulators used auto-detection until CiderPress started doing it (2003).

The ".2img" format was designed to resolve the ambiguity once and for all, but it never really took off.

One of my tests for CiderPress is to run MDC (Multi-Disk Catalog) on a partial mirror of asimov. (I occasionally update a copy here: https://faddensoftware.com/ .) One could instrument MDC to report the number of ambiguous / mis-labeled / unrecognizable images there are, were one sufficiently interested in doing that, but I'm not sure this discussion is really hampered by a lack of data. (FWIW, a simple grep reveals there are 10,411 disk image files with recognizable filesystems named "*.dsk" in my data set, though some of them are larger than 140KB and hence unambiguously ProDOS-ordered.)

[Steve Nickolas]

I consider .DSK to mean DOS order, and if a ProDOS order file is given the
.DSK extension I consider it a mistake.

But having the option to override on open is nice.

-uso.

[Oliver Schmidt]

Hi Kent,

It seems to me that your topic is by now rather far away from where we
(or at least I) started.

But let me first try to avoid potential misunderstandings:

1. Is it great that .DSK images are often DOS3.3 and often ProODS
ordered? No.

Personal note: If I could decide from scratch, I'd have .DSK images
simply being always in the pyhsical sector order. My first
pre-internet emulator worked that way.

2. Is it great that the most popular Apple II disk image type has to
go without any meta data at all. No.

Personal note: If I could decide from scratch, I'd have all disk
images come with extensible meta data.

Okay, now to what used to be the topic at hand:

1. Is the issues with .DSK images noticably reduced by Michael
renaming his .DSK file to .PO? No.

2. Is Michael's .DSK file hard to auto-detect? No.

Just my two cents,
Oliver

[Kent Dickey]

Actually, my thesis is most of these .DSK images in ProDOS sector order are
relatively recent creations (within the last 20 years), and are not historical.
And so asking people who are creating these images now to use .PO for ProDOS
order would achieve my goal.

>2. Is Michael's .DSK file hard to auto-detect? No.
>
>Just my two cents,
>Oliver

What is your suggested algorithm for auto-detecting the sector order of
.dsk images?

Kent

[Kent Dickey]

In article <5cdb9ecc-d234-4fe0...@googlegroups.com>,

Can you describe CiderPress's algorithm for determining sector order of
a .dsk file?

Kent

[fadden]

On Sunday, October 30, 2022 at 11:07:53 AM UTC-7, Kent Dickey wrote:
> Can you describe CiderPress's algorithm for determining sector order of
> a .dsk file?

"Complicated".

It tries DOS, ProDOS/Pascal, CP/M, and physical sector orders. (CP/M ordering is not really a thing, since nobody is creating disk images with CP/M software... but we might as well collect the whole set.) It tests the filesystem for DOS, ProDOS, Pascal, CP/M, RDOS, Gutenberg, and is probably checking for HFS even though I doubt anybody put that on a 5.25" floppy.

The tests for DOS / ProDOS filesystems have to take into account non-5.25" situations, like UNIDOS disks and DOS Master floppies, so there's a bunch of auto-detect code that doesn't really apply to 5.25" disks.

The tricky part is finding the right thresholds for the filesystem check. If you find 15 catalog sectors in the right order on track 17, you're done. If you find 2, you've got the wrong order. If you find 4... could be the wrong order plus some lucky bytes, could be a DOS disk with a custom short catalog track. So you try each order, and come up with a "yes definitely", "no definitely", or "maybe go with this one if you can't find anything better".

The filename extension is used as a hint, but enough files were named the wrong way that CiderPress doesn't pay much attention to it. It does allow the code to try the most likely order first, so if we get a "yes definitely" from a filesystem we can just stop looking. Also, if we can't identify the filesystem, we can open the sector editor with order indicated by the extension.

It Would Be Nice if CiderPress complained about incorrect or ambiguous filename extensions to encourage people to correct them.

cf. https://retrocomputing.stackexchange.com/q/15054/56

[Oliver Schmidt]

Hi Kent,

>What is your suggested algorithm for auto-detecting the sector order of
>.dsk images?

Maybe there's yet another misunderstanding because I wasn't precise
enough: In my POV, the only ProDOS ordered .DSK files acceptable are
ones actually containing ProDOS.

For a ProDOS disk in ProDOS order, the complete 512 bytes ProDOS boot
block is located at offset $000 - in contrast to a ProDOS disk in
DOS33 order having the second 256 bytes of the ProDOS boot block at
offset $E00.

At least up to ProDOS 2.4.x, the ProDOS boot block of all ProDOS
versions has the string "PRODOS " in the second 256 bytes. So
if that string is found at $100-$1FF, the .DSK file contains ProDOS in
ProDOS order.

Every .DSK file that doesn't match this criteria, is to be considered
in DOS33 order.

Regards,
Oliver

PS: If the ProDOS 2.5 boot block doesn't contain that string anymore,
than ProDOS 2.5 disk images MUST be named .PO.

[David Schmidt]

On 10/30/22 5:43 PM, fadden wrote:
> On Sunday, October 30, 2022 at 11:07:53 AM UTC-7, Kent Dickey wrote:
>> Can you describe CiderPress's algorithm for determining sector order of
>> a .dsk file?
>
> "Complicated".

I've mirrored the same algorithm in AppleCommander. It works in all
cases where a person has a reasonable expectation of there being
reasonable data laid down by a reasonable operating system.

Of note - ADTPro (and ADT before it) always has written 140k Disk II
images in DOS order no matter what. Every other disk type writes images
in ProDOS order no matter what.

[D Finnigan]

Kent Dickey wrote:
>
> Can you share some of this information? Do you know roughly what is the
> percentage of .dsk images that are in ProDOS order?

Not off hand, but I can get that if you are really curious.

>
> I think what you're saying is, if someone wants to throw together an Apple
> II
> emulator (say, written in whatever new language is hot this year), before
> releasing it they need to:
>
> - Write .dsk auto-detect code. As far as I know, my description of
> AppleWin's
> algorithm is the first public documentation of .dsk sector order
> detection. Otherwise, they would need to do a bunch of research to
> decide what to do. Note: other disk formats other than DOS 3.3 and
> ProDOS are not covered, such as Pascal.

Or CP/M. I wrote detectors for these formats too, and I can produce a
catalog of their contents programmatically.

>
> - Download every Apple II .dsk image available on the web.
> How should I go about doing this? I am unable to see how to download
> all Apple II files from macgui.com, can you provide some help?

I started at Asimov. I would recommend that as a source for an Apple II
emulator author who wants a big stack of disk images to test against.

>
> - Test their auto-detect code and make sure they don't make any mistakes.
>
> This seems like a pretty high bar, and I'm not sure any existing emulator
> meets this requirement.

Indeed, which is why I've often found using the real thing to be much more
satisfying.

[I am Rob]

> What is your suggested algorithm for auto-detecting the sector order of
> .dsk images?

I imagined this as an auto-detector.

I like the way Sweet16 loads disk images. It makes a best guess as to what order the disk image is and if unable to decide it loads it as a RAW file. And due to this RAW file format, one can load Commodore 64 disk images which are 174,848 bytes compared to Apple II disk image of 143,360 bytes. Which was then easy to convert C64 T/S's to DOS T/S's.

I would assume the best way to auto-detect an image is to load all disk images as RAW data which can be used by an OS that reads/writes blocks. It is just simple math that converts to Track/Sectors.

A Prodos image is identified as having these standard unchanged bytes that would be extremely rare to have duplicated on an image that is just data.

Block #2 - has byte #2 = $03 and byte #4 = $Fx
Block #3 - has byte #2 = $04
Block #4 - has byte #2 = $05


A DOS image has this:

Block #88 (which calculates to Track 17 Sector 0) - bytes #1 & #2 show the Track/Sector start of the first Sector of the directory. Normally shows $11 0F, but modified disks can be anything. I have on occasion changed this to $11 01 and then have Track $11 Sector $01 - byte #1 changed to a $00 to indicate the end of the directory. This allows only 7 file names but frees up the other 14 sectors of the directory for data on Track 17.

But technically, byte #1 of Track $11 Sector 0 should match byte #1 of the directory sector it points to. for example, if byte #1 and #2 of Track $11 Sector 0 is $11 0F, then byte #1 of Track $11 Sector $0F should also be $11, but can also be a zero for end of directory. This is very easy to find if the first directory sector is in the same track as the VTOC track number and since Sector $0F is in the second half of Block #88. From an emulators perspective should be very easy to get and compare byte #1 of the VTOC and byte #1 of the first sector of the directory to determine the disk image is a DOS disk image. But byte #1 of the next T/S pointer would still have to be read in to determine if this disk is DOS order or Prodos order.

PCDOS and HFS are also very easy to determine using this RAW block method, as well as I imagine CP/M would be. I think an emulator should read in the entire track of $0-F and $11-1F into memory, then it should be able to easily identify any disk image that gets loaded.

[Antoine Vignau]

With values in hex...

If I may, block 88 can be changed to getting the value of T1/SB/01. It contains the track of the VTOC. Using it allows to handle VTOC that are not on T11.

Antoine

[I am Rob]

> With values in hex...
>
> If I may, block 88 can be changed to getting the value of T1/SB/01. It contains the track of the VTOC. Using it allows to handle VTOC that are not on T11.
>
> Antoine

Is this part of DOS and is it the same for all versions of DOS? Including DOS3.2 and Third Party DOS's?

[fadden]

DOS: track 17 sector 0 holds the VTOC. Everything else is potentially variable.

ProDOS: block 2 holds the start of the volume directory. Everything else is potentially variable.

Most disks follow the standard pattern: track 17 is a full catalog track, ProDOS has 4 volume directory blocks followed by the bitmap. Detecting "most disks" isn't the interesting part of the problem though. :-)

[I am Rob]

> > > If I may, block 88 can be changed to getting the value of T1/SB/01. It contains the track of the VTOC. Using it allows to handle VTOC that are not on T11.
> > >
> > > Antoine
> > Is this part of DOS and is it the same for all versions of DOS? Including DOS3.2 and Third Party DOS's?
> DOS: track 17 sector 0 holds the VTOC. Everything else is potentially variable.
>
> ProDOS: block 2 holds the start of the volume directory. Everything else is potentially variable.
>
> Most disks follow the standard pattern: track 17 is a full catalog track, ProDOS has 4 volume directory blocks followed by the bitmap. Detecting "most disks" isn't the interesting part of the problem though. :-)

I was taking Antoine's comment of T1/SB/01 as the location of the VTOC within DOS. Unless that was a typing error and was supposed to be T11/SB/01, in which case it has a different meaning.

The value of Track 17 Sector 0 for the VTOC is stored somewhere in Tracks 0-2 which is usually DOS, unless the disk is DOSless. The question I was asking Antoine was, is the VTOC Track/Sector number in the same location for all versions of DOS including third party DOSes that one can use a standard routine to find the VTOC without assuming it is always Track 17.

[fadden]

On Wednesday, November 2, 2022 at 7:09:31 PM UTC-7, gid...@sasktel.net wrote:
> I was taking Antoine's comment of T1/SB/01 as the location of the VTOC within DOS. Unless that was a typing error and was supposed to be T11/SB/01, in which case it has a different meaning.

T1/S11 holds the code loaded into $ac00-acff. It starts with the tail end of a function that initializes some file manager stuff, including the VTOC location (the byte at +$01 in holds the track number). The value would not be useful if the disk weren't bootable. There are likely some disks with a relocated VTOC that would benefit from looking there, though you'd want to test the code around that location to see if it looked like DOS before putting too much faith in what you find.

For DOS 3.2, it would be T2/S1. Custom DOSs often just made a few in-place edits, so standard routines tended to stay in the standard places.

If you wanted to include this in the strategy, it might be best applied as a fallback if T17/S0 doesn't look like a VTOC, and you were about to write the disk off as "unrecognized".

[D Finnigan]

D Finnigan wrote:
> Kent Dickey wrote:
>>
>> Can you share some of this information? Do you know roughly what is the
>> percentage of .dsk images that are in ProDOS order?
>
> Not off hand, but I can get that if you are really curious.
>

Counting ProDOS format disks:

mysql> SELECT COUNT(*) FROM files WHERE file_format = 11;
+----------+
| COUNT(*) |
+----------+
| 2700 |
+----------+
1 row in set (0.10 sec)



Counting ProDOS format disks with file extension .dsk:


SELECT COUNT(*) FROM files WHERE file_format = 11
-> AND file_url LIKE '%.dsk.%';
+----------+
| COUNT(*) |
+----------+
| 2341 |
+----------+
1 row in set (0.10 sec)

[Oliver Schmidt]

Hi David,

> SELECT COUNT(*) FROM files WHERE file_format = 11;
> +----------+

> | 2700 |
> +----------+

> SELECT COUNT(*) FROM files WHERE file_format = 11
> -> AND file_url LIKE '%.dsk.%';
> +----------+

> | 2341 |
> +----------+

Thanks for sharing :-)

The result seems even (much) more clear than I expected: .DSK files with
ProDOS order are simply the de facto standard. Period.

Regards,
Oliver

[fadden]

On Monday, November 7, 2022 at 3:38:07 PM UTC-8, Oliver Schmidt wrote:
> The result seems even (much) more clear than I expected: .DSK files with
> ProDOS order are simply the de facto standard. Period.

The results indicate that 86% of ProDOS disk images are called ".dsk". It does not indicate what percentage of ".dsk" are ProDOS. I expect that's just as easy to pull out... but it doesn't matter. The goal should be to eliminate ambiguity, not clarify its bounds so we can cope with it using best-effort heuristics.

Is this discussion moving in circles at high speed? :-)

[Kent Dickey]

In article <dog_cow-1...@macgui.com>,
D Finnigan <dog...@macgui.com> wrote:

Can you provide some background on what your total images are, and
what it means to be ProDOS format? Using the output from Andy's file list
can only show which disks have a ProDOS filesystem on it, not which have
sectors in ProDOS order.

Kent

[Oliver Schmidt]

Hi Kent,

> Using the output from Andy's file list
> can only show which disks have a ProDOS filesystem on it, not which have
> sectors in ProDOS order.

That's of course the relevant question. Sorry for jumping to conclusions in
my last post.

Regards,
Oliver

[D Finnigan]

I made a mistake. :-( These queries don't answer the question of how many
disk images are in ProDOS order. Instead, these queries count disk images
which use the ProDOS file system.

I can still answer the desired question, but I'll need to write a little bit
of code to record whether a disk image was in DOS or ProDOS order. Right
now, my code discovers what order is used, but after the disk catalog is
generated, the order is not recorded. (I'm talking about my code that
generates disk catalogs for the Downloads section of the Mac GUI web site.)

[D Finnigan]

Statistics on disk images with ProDOS file system:

2,639 total disk images with ProDOS file system

2,197 in DOS sector order (83.25%), and 442 (16.75%) in ProDOS sector order


Disk image filename extensions:

2,161 .dsk in DOS order
144 .dsk in ProDOS order

0 .po in DOS order
257 .po in ProDOS order

34 .do in DOS order
0 .do in ProDOS order

43 files have no file extension, or have some other file extension


NOTE: 2MG, DC42, SDK, and all other disk image wrappers are NOT COUNTED in
these statistics.

[Oliver Schmidt]

Hi David,

> 2,161 .dsk in DOS order
> 144 .dsk in ProDOS order

So only 6.5% of the .DSK files are in ProDOS order.

> 257 .po in ProDOS order

So the vast majority of ProDOS order images are .PO files.

Looking at those numbers, it seems to me that Kents desire to get rid of
.DSK files in ProDOS order (with all the trouble they cause) is totally
reasonable.

Regards,
Oliver

[David Schmidt]

On 11/8/22 7:26 PM, D Finnigan wrote:
> Statistics on disk images with ProDOS file system:
>
> 2,639 total disk images with ProDOS file system
>
> 2,197 in DOS sector order (83.25%), and 442 (16.75%) in ProDOS sector order
>
>
> Disk image filename extensions:
>
> 2,161 .dsk in DOS order
> 144 .dsk in ProDOS order

That's in keeping with my experience too - "just about everything" 140k
with a .dsk extension is in DOS order, regardless of the filesystem inside.

[D Finnigan]

And the other observation is that when it comes to disk images with ProDOS
file system, the .do and .po extensions are reliable indicators of sector
order.

[D Finnigan]

Agreed. And one might even go so far as to write a script to change the file
extension of ProDOS-ordered .dsk files to .po.

Next point of investigation for me is to see a list of these 144 .dsk images
that are in ProDOS order. What application made these images? Are
ProDOS-sector ordered .dsk images still being made by some application
today?

[I am Rob]

> Next point of investigation for me is to see a list of these 144 .dsk images
> that are in ProDOS order. What application made these images? Are
> ProDOS-sector ordered .dsk images still being made by some application
> today?

Probably the most popular application would have been Dsk2File5.8. It has been around the longest and can handle both Dos order and Prodos order disks as well as create the disk image that can be stored on a hard drive, and from there transferred to a modern computer for upload.

I think Disk.Maker.8 would have got some usage when it came out. And the earlier version of Copy2Plus could make disk images as well, but unsure if it created Prodos disks in Dos order.

I believe one of the questions is still lingering unanswered is, what algorithm did you use to differentiate a Dos order disk from a Prodos order one?

Some emulators use the .po extension to indicate it is an 800kb Prodos disk and doesn't work if a 144 kb disk is used with .po instead. And .dsk was still used to indicate an 800kb AMDos disk as well.

[D Finnigan]

I am Rob wrote:
>
> I believe one of the questions is still lingering unanswered is, what
> algorithm did you use to differentiate a Dos order disk from a Prodos
> order
> one?
>

I look for the catalog sectors.

[Oliver Schmidt]

Hi David,

> And one might even go so far as to write a script to change the file
> extension of ProDOS-ordered .dsk files to .po.

That was - of course - my first thought. But as I'm not the one to perform
such a change, I didn't feel entitled to bring up the topic.

Regards,
Oliver

[fadden]

On Wednesday, November 9, 2022 at 11:54:59 AM UTC-8, gid...@sasktel.net wrote:
> I think Disk.Maker.8 would have got some usage when it came out. And the earlier version of Copy2Plus could make disk images as well, but unsure if it created Prodos disks in Dos order.

IIRC, there's a version that could create disk images in *physical* sector order, but didn't become popular. I'm not sure what version did that; I found copies of 7.4, 8.2, and 9.0 but didn't see the feature in the menus.

[fadden]

On Wednesday, November 9, 2022 at 3:50:41 PM UTC-8, fadden wrote:
> > I think Disk.Maker.8 would have got some usage when it came out. And the earlier version of Copy2Plus could make disk images as well, but unsure if it created Prodos disks in Dos order.
> IIRC, there's a version that could create disk images in *physical* sector order, but didn't become popular. I'm not sure what version did that; I found copies of 7.4, 8.2, and 9.0 but didn't see the feature in the menus.

Found it. With Copy ][+ 7.x, use Copy > Disk, and try to copy a 5.25" disk onto a larger ProDOS volume. It will ask for a filename. The result is a physical-ordered image with type $F7 and auxtype $0118 (==280, indicating the number of blocks).

Attempting to do this with Copy ][+ v6.0 yields the error "both drives must be the same size", while v8.4 reports "device size mismatch". So this was a very short-lived feature.

The procedure works for 800K disks as well, creating an unadorned ProDOS block-order image.

I'm guessing that the feature's obscurity and lack of compression led to its demise. Further, I discovered that if you run out of space on the target volume, it marks the blocks as in-use but doesn't create the necessary index entries, so you have to run Mr. Fixit on the volume afterward.

The preferred extension appears to be ".img".

[I am Rob]

> > Can you describe CiderPress's algorithm for determining sector order of
> > a .dsk file?
> "Complicated".
>

> It tries DOS, ProDOS/Pascal, CP/M, and physical sector orders. (CP/M ordering is not really a thing, since nobody is creating disk images with CP/M software... but we might as well collect the whole set.) It tests the filesystem for DOS, ProDOS, Pascal, CP/M, RDOS, Gutenberg, and is probably checking for HFS even though I doubt anybody put that on a 5.25" floppy.
>
> The tests for DOS / ProDOS filesystems have to take into account non-5.25" situations, like UNIDOS disks and DOS Master floppies, so there's a bunch of auto-detect code that doesn't really apply to 5.25" disks.


Just realized you were testing for file systems and not sector ordering.

There should only be 2 sector orderings, is there not? Sequential like Prodos does it, and DOS ordering? Or does formatting with interleave change the sector orders as well?

Is a pascal disk on a floppy using sector order, whereas a Pascal volume on a hard drive uses Prodos order blocks?
Same with CP/M, does it use both, depending on which disk it is on, floppy or hard drive?

[fadden]

On Thursday, November 10, 2022 at 12:36:58 PM UTC-8, gid...@sasktel.net wrote:
> Just realized you were testing for file systems and not sector ordering.

I'm doing both. If you can't recognize the filesystem, you can't figure out if you have the correct sector ordering. And if you have the wrong sector ordering, the filesystem won't look right.

> There should only be 2 sector orderings, is there not? Sequential like Prodos does it, and DOS ordering? Or does formatting with interleave change the sector orders as well?

The formatters that alter the interleave are moving the physical sectors around. This has no effect on disk images, because those are created by software that requests specific sectors rather than just copying whatever passes under the drive head.

DOS 3.3, ProDOS, UCSD Pascal, and CP/M remap the sector numbers in software. DOS and ProDOS are different from each other because the software sector skew tables are different, so when DOS and ProDOS request sector 1 they get different physical sectors. (Sector 0 is always mapped to 0, so checking its contents doesn't tell you anything about the ordering.)

"DOS order" images come from anything that runs under DOS and reads sectors in sequential order. "ProDOS order" images come from anything that runs under ProDOS and reads blocks in sequential order. If we had CP/M disk image utilities, we'd need to worry about CP/M order too.

> Is a pascal disk on a floppy using sector order, whereas a Pascal volume on a hard drive uses Prodos order blocks?
> Same with CP/M, does it use both, depending on which disk it is on, floppy or hard drive?

ProDOS/SOS and Pascal use the same sector skew. CP/M uses a different skew. Copy ][+ images ignore the software mapping and just copy the sectors in physical order. See _Beneath Apple DOS_ pages 3-22 and 3-23 for an explanation and a table of values.

All of this is applicable only to 16-sector 5.25" floppy disks. DOS 3.2 doesn't do this, and pretty much everything else is treated like a sequential block device.