Question about Disk II Emulation (.woz, .dsk, ...)

[Jens Gaulke]

Hi there,

I'm writing an Apple II Emulator. It boots up perfectly and enters Applesoft. Basic Programs are working fine. To test the sound and graphics, I want to emulate a Disk Drive to be able to load some of the old games and I am quite confused at the moment.

I found out during boot up the Disk II firmware makes heavy use of softswitches - so far, so good.

Where I am stuck is: How do I get a "disk image" in my emulation? Which format must it have? Woz? DSK? Up to know I know "that these formats exist". How can I deal with these formats? What do I have to consider?

How do I "read out" these images to satisfy the emulation and give it the bytes it wants to make the boot up complete?

I don't want just to look up someone else's source code - I want to understand at an more abstract level, what's really going on and start a conversation on this topic.

Cheers,
Jens

[David Schmidt]

On 2/20/23 1:52 AM, Jens Gaulke wrote:
> Hi there,
>
> I'm writing an Apple II Emulator.

> [...]

> Where I am stuck is: How do I get a "disk image" in my emulation? Which format must it have? Woz? DSK? Up to know I know "that these formats exist". How can I deal with these formats? What do I have to consider?

I would start with what your goals are and work backwards from that.
The Disk II is an analog device at is core, and has various layers of
hardware, firmware, and software to decode the analog signal into a
stream of bits. You can insert your emulation at any level along that
spectrum, depending on the fidelity you are interested in achieving -
which will dictate the ultimate capabilities of your disk emulation (and
by extension, your emulator).

At one extreme is the ".dsk" which is only the bytes that the operating
system would ever present to a user; the data within all tracks and
sectors. A layer beneath that is the nibble stream, represented by the
decoded nibbles that are actually recorded on a disk; that affords you
access to all the in-band data and enveloping headers and trailers. And
beneath that is a ".woz" representation that defines lots more
meta-information about the recorded signals and their environment which
are important for encapsulating and decoding the many copy protection
schemes that were devised using the Disk II. That level of information
isn't available in any other encoding.

The place to start getting a complete understanding of the physical Disk
II ecosystem is Sather's seminal work, Understanding the Apple II (and
Apple IIe) - chapter 9, "The Disk Controller". That's every detail at
the bottom layer.

Sather's books (and more) are available here:
https://mirrors.apple2.org.za/Apple%20II%20Documentation%20Project/Books/

The Woz specification is available here:
https://applesaucefdc.com/woz/reference2/

[Michael 'AppleWin Debugger Dev']

On Sunday, February 19, 2023 at 10:52:24 PM UTC-8, Jens Gaulke wrote:
> I'm writing an Apple II Emulator. It boots up perfectly and enters Applesoft. Basic Programs are working fine.

Gratulations on your emulator working! You are (mostly) over the hard/tedious part!

Usually emulation questions are handled over in comp.emulators.apple2 but I'll answer here for convenience.

> I found out during boot up the Disk II firmware makes heavy use of softswitches - so far, so good.

Eventually you will probably need to emulate the Logic State Sequencer (P6/P6A) PROM but for now you can skip it.
i.e.
http://mirrors.apple2.org.za/ground.icaen.uiowa.edu/Mirrors/uni-kl/hardware/diskIIcontroller_hardware

> Where I am stuck is: How do I get a "disk image" in my emulation?

First, the usual pre-requisite reading:

* Beneath Apple DOS (or the combined Beneath Apple DOS and Beneath Apple ProDOS 2020)
* Understanding the Apple II (or Understanding the Apple IIe)
* Tome of Copy Protection -- https://www.amazon.com/Copy-Protection-Bruce-Jones/dp/1387907271
* Be familiar with 4&4 encoding
* Be familiar with 6&2 encoding
* Know how to read nibbles from assembly language, i.e. https://stackoverflow.com/questions/69369122/apple-iie-6502-assembly-accessing-disk
* Optional: SWIM_Chip_Users_Ref_198801.pdf While this covers the IWM (Integrated Woz Machine in the IIgs) it has some useful diagrams.

I would start with:

* motor on/off
* track movement
* write sensor
* add .DSK reading support
* add .DSK writing support
* add .WOZ reading support
* add .WOZ writing support.

> Which format must it have? Woz? DSK?

You probably "need" .DSK and .WOZ since those will cover the majority of disk images. There are also .do and .po extensions but that only effects the logical <--> physical sector order.


You will need to keep track of a few things:

* motor on/off
* reading/writing state
* track number (eventually you need quarter track support but stick with integers for now)
* byte offset (eventually will be a bitstream offset)

For .DSK images when your emulator mounts a virtual image:

* Have a 35 tracks * 16 sector disk image buffer
* Whenever the disk head is moved to a new track build a virtual disk nibble trackr by denibblizing a track. This mean converting each 256 8-bit values back into the 342 6-bit values. This means filling in for each sector
* the FF sync byte gap, say 48 of them,
* the address field,
* another FF sync gap, say 6 bytes,
* the data field,
* the checksum, and
* another FF sync gap, say 27 bytes

* Then when $C0EC is accessed (assuming the motor is on and in read mode), return trackbuffer[ nibbleoffset++ & TRACK_BUFFER ]

This will fail most copy-protection since you aren't emulating any timing but this should let you boot most non-protected software.

> Up to know I know "that these formats exist". How can I deal with these formats? What do I have to consider?

Make sure your 6502 has accurate cycle emulation.

Eventually you will need to emulate the LSS, specifically at the microsecond level, for correctness. i.e. $C0ED will reset the LSS.

> How do I "read out" these images to satisfy the emulation and give it the bytes it wants to make the boot up complete?

See above.

> I don't want just to look up someone else's source code - I want to understand at an more abstract level, what's really going on and start a conversation on this topic.

Have you read the pre-requisite books?

Good luck!

m.

[Michael 'AppleWin Debugger Dev']

Addendum:

When you are ready to dive in the LSS world of timing for 6-bit values (24 cycles), 7-bit values (28 cycles), 8-bit values (32 cycles), 9-bit values (36 cycles), and 10-bit values (40 cycles), or maybe just curious ...

qkumba has this most excellent write-up:

In Search of the Most Amazing Thing; or Towards a Universal Method to Defeat E7 Protection on the Apple ][ Platform
http://pferrie.epizy.com/papers/apple2_e7.pdf?i=1


HIGHLY recommend watching is Mark's Kansasfest videos. The first video is "mandatory", remaining are optional, but you will probably want to watch all of them:

Kansasfest 2015: Mark Pilgrim - E7 E7 E7 EE: the story of the greatest copy protection ever invented
https://archive.org/details/2015_Kfest_Pilgrim_Copy_Protection

Kansasfest 2016: Automated Verification of Copy-Protected Disk Images
https://www.youtube.com/watch?v=T9J_zCVLUjs

KansasFest 2017: Cracking at Scale – Mark Pilgrim
https://www.youtube.com/watch?v=tDVI0OV9e2w


Funny enough there was an OLD post from 1993 that briefly discussed the E7 E7 E7 protection and how to remove it for some games:

Copy protection of most 86-90 II programs
https://groups.google.com/g/comp.sys.apple2/c/3zdIhTxSqY4/m/IddZHUro20oJ


You may also be interested in the source code for Roland Gustaffson's famous RWTS18 where he stores **18 sectors** on a disk instead of the usual 16.
https://github.com/Michaelangel007/apple2_rwts18

Welcome to the rabbit hole of managing the LSS clock and state. =P

Cheers,
m.

[mmphosis]

Other nasty protection schemes come to mind, ultimately they be defeated but
emulating edge cases is a challenge.

I've wondered how far those stepper motors could be pushed. Are we really
only limited to quarter tracks? Fractional tracking anyone?

https://pages.cpsc.ucalgary.ca/~aycock/ra/disk2.html

The spiral disk protection used on Choplifter gives a neat effect.

[Michael 'AppleWin Debugger Dev']

On Monday, February 20, 2023 at 12:29:11 PM UTC-8, mmphosis wrote:
> I've wondered how far those stepper motors could be pushed. Are we really
> only limited to quarter tracks? Fractional tracking anyone?

John (from AppleSauce) commented in the AppleSauce slack channel which Tom copied into our this AppleWin issue: https://github.com/AppleWin/AppleWin/issues/930
"A track is never an integral number."

To expand upon that.

Technically there are infinite tracks due to the voltage of the stepper motors (a clever user could use PWM of the stepper motors for (relative) positioning) however for practical purposes there are only quarter tracks resolution due to the area of flux interference.

i.e. My understanding is that you can't write data with a smaller "width" then at the 1/2 track width. i.e. Data on Track 1 will have flux influencing tracks 0.75 and tracks 1.25. I can't recall who had a graph of this years ago? I would have thought the Applesauce Wiki had it but I guess not.
https://wiki.reactivemicro.com/Applesauce

> https://pages.cpsc.ucalgary.ca/~aycock/ra/disk2.html
> The spiral disk protection used on Choplifter gives a neat effect.

Neat link!

m.

[sc...@alfter.diespammersdie.us]

mmphosis <mmph...@macgui.com> wrote:
> I've wondered how far those stepper motors could be pushed. Are we really
> only limited to quarter tracks? Fractional tracking anyone?

NEMA 17 motors in 3D printers are routinely driven by anywhere from 1/16
steps to 1/256 steps. This sort of fractional stepping, though, is produced
with dedicated stepper-motor controllers. If I remember right, two steps
span one track, with half-steps (quarter tracks) produced by turning
adjacent coils on. That's about as much control as you can get without
twiddling coils on the fly, and since just reading the disk takes the CPU's
full attention, I don't think there's time left to do more than
quarter-tracking.

--
_/_
/ v \ Scott Alfter (remove the obvious to send mail)
(IIGS( https://alfter.us/ Top-posting!
\_^_/ >What's the most annoying thing on Usenet?

[Jens Gaulke]

I have understood that so far, thank you for the explanation. What I have problems with is "You can insert your emulation at any level along that spectrum". I don't understand how I could write only one emulation for the dsk format when my emulator gives me one Disk II softswitch after another. Then don't I have to start at that level and edit/implement the softswitches?

[Jens Gaulke]

Thank you for your reply. That's about what I was thinking too. Then I know which way I have to go. I didn't know the "Tome of Copy Protection", I ordered it once as a reading on Amazon. Sorry for posting in the wrong group, I'll sign up for the group you suggested as well.

[Jens Gaulke]

That is a very useful link. Thanks for sharing!