DSK to WOZ converter

[Thomas Harte]

Quick announcement: at https://github.com/tomharte/dsk2woz you will now find a plain C command-line tool for converting DSK images to the higher-fidelity WOZ format.

The utility was requested for use in a development toolchain; there's obviously no inherent reason to convert DSK images to WOZ — it can't magically put back any of the content that was lost in the original conversion. But in case anybody ever needs such a tool, there it is.

[Steve Nickolas]

I can see a potential use: in emulation processing a disk image natively
in the WOZ format (and possibly translating back upon exit). I think EMU2
did this with NIB instead of WOZ as the internal format.

-uso.

[Thomas Harte]

On Tuesday, 5 June 2018 20:50:57 UTC-4, Steve Nickolas wrote:
> I can see a potential use: in emulation processing a disk image natively
> in the WOZ format (and possibly translating back upon exit). I think EMU2
> did this with NIB instead of WOZ as the internal format.

It's MIT-licensed so hopefully that'll be permissive enough for anything people might like to do with it as code; I've added Doxygen-style documentation to that end.

For the use you describe a program could use serialise_track as an intermediary, as it converts DSK-style tracks to WOZ-style, and then have everything from track inspection onwards blind as to the underlying format.

[David Schmidt]

On 6/5/2018 8:21 PM, Thomas Harte wrote:
> Quick announcement: at https://github.com/tomharte/dsk2woz you will now find a plain C command-line tool for converting DSK images to the higher-fidelity WOZ format.

Nice work. I'm working on the inverse, too.

In my Woz spec 1.0, it indicates that the TMAP +- 1/4 tracks should also
be set to that track value. So for example track 1.00 gets 0x01, as
does 0.75 and 1.25. Only a single 0xFF separates tracks (at the 1/2
track entry), supported by Woz images I've seen so far. I think you're
probably safe with your minimalist setting of TMAP, but simulating the
width of the write head is probably truer to the spec.

[Thomas Harte]

Agreed; I'm inappropriately modelling a very slender disk head indeed. Will correct this evening along with maybe some other tweaks.

[David Schmidt]

On 6/6/2018 8:52 AM, Thomas Harte wrote:
> [...]

> Agreed; I'm inappropriately modelling a very slender disk head indeed. Will correct this evening along with maybe some other tweaks.

Cool. I sent in a pull request for the head width, but I'm sure you'll
find a more elegant way of doing it!

[Thomas Harte]

Oh, no, fantastic. Saved me the effort! Merged.

[John Morris]

Nice work Thomas! I did a little bit of code cleanup to it and submitted a pull request. This seems to have fixed all of the outstanding issues and is successfully creating woz images!

[Thomas Harte]

On Wednesday, 6 June 2018 17:16:24 UTC-4, John Morris wrote:
> Nice work Thomas! I did a little bit of code cleanup to it and submitted a pull request. This seems to have fixed all of the outstanding issues and is successfully creating woz images!

Confirmed; the amount of data being written was 12 bytes too long; the declared size of the TRKS was incorrect; the gap lengths were idiosyncratic. All fixed by your pull request.

[Melody Ayres-Griffiths]

microM8 does this, but we don't bother translating it back to a DSK on exit - if you write to it at any point it just saves the image as a .woz

[Ianoid]

On Tuesday, June 5, 2018 at 7:21:09 PM UTC-5, Thomas Harte wrote:
> Quick announcement: at https://github.com/tomharte/dsk2woz you will now find a plain C command-line tool for converting DSK images to the higher-fidelity WOZ format.
>
> The utility was requested for use in a development toolchain; there's obviously no inherent reason to convert DSK images to WOZ — it can't magically put back any of the content that was lost in the original conversion. But in case anybody ever needs such a tool, there it is.

I hope your program includes information in one of the .woz metadata fields that explains the origin of the file, that it is a .dsk to .woz conversion, so we don't confuse it with a .woz Applesauce acquisition.

[David Schmidt]

It does include metadata about the creator, unique to this utility
program. But make no mistake - just because Applesauce toolchain
captured a real disk and created a .woz file out of it doesn't give it
magical powers to know the metadata surrounding the original disk. The
user is prompted to type in that information at initial capture time,
but it's not required by any means. This synthesized .woz image is no
less "valid" than one originating from a physical capture process.
Veracity of (meta)data is going to ultimately be about as good as
Wikipedia, really. Kind of policed by the crowd, but there is no
ultimate truth.

[John Morris]

> > I hope your program includes information in one of the .woz metadata fields that explains the origin of the file, that it is a .dsk to .woz conversion, so we don't confuse it with a .woz Applesauce acquisition.
>
> It does include metadata about the creator, unique to this utility
> program. But make no mistake - just because Applesauce toolchain
> captured a real disk and created a .woz file out of it doesn't give it
> magical powers to know the metadata surrounding the original disk. The
> user is prompted to type in that information at initial capture time,
> but it's not required by any means. This synthesized .woz image is no
> less "valid" than one originating from a physical capture process.
> Veracity of (meta)data is going to ultimately be about as good as
> Wikipedia, really. Kind of policed by the crowd, but there is no
> ultimate truth.

I think that Ianoid is concerned about end users not being able to know if a particular woz file is from an original or is a converted crack from dsk or whatever. At this point, there is no way for a user to know as no software/emulators will actually display the creator field of the info chunk. The woz structure might be valid, but that doesn’t mean the data in it is valid or original.

The metadata coming out of Applesauce is user inputted, but it is also designed to just be transcribed from the label of the disk. As long as people enter that correctly, I don’t think there is going to be an issue with veracity. But yeah, some people are stupid and will screw it up.

[David Schmidt]

Today, there is a kind of "authority" that is producing woz images. A
time will come when there is no authority but consensus, and the
veracity will be difficult/impossible to discern through inspection of
the image itself.

[John Morris]

Actually, when looking at images visually like in the Applesauce software, you can very often tell originals from conversions. You can even see the telltale signs that it has been copied by Copy II+ or Locksmith.

But it would definitely be a good thing to get ahead of this as much as possible. Maybe add a metadata entry to the spec for what the source image was?

[Zellyn]

On Thursday, June 7, 2018 at 10:22:26 AM UTC-4, John Morris wrote:
> Actually, when looking at images visually like in the Applesauce software, you can very often tell originals from conversions. You can even see the telltale signs that it has been copied by Copy II+ or Locksmith.

It would be a fun project to try to automate that classification… :-)

Zellyn

[Thomas Harte]

On Thursday, 7 June 2018 10:22:26 UTC-4, John Morris wrote:
> Actually, when looking at images visually like in the Applesauce software, you can very often tell originals from conversions. You can even see the telltale signs that it has been copied by Copy II+ or Locksmith.
>
> But it would definitely be a good thing to get ahead of this as much as possible. Maybe add a metadata entry to the spec for what the source image was?

This utility provides a creator name of 'dsk2woz 1.0' inside the INFO chunk; it doesn't actually write any METAs.

Thinking about it, the capability to hold metadata might actually be another motivation for DSK to WOZ conversion beyond just the intended step-in-a-development-process: you can explicitly bundle title, subtitle, publisher, developer, etc into the file.

[Thomas Harte]

My emulator uses an internal representation that's even slightly more advanced than WOZ by supporting different data densities across the surface. Other than adjusting of the drive rotation speed dependent on Disk II mode I can think of no other way you can emulate writing correctly unless the source WOZ just-so-happened to have exactly the number of bits per track that matches the combination of your ordinary rotation speed and the Disk II clock.

I run a PLL over the entire surface to reproduce the bit stream for output to WOZ, so that's inauthentic in that if you've written to the disk then you might implicitly have affected the timing of the contents of the rest of the track. I don't see a way to avoid that given that the number of bits that can represent a track must fit within a 16-bit number. So you can't just take the least-common-multiple of the various rates and pad out the 1s.

Then for NIBs I do the idiomatic: throw all the syncs away, pad out to a longer-than-would-ever-be-realistic length and wish the recipient good luck with that.

For DSKs I decode and grab sector contents. Which can potentially be very lossy indeed, depending on how you modified the thing.

I should probably warn the user when it looks like interesting content might be lost and offer an upgrade to WOZ, but it's not automatic.

[Zellyn]

On Thursday, June 7, 2018 at 11:15:46 AM UTC-4, Thomas Harte wrote:
> On Wednesday, 6 June 2018 22:05:15 UTC-4, Melody Ayres-Griffiths wrote:
> My emulator uses an internal representation that's even slightly more advanced than WOZ by supporting different data densities across the surface. Other than adjusting of the drive rotation speed dependent on Disk II mode I can think of no other way you can emulate writing correctly unless the source WOZ just-so-happened to have exactly the number of bits per track that matches the combination of your ordinary rotation speed and the Disk II clock.

I believe John stores something like this as the raw output from his hardware (except with multiple copies of each quarter-track), and then converts down to woz.

[John Morris]

Yeah, the A2R format for Applesauce records the number of nanoseconds between flux transitions across the entire disk surface. Things like bit rates are completely irrelevant to it. The Applesauce software then converts the timing information into bitstreams.

[Thomas Harte]

On Thursday, 7 June 2018 13:49:17 UTC-4, John Morris wrote:
> Yeah, the A2R format for Applesauce records the number of nanoseconds between flux transitions across the entire disk surface. Things like bit rates are completely irrelevant to it. The Applesauce software then converts the timing information into bitstreams.

My emulator could consume that, or any intermediate format (e.g. sampling the flux stream at 2.046Mhz or any other arbitrary high resolution); since I strongly suspect you've discovered that it isn't useful I don't think it's really important.

The patches the emulator keeps in-memory on top of the track to record write patches are actually at 2.046Mhz because as discussed elsewhere I'm just running the Disk II state machine ROM, because I'm lazy. Which means that I don't actually know that much in advance about when a transition will be written other than that the options are discrete, so just sampling it at the Disk II clock rate saves me some effort there. I'm very, very lazy.

In case anybody is curious, it then costs me about 75% as much to emulate the Disk II as it does to emulate the Apple II. But that's still only a net 10% to 15% of the 1.1 Ghz Core M in my 2015 MacBook so I'm not that bothered. Though the emulator is smart enough to clock it selectively, based on the combination of whether the drive motor is on, selected operation and current shift register contents.

Apologies for the digression!