Musings of sound emulation
BLuRry
I've always had issues getting the sound to work just right. Either
the timing of the whole emulator would shift dramatically too slow/too
fast when sound emulation is activated/deactivated, or there would be
sound buffer underruns/overruns and so on causing sound to sound
glitchy or the whole emulator barfs with a stack trace if things
aren't going well (say, an expensive context switch de-prioritized the
emulator process and glitched the whole timing logic temporarily.) I
think I've finally got it nailed down, and checked in my source
changes to java-ace (java apple computer emulator) in sourceforge for
anyone poking about in Java for emulators, realtime DSP or otherwise.
Main emulator thread evokes devices on every "tick", and the overall
main loop is self-tuned at set intervals. In Java 1.5 and above,
LockSupport has a nano-accurate sleep function that is much more
efficient than Thread.sleep: parkNanos. Some emulators use sound
output to control emulation speed, but I've always wanted to avoid
this since I don't want to require sound output blocking just to offer
timing accuracy.
The Speaker device "tick" method is called on every CPU cycle by the
main thread, and the amplitude of the sample is determined by how many
times the speaker is toggled during the sampling period (ratio of
clock cycles per sampling period or 1.25-ish mhz divided by 44.1 khz
-- roughly 23 and some change). I can adjust the timing to emulate
the effect of a sped-up apple (higher-pitch) or slow down when the
sound comes on, only requires some configuration to be exposed to
highlight that as a "feature." When a sample is collected, it is
added to the sound buffer. If the buffer is full, awaiting playback,
the main thread blocks until the sound playback thread has caught up.
This avoids buffer overruns, which manifest as array out of bounds
exceptions.
Speaker sound output is managed in a separate thread. If the sound
was recently re-enabled but the buffer is rather empty, sound will not
start playing back right away. This avoids a rather nasty/expensive
waste of resources caused by the native playback call blocking for
long periods when there isn't enough work to balance out the expense
of the call (buffer underrun), which result in emulation slowdowns,
CPU spikes, and gaps in audio. When the sound out has room, and there
is enough to playback, a semaphore is borrowed and a flag is thrown to
reset the buffer -- then the buffer itself is spit out for playback.
Back in the main thread, before queuing sound samples, the reset flag
is evaluated and if flagged the sound buffers are switched and sound
is stored in the second buffer. This double-buffering and semaphore
usage avoids race conditions where sound is being played from the
buffer while new samples are being recorded. The critical sections
have been made as small as possible to avoid unnecessary blocking.
All in all, it's not very simple but it is rather robust. Now my more
expensive emulation slowdowns are the video output (emulator uses 20%
of my dual-core 2ghz CPU) and I'm looking at the Dapple source and
noticing it flags each scanline as dirty when the ram is updated and
uses dirty flags to avoid redrawing unchanged scanlines. I think I
should investigate adding this to the mix and I expect it will have a
big change in performance. Freeing up more CPU means that realtime
NTSC emulation could be added, yet optimized to avoid a huge
performance hit by skipping the redraw for unchanged screen contents.
I think that getting the sound just right was a rather large hindrance
in continuing to work on an emulator, and also my overall belief that
it was feasible to continue development with this code architecture.
With rather lofty goals in reach (e.g. cards are just objects placed
into slots, nothing is hard-coded so configuring hardware is very
trivial at the low-level), I think that it is possible to take more
large steps in the future -- at some point. Progress is still only
vaguely sporadic, though the emulator is in its current form
functional enough for keyboard-based games and sound programs.
Largely it serves as the testbed for Apple Game Server development.
I'll not expect to ever capture the feature list of say, Bernie or
AppleWin. But my goal was to learn about how the Apple //e works
inside and out, and tune my ability to write well-performing code at
the same time. For that I've been pretty successful -- and am happy
to offer some inspiration of ideas to other emulators (TCP/IP SSC
emulation -- nifty keen ain't it?) :-)
][ infinitum!
-B
Oliver Schmidt
>I
>think I've finally got it nailed down, and checked in my source
>changes to java-ace (java apple computer emulator) in sourceforge for
>anyone poking about in Java for emulators, realtime DSP or otherwise.
Thanks for the detailed background information! Reminds me a little of
a very old posting of mine ;-)
http://groups.google.de/group/comp.emulators.apple2/msg/7a1b9317e7905152
Best, Oliver
TomCh
That clears up a bit of confusion I had when refactoring the Disk code
recently:
void ImageClose(const HIMAGE hDiskImage, const bool bOpenError /
*=false*/)
{
//...
for (UINT uTrack = 0; uTrack < ptr->uNumTracks; uTrack++)
{
if (!ptr->ValidTrack[uTrack])
{
// What's the reason for this?
bDeleteFile = true;
break;
}
}
//...
}
I can now replace this comment with your posting from 2001 (albeit
abridged :)
I had just assumed that ValidTrack[] was from the original 1995 code
base.
Tom
BLuRry
> expensive emulation slowdowns are the video output (emulator uses 20%
> of my dual-core 2ghz CPU) and I'm looking at the Dapple source and
> noticing it flags each scanline as dirty when the ram is updated and
> uses dirty flags to avoid redrawing unchanged scanlines.
Well that was fun to implement. :-) I added two levels of tracking
for this one. Video generation is handled at the lowest level by
modular writer objects. For each mode and for page 1 or 2 there is a
separate writer -- that way the active writer can be swapped out when
softswitches are toggled. Because of this I can easily track, at the
time a byte is written to screen, which writer was active. On the
next screen refresh I can see if the current writer is the same or
different than last time for that byte. If it is changed, the byte
should be redrawn.
If the writer is the same but there was a change on the scanline
anywhere, then it is redrawn. I decided to do the second check at the
scanline level to avoid a lot of extra overhead tracking changes.
Scanline changes are detected by memory writes to the video ram areas
-- when there is a write, I do a fast lookup to determine what
scanline is affected and throw a dirty flag, which is not cleared
until the end of the line (horizontal refresh).
In addition to all of that, if no screen drawing happens for a full
frame, the resulting image is not refreshed to avoid scaling/blitting
overhead. This is pretty cool balancing of resources since sound
output can be as time consuming as video -- but there's rarely video
updates when a program plays sound and vice versa!
Limitations:
-Flashing text is still supported, but only by setting all dirty bits
on the current active text page whenever the flash period elapses.
This doesn't happen when mousetext is active at least.
-Floating bus support is [even more] broken by this since relevant
code parts are not getting executed when the screen or parts of it
don't change.
-Race conditions could occur if there is any change to video
softswitches while in the middle of a scanline, I could change this by
replacing dirty flags with "last touched" flags using current system
time in nanos. Seems like overkill, will require more play testing
before I make any bold moves.
Anyway, I profiled the emulator and it really scales down a lot when
idle. The 65c02 CPU is now the most time-consuming part of the main
loop, but that should be expected. I have some other crazy ideas for
emulation experimentation, and now I'm not bogged down in bugfixes I
can start to play with other ideas.
I wonder, wouldn't it be interesting if Prodos saw a folder of my
harddrive as a partition? Could be a fun venture, if not relatively
insane. :-D
-Brendan
Michael J. Mahon
>>All in all, it's not very simple but it is rather robust. Now my more
>>expensive emulation slowdowns are the video output (emulator uses 20%
>>of my dual-core 2ghz CPU) and I'm looking at the Dapple source and
>>noticing it flags each scanline as dirty when the ram is updated and
>>uses dirty flags to avoid redrawing unchanged scanlines.
>
>
> Well that was fun to implement. :-) I added two levels of tracking
> for this one. Video generation is handled at the lowest level by
> modular writer objects. For each mode and for page 1 or 2 there is a
> separate writer -- that way the active writer can be swapped out when
> softswitches are toggled. Because of this I can easily track, at the
> time a byte is written to screen, which writer was active. On the
> next screen refresh I can see if the current writer is the same or
> different than last time for that byte. If it is changed, the byte
> should be redrawn.
I don't understand why which writer changed a byte is important. The
appearance of the screen depends only on the bytes themselves. If the
display mode changes, then all the bytes will have to be redrawn
while the display mode is active, not just the ones written in that
mode.
> If the writer is the same but there was a change on the scanline
> anywhere, then it is redrawn. I decided to do the second check at the
> scanline level to avoid a lot of extra overhead tracking changes.
> Scanline changes are detected by memory writes to the video ram areas
> -- when there is a write, I do a fast lookup to determine what
> scanline is affected and throw a dirty flag, which is not cleared
> until the end of the line (horizontal refresh).
Also, for NTSC rendering, changing a byte may affect the rendering
of surrounding bytes, so redrawing the whole line handles that.
> In addition to all of that, if no screen drawing happens for a full
> frame, the resulting image is not refreshed to avoid scaling/blitting
> overhead. This is pretty cool balancing of resources since sound
> output can be as time consuming as video -- but there's rarely video
> updates when a program plays sound and vice versa!
Good point!
> Limitations:
> -Flashing text is still supported, but only by setting all dirty bits
> on the current active text page whenever the flash period elapses.
> This doesn't happen when mousetext is active at least.
At least flashing only occurs every several frames, so there's still
plenty of saving.
You could try setting only lines containing flashing text dirty to
save even more...
> -Floating bus support is [even more] broken by this since relevant
> code parts are not getting executed when the screen or parts of it
> don't change.
But isn't the video byte just a function of cycles elapsed since
VBL (or whatever frame reference point you choose)?
If a reference to the floating bus is detected, you can easily compute
which byte of the video buffer was last accessed if you know the cycle
count, completely independent of the state of the PC's video buffer.
> -Race conditions could occur if there is any change to video
> softswitches while in the middle of a scanline, I could change this by
> replacing dirty flags with "last touched" flags using current system
> time in nanos. Seems like overkill, will require more play testing
> before I make any bold moves.
This would be needed for "vapor locked" mixed modes.
> Anyway, I profiled the emulator and it really scales down a lot when
> idle. The 65c02 CPU is now the most time-consuming part of the main
> loop, but that should be expected. I have some other crazy ideas for
> emulation experimentation, and now I'm not bogged down in bugfixes I
> can start to play with other ideas.
Excellent!
> I wonder, wouldn't it be interesting if Prodos saw a folder of my
> harddrive as a partition? Could be a fun venture, if not relatively
> insane. :-D
I wonder how all the attributes map?...
-michael
NadaNet and AppleCrate II: parallel computing for Apple II computers!
Home page: http://home.comcast.net/~mjmahon
"The wastebasket is our most important design
tool--and it's seriously underused."
BLuRry
> appearance of the screen depends only on the bytes themselves. If the
> display mode changes, then all the bytes will have to be redrawn
> while the display mode is active, not just the ones written in that
> mode.
Well, it's an optimization meant to catch cases when screen modes
change in the middle of a scanline. Perhaps it is a bit overkill
though!
> Also, for NTSC rendering, changing a byte may affect the rendering
> of surrounding bytes, so redrawing the whole line handles that.
You have a good point. That reminds me I had originally planned to
capture rendering bits during the horizontal rendering period and then
blit them to the screenbuffer during the horizontal refresh.
> You could try setting only lines containing flashing text dirty to
> save even more...
I thought about that, but then I'd have to be looking at every byte
stored to text ram -- and that means more conditional logic executed
during memory operations. Not sure I want to take it that far, as it
might not be worth the trade-off.
> > -Floating bus support is [even more] broken by this since relevant
> > code parts are not getting executed when the screen or parts of it
> > don't change.
>
> But isn't the video byte just a function of cycles elapsed since
> VBL (or whatever frame reference point you choose)?
Was it? I thought it was the last byte read by the video scanner.
All the modes have floating bus support, since they have to read
memory to draw the screen. The last read value is put into a global
variable. When the floating bus is polled, the read operation is
caught by a memory listener which intervenes and returns the floating
bus value from the same global. In theory that should work, but I
haven't quite got around to debugging it yet.
> If a reference to the floating bus is detected, you can easily compute
> which byte of the video buffer was last accessed if you know the cycle
> count, completely independent of the state of the PC's video buffer.
I just store the value into a variable while I already have it, that
way I don't have to do much extra work later on. It all requires
extra testing. But for now, the video overhead is so reduced, I can
full-screen at 1920x1080 and let Java2D blit the whole surface without
losing framerate! I ran a CPU vs CPU round of BallBlazer and the
sound and video were spot on. I should just release the damn emulator
as 1.0 already -- it DOES work for some things. :-D
> > I wonder, wouldn't it be interesting if Prodos saw a folder of my
> > harddrive as a partition? Could be a fun venture, if not relatively
> > insane. :-D
>
> I wonder how all the attributes map?...
A good question. Ciderpress naming conventions might be useful, where
there is some sort of delimiter on the filename to add type and offset
attributes. Or I could come up with my own, e.g. three letter
extension maps to the file type (and friendly ones for BAS, BIN, TXT
and SYS), and the memory offset is captured ciderpress-style, e.g.
#a2000 -- so a nibble two-liner that runs at $300 would be
myprog#a2000.bin and a system file that runs from $2000 (standard)
takes a default by way of omission, e.g. prodos.sys. Other types
could be represented by Thh where hh is the hex code of the file
type. So T06 == BIN (but much less readable). Or combine all
attributes into the extension, e.g. MYGAME.BIN#a6000. I suppose the
tricky part is to translate prodos block writes into updates to
physical files and also keep tabs on the real filesystem to track if
files have been refreshed, etc. There are bound to be lots of gotchas
and limits to this approach.
It could be possible to go beyond the "limit" of prodos file system
storage if it is possible to hook in such a way that you know what
file is being accessed all the time. Files could be represented in a
dynamic manner that appears like they overlap, but depending on which
file is accessed will determine what a block read/write will really by
working with. This would not be compatible with other utilities
(obviously not disk copiers or dos master, reporting free space,
etc). But it could let you manage a larger superset of information
without reaching the "physical" limitation so quickly.
-B
Michael J. Mahon
>>I don't understand why which writer changed a byte is important. The
>>appearance of the screen depends only on the bytes themselves. If the
>>display mode changes, then all the bytes will have to be redrawn
>>while the display mode is active, not just the ones written in that
>>mode.
>
>
> Well, it's an optimization meant to catch cases when screen modes
> change in the middle of a scanline. Perhaps it is a bit overkill
> though!
>
>
>>Also, for NTSC rendering, changing a byte may affect the rendering
>>of surrounding bytes, so redrawing the whole line handles that.
>
>
> You have a good point. That reminds me I had originally planned to
> capture rendering bits during the horizontal rendering period and then
> blit them to the screenbuffer during the horizontal refresh.
>
>
>>You could try setting only lines containing flashing text dirty to
>>save even more...
>
>
> I thought about that, but then I'd have to be looking at every byte
> stored to text ram -- and that means more conditional logic executed
> during memory operations. Not sure I want to take it that far, as it
> might not be worth the trade-off.
I'd be surprised if a single IF would make much difference in this
path, but then flashing text is not that usual anyway...
>>>-Floating bus support is [even more] broken by this since relevant
>>>code parts are not getting executed when the screen or parts of it
>>>don't change.
>>
>>But isn't the video byte just a function of cycles elapsed since
>>VBL (or whatever frame reference point you choose)?
>
>
> Was it? I thought it was the last byte read by the video scanner.
> All the modes have floating bus support, since they have to read
> memory to draw the screen. The last read value is put into a global
> variable. When the floating bus is polled, the read operation is
> caught by a memory listener which intervenes and returns the floating
> bus value from the same global. In theory that should work, but I
> haven't quite got around to debugging it yet.
It is exactly the last byte read by the video scanner, which is
determined by how many cycles have elapsed since the last frame
reference point (start, end, what-have-you).
Since floating bus reads are so rare, I wouldn't even bother to
keep the "current byte"--just use the cycles elapsed since the last
"frame reference point" to index it when required.
You already keep an accurate cycle count, and the video scanner
takes a constant number of cycles per frame--so it's just a mod
operation.
>>If a reference to the floating bus is detected, you can easily compute
>>which byte of the video buffer was last accessed if you know the cycle
>>count, completely independent of the state of the PC's video buffer.
>
>
> I just store the value into a variable while I already have it, that
> way I don't have to do much extra work later on. It all requires
> extra testing. But for now, the video overhead is so reduced, I can
> full-screen at 1920x1080 and let Java2D blit the whole surface without
> losing framerate! I ran a CPU vs CPU round of BallBlazer and the
> sound and video were spot on. I should just release the damn emulator
> as 1.0 already -- it DOES work for some things. :-D
You want to avoid work in the normal case, not in the unusual case.
But it sounds like it's working!
It would be useful to write down exactly what has to work to satisfy
most needs, and what does not need to work.
If the limitations become numerous or capricious, then it will not
be as useful as you wish.
BLuRry
> most needs, and what does not need to work.
Good point. Basic file load/store and directory/subdirectory support
is ideal. Attributes describing file that are stored in prodos
directory block should be part of each file's name to avoid having
dangling tracking metadata stored in another file somewhere. Emulated
drive should represent itself as a block storage device, occupying a
slot in memory and acting like a prodos block storage device. All
filesystem structures will be built on-demand to avoid wasting
memory. The internal memory data structure will be a similar tree to
the prodos format, everything being either a leaf (seedling files) or
a node. Subtypes of node will handle directory blocks, as well as
sapling and trees for files. As an initial implementation, the
virtual prodos volume will represent itself as the largest amount of
space possible (65536 blocks). Files will be allocated in the virtual
space as they, or their containing directories, are queried -- but if
space is running thin, then least recently used files will be unmapped
from the tree and the virtual block mapping can be reused for
additional files.
Case: Directory block is requested
- Directory entries must be created for each file, so allocate each
consecutive available block as the key block for each file, map
internal structure as not yet defined
- Return newly constructed directory block
Case: File is requested
- If file mapping is current, reuse existing block numbers and map
each block to a section of the physical file on disk
- If file mapping is not defined yet, build virtual index blocks and
allocate available unused blocks to represent file structure.
-- At any point any of the blocks are accessed, the file can remain
on the most recently used list to retain its mapping longer
Case: File is updated
- File block writes are retained for further analysis
- Depending on write order (requires investigation), it may not be
until the directory block or the key block is updated until we know if
it is a file update or a file creation event. Any changes to memory
mapping will be necessary to accommodate the write event (say, if a
file grows into a sapling or tree, the internal data structure must be
refreshed). How would the end of the complete file operation be
signaled though? Would I have to peek at Prodos memory registers to
snoop out the necessary info?
I might be able to fool prodos into revealing its logic if I send it a
free space volume map that tricks it to writing new blocks to the same
set of blocks every time, then I might have less work trying to
determine some events like creating a directory or new file. But that
might have some nasty side effects. At the same time, reading memory
locations to determine what is going on would make the emulation less
generic or portable.
Given the compact nature of the supporting logic, this might not be as
hard to emulate as I thought it might be. A virtual harddrive isn't
too hard to do as long as it presents itself correctly to prodos as a
block device. Bootup can be achieved by a simple stub of code that
signals the emulator to do something specific. I could write a
listener that looks for successive store attempts to $CxNN (where x is
the slot and NN is anywhere I want, say 42) can indicate the desired
action ($00 = boot, $01 = debug, whatever). Prodos MLI calls to read/
write routines can be trapped and intercepted with java code that
implements the appropriate load/store emulation). The emulator thinks
it is issuing a JSR, but what happens it it becomes a trap to halt the
virtual CPU and spring a java method call, which then returns via
emulated RTS when it is done. Was sort of my evil plan all along to
go this route.
Could also be a fun way to introduce an easter egg vector (e.g. trap
the ?SYNTAX ERROR routine and do something funny with it if it keeps
getting triggered over and over -- like a helpful popup featuring
Clippy taunting the user: "Looks like you're trying to unsuccessfully
write an Applesoft basic program.")
>
> If the limitations become numerous or capricious, then it will not
> be as useful as you wish.
>
Right, which is why I would emulate at the block structure level and
build from there, hopefully covering all necessary ground except for
DOS.MASTER or anything else that writes to blocks that are not mapped
to any directory anywhere. Would that be sufficient for most prodos
programs?
-Brendan
Oliver Schmidt
>> http://groups.google.de/group/comp.emulators.apple2/msg/7a1b9317e7905152
>That clears up a bit of confusion I had when refactoring the Disk code
>recently:
:-)
>I had just assumed that ValidTrack[] was from the original 1995 code
>base.
Yeah, by that time on-demand source code control systems like
SourceForge were just not available/popular/on my mind like today -
unfortunately...
Best, Oliver