Dolphin Emulator Sound Crackling Android

[Blanchefle Strycker]

Likeany artistic medium, games are emotional experiences filled with joy, sadness, frustration, and more. Special moments can bring tears, cause shouting, or even screams. But imagine during one of those emotional highs if the audio simply died, and the game continued onward in a deafening silence before eventually freezing. That kind of marred experience was commonplace under Dolphin's old way of handling HLE audio.

The users of today don't have to face those problems; modern High Level Emulation (HLE) audio is both fast and accurate, mostly matching the conventions and output of its high-accuracy counterpart, Low Level Emulation (LLE) audio. This change in behavior is thanks to the work by delroth and magumagu that corrected the main fundamental flaw that afflicted old HLE audio. Fixing this defect and cleaning up the audio brought a multitude of features and fixes to the emulator that helped bring us into this modern era of speedy accuracy.

While the end result is great, the journey was anything but smooth, filled with uncovered bugs, regressions, controversy and more! Thanks to the help of contributors degasus, booto, skidau, konpie and phire Dolphin has nearly perfect audio emulation.

As the first GameCube emulator to boot commercial games, there was a time when having any audio emulation was an achievement! The GameCube's Digital Signal Processor (DSP) was effectively a black box to the early developers of Dolphin. As such, they did what any normal person would do and poked it with data and analyzed what it put out. While not the easiest or most efficient way to work, ector (hrydgard), StapleButter and many others brought the DSP-HLE plugin into Dolphin before the 1.0 releases. The original DSP-HLE mostly just played music in games that streamed audio, but was capable of sometimes playing sounds and other music.

This HLE audio core continued to evolve along with the rest of the emulator. By the time Dolphin hit open source, games could play music and sounds, even if things didn't sound exactly right. With fresh life breathed into Dolphin from the move to open source, DSP-HLE continued to evolve, providing the source of primary audio for the emulator for year after year. But that evolution would slow to a crawl in its final years, even as glaring issues such as crashing, hangs, and audio stopping plagued the emulator. The truth was that the original HLE audio implementation handled things in a broken way that could not be reconciled.

The old HLE code was unable to do this properly, thanks to an asynchronous processing method. The 5ms interrupts weren't guaranteed to happen after 5ms* of CPU time. Old HLE audio could play these samples with up to 100ms of variance. In a worst case scenario, twenty updates could happen instantly, followed by 100ms of total silence or messed up audio. And if the emulator wasn't running full speed, things could get even more confusing as the DSP would plow ahead as if everything was fine. These behaviors resulted in many of the numerous problems that plagued DSP-HLE for years.

AUX processing is a feature of the DSP program that allows the DSP to apply audio effects on the sounds, such as echo, reverb, chorus, etc. AUX uses DSP commands to download and upload data to the CPU in order for the CPU to process the AUX effect while the DSP is working on something else.

Since the DSP always operates at set intervals, game developers used this to time when sounds should start and stop, especially when composing music from samples. Of course, 5ms accuracy is often not enough for music; so the DSP provides a feature called "sub-5ms updates" which can specify some slight changes on the sound data blocks for each millisecond. Failing to emulate this nifty feature can cause games to get exceedingly confused as the DSP doesn't make the changes it's expecting. Once the timings are off, anything can happen: audio garbling, instruments out of sync, popping/cracking, along with many, many more possible problems. It's not pretty.

One of the more infamous problems in Dolphin emulation history was the fact that The Legend of Zelda: The Wind Waker would hang a lot when collecting items. It was so common, cheat codes were added to the GameINI to work around the hang. Link picks up an item, the jingle plays, and then the game just sits there waiting, not frozen, not crashed, still emulating, just... waiting. What has happened in this case?

In a basic sense, some games wait for sounds to be completed before they will continue. When the sound processing is done asynchronously, the game may miss the moment when the sound has finished playing (because it went too fast compared to the emulated CPU speed) and just freeze. This tended to affect the Zelda code games more than anything. This separate code posed extra challenges for Dolphin due to tighter timings and tons of features unique to itself. In HLE audio, it's completely isolated from the handling of the AX code.

There were a lot of problems unique to these games because of this, such as issues with certain sounds and tons of music hangs. Mario Kart: Double Dash!! and Super Mario Galaxy 1/2 both suffered consistent music hangs, leading to the infamous Grand Star hang in the Super Mario Galaxy series.

To work around the issues of DSP-HLE, nakeee, LordMark and others created the DSP-LLE Interpreter. It represented an extremely accurate way of handling audio by perfectly reproducing what the DSP does, including synchronous audio processing. HLE audio could only approximate what the DSP does with C++ code, while LLE audio translated the binary code into x86, producing far more accurate results.

Unfortunately, using an interpreter for the DSP is painfully slow and grinds the entire emulator to a halt. skidau and kiesel created DSP-LLE Recompiler to combat this slowdown the same way Dolphin uses a JIT for the main processor of the GameCube/Wii. Everything in DSP-LLE recompiler is the same as the interpreter, except it produces those very same results much faster.

However, DSP-LLE required DSP ROM dumps from a GameCube or Wii. While the procedure was fairly involved on a GameCube, later on the ease of softmodding a Wii would make retrieving DSP dumps simple - if you had one. Because of the dumps requirement and the chore required to get DSP ROM dumps, DSP-LLE could not be set as a default, limiting user exposure. That mixed with its slower way of handling audio made it difficult for users to swallow.

A change in audio hit in 3.5-78, right after the release of Dolphin 3.5 late December 2012. Without having to worry about marring the release with a potentially buggy feature, delroth sprung one of the largest rewrites to Dolphin in years: New-AX-HLE. This not only rewrote and improved huge portions of the HLE audio system, but it also finally brought synchronous audio to DSP-HLE within the GameCube AX code.

The result was astounding; many games began outputting audio in DSP-HLE for the very first time, stability increased and the emulator was altogether faster because less titles needed DSP-LLE in order to run.

While this merge was sudden to Dolphin users, Delroth had already spent several months prior rewriting most of the AX HLE code. Of course, the first merge was only the beginning. His reverse engineering of the DSP resulted in him being able to write his own DSP-ROM that could be included with dolphin! For the first time, users could get through problematic games with LLE audio without needing to go through the trouble of dumping one from their console!

Then DSP-LLE became mostly unneeded; delroth's second AX-HLE merger hit in 3.5-1154, fixing up the problems from the first one while also adding support for Wii AX games. Hundreds of games saw huge improvements in stability, HLE audio became usable in popular features such as netplay, and Dolphin found itself in a much stronger position for the future.

Sometimes unexpected consequences completely alter how a fix is perceived. The developers expected New-AX-HLE to be one of the most heralded features added to Dolphin in years; it improved speed and accuracy equally. We thought users would celebrate these changes! However, several factors mixed together to make New-AX-HLE a controversial feature.

While not related to audio; Dolphin was going through a phase where it was getting slower due to hacks being removed and compatibility being raised. Games weren't running as fast as they were in older releases.

By far the biggest change in behavior was for users who couldn't run games fullspeed. The asynchronous behavior of old HLE allowed them to mostly ignore the slowdown because the game would sound reasonable unless audio outright crashed. The addition of synchronous audio processing took away that ability and made it very obvious that the game was lagging. Complaints mounted against the new way of handling HLE audio, and a small minority of users demanded the old system back.

The developers didn't just change audio overnight, though. The past months prior to merge were filled with discussions about what synchronous audio would bring to the emulator, and the best ways to transition. All that could be said is that the changes in behavior were intended; if the host machine could not run the emulator at 100%, sound should not be fullspeed. In order to guarantee stability in games, the audio must be synchronous with the action.

As you might expect, the latent explanation wasn't very convincing to the people that just wanted to play their games and didn't care how the emulator worked. What was thought to be one of the biggest features added to Dolphin ended up being one of its most controversial. Threads were started, emails sent, and it became very loud and agitated. To ease the process, delroth created an article to explain the changes in more detail, but this was long after the initial outrage.

As with any outcry against a new feature, there were a lot of very valid complaints mixed in with the unreasonable ones. Long-time issues in Dolphin's audio interface made the jump from LLE to HLE thanks to the two systems working in similar manners. This included a massive 200ms latency on audio and a strange buffer bug that caused audio distortion after ninety minutes of play! By 4.0-879 those bugs and several others were squashed.

3a8082e126