AudioSprite as complement to TextureAtlas

[Alex Gann]

After working with TexturePacker for the first time, I was wondering if there was an audio complement to the TextureAtlas concept.

Some Googling led me to AudioSprites, which are exactly what I was envisioning.

TexturePacker's audio analogue: https://github.com/tonistiigi/audiosprite

(looked like zynga had open-sourced a js library called jukebox that the audiosprite json format targeted, but seems to have since been close-sourced)

JS implementation that supports this concept: http://goldfirestudios.com/blog/104/howler.js-Modern-Web-Audio-Javascript-Library - one could easily map audiosprite json format to their method arguments.

Anyone have any experience?  Are there any stumbling blocks here compared to keeping sound files separate?  Could be a neat StageXL Runtime to add, a useful addition to ResourceManager, and if community is interested I would be wanting/willing to help as I could.

Thanks!

Alex

[Bernhard Pichler]

Hi Alex,

This is a nice idea and looks very interesting! I also looked at the implementation in howler.js and it looks pretty straight forward. I only have one concern, and this is browser compatibility. Audio is a big mess today. The WebAudio API will solve all of those problems in the future and it should be easy to implement something like this. Today it is even hard to get an event when an audio sample has finished playing (i opened an issue on the Chromium bug tracker for this). Firefox will bring WebAudio in a future version, the current implementation in their nightly builds looks great.

So i will keep an eye on this!

Bernhard

[Alex Gann]

Hi Bernhard!

I took another look at this and have prepared a pull-request which I just submitted to add SoundSprite functionality.

Functioning demo using SoundSprite here: http://realbluesky.github.io/DartPiano/build/web/

(Piano note samples from http://theremin.music.uiowa.edu/MISpiano.html)

I used sox to convert, trim silence, mono, truncate and fade the samples.

Then I used a newly forked node.js package: https://github.com/realbluesky/soundsprite - which stitches the samples together (with 0.25 second gaps) and generates compatible json file and primary audio formats.

In the pull request I know there is a bug with AudioElementSound and SoundSprite.  Would you please take a look?  If you let each sample finish playing then it runs fine, but if you try to start another sample while the first is still playing it seems to botch the AudioElement pool logic and reports the Object is no longer available or not able to be used.

Also, you'll notice I switched the Sound url parameter from String to dynamic.  The soundsprite.json format matches the one used by https://github.com/goldfire/howler.js/ and as such provides a list of the generated formats.  So instead of adding all supportedTypes and try/catching, it simply removes the non-supported types from the array list and goes from there.

From my testing this PR has no breaking change for the existing audio implementation, but you'll want to confirm :)

I hope you like it - if it needs some cleaning let me know.

Thanks!

Alex

[Alex Gann]

https://github.com/realbluesky/SoundDart - create SoundSprites with DartVM!

Only tested on Windows, so if anyone running Linux or OSX wants to try it out I'd appreciate it.

Fun stuff,

Alex

[Bernhard Pichler]

This is very cool! Thanks for all your efforts for bring audio sprites to StageXL!

[GregW]

I was wondering if anyone has had success with soundsprites in Internet Explorer 11? I'm having problems with the sounds starting at the wrong spot, finishing early and/or running into each other. I assume Internet Explorer 11 IS the problem, but hoping someone may have a work-around (other than not using it!). Thank you to both Bernhard and Alex for the work on this! Everything works fine with other browsers.

[Alex Gann]

Sorry Greg - no luck here - IE11 and WebAudio API do not play nice.  I tested my old demo http://realbluesky.github.io/DartPiano/build/web/ in Microsoft Edge though and it works fine - so there's at least one Microsoft browser that handles it :)

[GregW]

Thanks for confirming Alex. Yes, Edge handles everything great. I wish they would fix IE, but wish in one hand.. ;)

[Bernhard Pichler]

Edge works great because it supports WebAudioApi. With Internet Explorer we still have to use the AudioElement - which is bad because it does not load the audio file as a whole, but only a few seconds at the current playback position. With SoundSprites you have to play the audio file from different positions and therefore the browser starts loading the audio file every time you want to play a different segment.

Btw. i have a demo for SoundSprites, if you open the developer tools in the browser (F12) you can see how the audio file is loaded:

http://www.stagexl.org/temp/piano/index.html

There is one workaround which is not very practical but may work in some situations. You can convert your audio file to a base64 encoded DataUrl string. If you use this DataUrl instead of the http url, the SoundSprite probably works fine in InternetExplorer.

GregW <gwh...@gmail.com> schrieb am Do., 31. Aug. 2017 um 19:48 Uhr:

Thanks for confirming Alex. Yes, Edge handles everything great. I wish they would fix IE, but wish in one hand.. ;)

--
You received this message because you are subscribed to the Google Groups "StageXL" group.
Visit this group at https://groups.google.com/group/stagexl.
To view this discussion on the web visit https://groups.google.com/d/msgid/stagexl/1e21faa8-695b-480f-8469-3931a728b0f3%40googlegroups.com.

[GregW]

Thank you for the explanation Bernhard. Now I'm curious about converting as you mentioned! Not for the job I'm currently working on due to time constraints, but for the learning experience. 

The project I'm currently doing has 30 to 40 sounds and I thought Soundsprite would be great - and it worked exceptionally well in all browsers but IE. For now I'm going back to loading in .mp3's / ogg's individually.

Once again, I appreciate you & the effort you put into StageXL!