OGG/Vorbis decoder in Javascript

[Morten W. Petersen]

Hi there.

I thought I'd tell you all that I've worked a bit with Emscripten lately, and managed to port the

Xiph.org OGG/Vorbis demuxer and decoder using Emscripten.

The example is here:

http://blogologue.com/player/

It works in some browsers, but for example in Safari 5.1.7 (latest available on Windows),

I get an error:

INVALID_STATE_ERR: DOM Exception 11: An attempt was made to use an object that is not, or is no longer, usable.

Does anyone know how to debug that?

Regards,

Morten

[Floh]

I tested on desktop Safari 10.0.1 (exact version string: Version 10.0.1 (12602.2.14.0.6)) on OSX 10.12.1 Beta and everything works, may be the webpage uses a HTML5 feature that's been missing in older Safaris?

I have not tested on iOS Safari or Windows Safari.

Javascript console looks clean, just a few messages and also messages which Safari thinks are warnings:

[Warning] Comment: GENRE=Power pop, alternative (a.out.min.js, line 1)

[Warning] Comment: TITLE=Freshbeat with solo, bass and vocal (a.out.min.js, line 1)

[Warning] Comment: DATE=2016 (a.out.min.js, line 1)

[Warning] Comment: ARTIST=Morphex (a.out.min.js, line 1)

[Warning] Comment: COMMENTS=http://blogologue.com (a.out.min.js, line 1)

[Log] Synchronizing.. (a.out.min.js, line 1)

[Log] 0 (a.out.min.js, line 1)

[Log] 2 (a.out.min.js, line 1)

[Log] Worker said: 6005276 (player, line 421)

[Log] Uint8Array (player, line 422)

[Log] Done with page load (player, line 430)

[Warning] Synchronized seconds: 500 (a.out.min.js, line 1)

[Warning] Done. (a.out.min.js, line 1)

[Warning]  (a.out.min.js, line 1)

[Warning] Bitstream is 2 channel, 44100Hz (a.out.min.js, line 1)

Cheers,

-Floh.

[Morten W. Petersen]

Ah, excellent. :)

Maybe I won't have to debug this thing, if an old enough version of Safari works.

I googled Safari 5 and the audio tag, and it looks like that is supported, so my

guess is that it is some Javascript feature that is missing.

I don't know where to begin though, as the debug information from Safari doesn't

give any place to even start.

-Morten

[Brion Vibber]

On Sun, Oct 16, 2016 at 3:57 PM, Morten W. Petersen <mor...@gmail.com> wrote:

I thought I'd tell you all that I've worked a bit with Emscripten lately, and managed to port the

Xiph.org OGG/Vorbis demuxer and decoder using Emscripten.

The example is here:

http://blogologue.com/player/

Nice!

You might also want to check out my ogv.js project, which plays audio and video from Ogg and WebM files: https://github.com/brion/ogv.js

 

It works in some browsers, but for example in Safari 5.1.7 (latest available on Windows),

I get an error:

INVALID_STATE_ERR: DOM Exception 11: An attempt was made to use an object that is not, or is no longer, usable.

Does anyone know how to debug that?

Safari for Windows is long-abandoned and isn't representative of Safari on Mac or iOS, unfortunately. To test how things work for actual Safari users you'll need to track down a Mac or iOS device. I can confirm it seems to work in Safari Technical Preview on macOS 10.12.

-- brion

[Morten W. Petersen]

On Monday, October 17, 2016 at 5:43:31 PM UTC+2, Brion Vibber wrote:

On Sun, Oct 16, 2016 at 3:57 PM, Morten W. Petersen <mor...@gmail.com> wrote:

I thought I'd tell you all that I've worked a bit with Emscripten lately, and managed to port the

Xiph.org OGG/Vorbis demuxer and decoder using Emscripten.

The example is here:

http://blogologue.com/player/

Nice!

Thanks. :) 

You might also want to check out my ogv.js project, which plays audio and video from Ogg and WebM files: https://github.com/brion/ogv.js

Yes I did have a look at that as a possible solution for me, but I think I found it too big a project,

as I'm aiming squarely at audio.  At some point I'd like to make decoding a lot faster than what

it is now.

I'd like to hear how you handle decoding, buffering etc. though - as performance is acceptable

if 10-second chunks are decoded at a time for example.

 

 

It works in some browsers, but for example in Safari 5.1.7 (latest available on Windows),

I get an error:

INVALID_STATE_ERR: DOM Exception 11: An attempt was made to use an object that is not, or is no longer, usable.

Does anyone know how to debug that?

Safari for Windows is long-abandoned and isn't representative of Safari on Mac or iOS, unfortunately. To test how things work for actual Safari users you'll need to track down a Mac or iOS device. I can confirm it seems to work in Safari Technical Preview on macOS 10.12.

Yep.

I did find a list of WebKit-based browsers here:

https://en.wikipedia.org/wiki/List_of_web_browsers#WebKit-based

I think testing against one of those will be good enough for a while, as Floh reported the player did

work on Safari now (without me having done any testing against it).

-Morten

[Brion Vibber]

On Monday, October 17, 2016, Morten W. Petersen <mor...@gmail.com> wrote:

On Monday, October 17, 2016 at 5:43:31 PM UTC+2, Brion Vibber wrote:

On Sun, Oct 16, 2016 at 3:57 PM, Morten W. Petersen <mor...@gmail.com> wrote:

I thought I'd tell you all that I've worked a bit with Emscripten lately, and managed to port the

Xiph.org OGG/Vorbis demuxer and decoder using Emscripten.

The example is here:

http://blogologue.com/player/

Nice!

Thanks. :) 

You might also want to check out my ogv.js project, which plays audio and video from Ogg and WebM files: https://github.com/brion/ogv.js

Yes I did have a look at that as a possible solution for me, but I think I found it too big a project,

as I'm aiming squarely at audio.  At some point I'd like to make decoding a lot faster than what

it is now.

Yeah you can make some different trade offs if you're aimed at smaller audio clips. :)

I'd like to hear how you handle decoding, buffering etc. though - as performance is acceptable

if 10-second chunks are decoded at a time for example.

I'm decoding in realtime (a few packets at a time), maintaining about a 1 second buffer-ahead for audio. The output is done via Web Audio except on Internet Explorer which uses a Flash shim. This involves jumping through some hoops but works well for long streams and synchronization with the video.

Most of my logic is on the JavaScript side, with just demuxer and decoders in emscripten. For audio, decoding is driven by the Web Audio ScriptProcessorNode requesting additional data so this works even in a background tab (video currently uses timers which get throttled, though).

I'm also running decode operations in a Worker thread, so audio decode, video decode, and playback can run simultaneously. Workers are quite well supported these days, even on Internet explorer 11... This might be a good way to halve your decode time if you can split up the file fairly cleanly into alternating chunks -- dual-core CPUs are virtually everywhere now.

Feel free to follow up in more detail offlist if you like. :) I'm hoping to make the logic side a little lighter weight and make it work better for the short audio case too, but I can definitely see some advantages to the pre-decoded method you're using.

Safari for Windows is long-abandoned and isn't representative of Safari on Mac or iOS, unfortunately. To test how things work for actual Safari users you'll need to track down a Mac or iOS device. I can confirm it seems to work in Safari Technical Preview on macOS 10.12.

Yep.

I did find a list of WebKit-based browsers here:

https://en.wikipedia.org/wiki/List_of_web_browsers#WebKit-based

I think testing against one of those will be good enough for a while, as Floh reported the player did

work on Safari now (without me having done any testing against it).

Excellent... beware that iOS safari can be a little different than the rest in terms of media playback, so try to test it too!

-- brion

 

[Morten W. Petersen]

On Monday, October 17, 2016 at 6:26:39 PM UTC+2, Brion Vibber wrote:

On Monday, October 17, 2016, Morten W. Petersen <mor...@gmail.com> wrote:

On Monday, October 17, 2016 at 5:43:31 PM UTC+2, Brion Vibber wrote:

On Sun, Oct 16, 2016 at 3:57 PM, Morten W. Petersen <mor...@gmail.com> wrote:

I thought I'd tell you all that I've worked a bit with Emscripten lately, and managed to port the

Xiph.org OGG/Vorbis demuxer and decoder using Emscripten.

The example is here:

http://blogologue.com/player/

Nice!

Thanks. :) 

You might also want to check out my ogv.js project, which plays audio and video from Ogg and WebM files: https://github.com/brion/ogv.js

Yes I did have a look at that as a possible solution for me, but I think I found it too big a project,

as I'm aiming squarely at audio.  At some point I'd like to make decoding a lot faster than what

it is now.

Yeah you can make some different trade offs if you're aimed at smaller audio clips. :)

And keep it simple and easy to have a complete view of what's going on. :)

 

I'd like to hear how you handle decoding, buffering etc. though - as performance is acceptable

if 10-second chunks are decoded at a time for example.

I'm decoding in realtime (a few packets at a time), maintaining about a 1 second buffer-ahead for audio. The output is done via Web Audio except on Internet Explorer which uses a Flash shim. This involves jumping through some hoops but works well for long streams and synchronization with the video.

Most of my logic is on the JavaScript side, with just demuxer and decoders in emscripten. For audio, decoding is driven by the Web Audio ScriptProcessorNode requesting additional data so this works even in a background tab (video currently uses timers which get throttled, though).

I'm also running decode operations in a Worker thread, so audio decode, video decode, and playback can run simultaneously. Workers are quite well supported these days, even on Internet explorer 11... This might be a good way to halve your decode time if you can split up the file fairly cleanly into alternating chunks -- dual-core CPUs are virtually everywhere now.

Feel free to follow up in more detail offlist if you like. :) I'm hoping to make the logic side a little lighter weight and make it work better for the short audio case too, but I can definitely see some advantages to the pre-decoded method you're using.

OK.

Yes the project has been setup to work with chunks, but I opted for a quick and dirty

"decode everything" approach now just to see that things work.

I don't like Internet Explorer for different reasons, so I might not add support for it,

Edge I'm not sure about but I'm sceptical.  I see they are trying to follow the lead

from Chrome and match them, so maybe I'll just make sure it works with Chrome

and then let Edge support rely on that.

Safari for Windows is long-abandoned and isn't representative of Safari on Mac or iOS, unfortunately. To test how things work for actual Safari users you'll need to track down a Mac or iOS device. I can confirm it seems to work in Safari Technical Preview on macOS 10.12.

Yep.

I did find a list of WebKit-based browsers here:

https://en.wikipedia.org/wiki/List_of_web_browsers#WebKit-based

I think testing against one of those will be good enough for a while, as Floh reported the player did

work on Safari now (without me having done any testing against it).

Excellent... beware that iOS safari can be a little different than the rest in terms of media playback, so try to test it too!

Aha.  Well I might get away with a WebKit-based browser for most things, and then just

check that things work on the iPad, which is cheap enough used.

-Morten