[ANN] Sound support for script.aculo.us

[Thomas Fuchs]

Hi all,

I've added a new library called sound.js to script.aculo.us, which
allows for playing (mp3-based) sound effects. It works without using
Flash, by using IE's internal sound playback mechanism (BGSOUND) and
falling back on using whatever plugin is used for audio/mpeg on other
browsers (in real world terms, this means QuickTime or a QuickTime-
compatible plugin).

Thanks to Jules Gravinese of http://www.webveteran.com/ for coming up
with the initial idea and code for this.

Tested and works in IE7, FF2, Safari.

You can have a look at this by grabbing the latest SVN trunk, and
summoning the sound_test.html functional test file.

Note that this feature isn't meant for any heavy-duty music playback,
but just for little sound effects that enhance the user experience.
Some examples of where this can be useful:

- After a long upload, beep to let the user know it's done, even if
the browser window is not currently visible
- Notification sound for chats
- Beep on false password
- Warning beep for potential distrous operations
- Help messages/audio tutorials

The current implementation also supports tracks, which means you can
play sounds in parallel, or stop what's currently being played and
play a new sound. I'm not too sure this functionality is needed
(makes only sense for larger sounds, like narrations). I'm open to
discussion on this.

Anyway, it should be considered a bit experimental at this stage, so
any reports/bug fixes/enhancements are welcome, as always.

Best,
Thomas

[Jean-Philippe Encausse]

Why do you choose native browser support ?
- There's lots of version of customized Windows/IE that may not
support this feature.
- Installing QuickTime for Firefox is painfull.

Why don't you choose something like:

SoundManager 2: A Sound API for JavaScript
http://ajaxian.com/archives/soundmanager-2-a-sound-api-for-javascript

Which is:
- really crossbrowser
- suported since years
- specialized for sounds effects

Regards

--
Jean-Philippe Encausse - R&D Jalios SA
Jp [at] encausse.net - http://www.encausse.com - http://www.jalias.com
GTalk: jp.encausse [at] gmail.com - SMS: sms [at] jp.encausse.net
Mob: +33 6 82 12 56 99 Jalios: +33 1 39 23 92 83 Tel: +33 1 39 18 90 15
Do it Once, Use it Twice ~ Do it Twice, Make It Once

[Robert Wallis]

Thomas Fuchs wrote:
> Hi all,
>
> I've added a new library called sound.js to script.aculo.us, which
> allows for playing (mp3-based) sound effects. It works without using
> Flash, by using IE's internal sound playback mechanism (BGSOUND) and
> falling back on using whatever plugin is used for audio/mpeg on other
> browsers (in real world terms, this means QuickTime or a QuickTime-
> compatible plugin).
>
>
>

> Anyway, it should be considered a bit experimental at this stage, so
> any reports/bug fixes/enhancements are welcome, as always.
>

I think the Quicktime template on line 55 should have:
id="sound_#{track}_#{id}"
rather than:
id="#{id}"
?

Apart from that it appears to work very well.
I think using the default browser playback method as a fallback is a
good idea; but couldn't Flash be used if available? I struggle to see
why Quicktime gets special treatment but Flash does not, especially as
http://www.adobe.com/products/player_census/flashplayer/ would seem to
indicate Flash has considerably more use than Quicktime.

Robert Wallis

[Thomas Fuchs]

Flash vs. QuickTime:
I've a computer where Flash just doesn't produce sound :( -- whereas
QuickTime works fine.

The QuickTime plugin for Firefox comes free with iTunes, so for the
Win/FF combination chances are high it's there; plus on the Mac it
comes out of the box.

Also, it's choosing whatever plugin handles audio/mpeg, which may or
may not be Quicktime (although in the majority of cases, it will be).

For Flash support, we'd need an additional .swf file-- I guess that's
something that can be added and used as a fallback.

Btw, the SoundManager thing is just too overfeatured for me, I want
something small that uses what's there and reliable. YMMV. :)

Best,
Thomas

[Sébastien Grosjean - ZenCocoon]

Hi,

Great work Thomas !

For the use you specified just above this ultra light weight library
(about 2Ko) sound wonderful to me !

P.S. : I actually got a small bug on Win/FF2 line 28 (revision 6299)
--
Sébastien Grosjean - ZenCocoon

> >http://www.adobe.com/products/player_census/flashplayer/would seem to

[Robert Wallis]

Sébastien Grosjean - ZenCocoon wrote:
> P.S. : I actually got a small bug on Win/FF2 line 28 (revision 6299)
>

This is probably the same bug I was getting (it's on the same line), It
can be fixed as below, and I've submitted a patch.
http://dev.rubyonrails.org/ticket/7709

>>
>> Am 04.03.2007 um 21:10 schrieb Robert Wallis:
>>> I think the Quicktime template on line 55 should have:
>>> id="sound_#{track}_#{id}"
>>> rather than:
>>> id="#{id}"

Robert Wallis

[Thomas Fuchs]

Thanks a bunch, applied!

Now, unit tests would be great... *hinthint* ;)

Best,
Thomas

[Robert Wallis]

Thomas Fuchs wrote:
> Now, unit tests would be great... *hinthint* ;)

Sorry, it wasn't written in the best of ways to allow unit testing of
all the different methods of attaching the sound to the page, I've made
some modifications to allow more extensive unit testing and added them
in: http://dev.rubyonrails.org/ticket/7712
Hopefully setting the templates out in this way should make it easier to
see where and how to add different templates too, should future support
for Flash and such be put in.

Robert Wallis

[Michael Schuerig]

On Sunday 04 March 2007, Thomas Fuchs wrote:
> You can have a look at this by grabbing the latest SVN trunk, and  
> summoning the sound_test.html functional test file.

sound.js ends with a snippet of code that skeletized looks like this

(function(){
if(...){
if(...)
...
else
...
}
})();

I understand what the code does, but I don't understand why it does it
in this way. What's the point of the anonymous function when it is not
used to close over anything in its scope?

Michael

--
Michael Schuerig
mailto:mic...@schuerig.de
http://www.schuerig.de/michael/

[jules]

By your thinking, we might as well not even use Scriptaculous. Flash
does animation, too, right?

The point is to use no plugins at all - which is bgsound or embed. But
I made it quietly use a plugin if one is readily available. Which is
QT (handles every mac browser). If you're on a PC with FF and QT is
avail we use it - again, quietly with no prompts. In time to come
it'll fall back to WMV if QT is not present.

The absolute very last thing I wanted out of this is for the user to
be prompted to install a plugin or run an active-X.

-Jules

On Mar 4, 6:55 am, "Jean-Philippe Encausse" <J...@encausse.net> wrote:
> Why do you choose native browser support ?
> - There's lots of version of customized Windows/IE that may not
> support this feature.
> - Installing QuickTime for Firefox is painfull.
>
> Why don't you choose something like:
>

> SoundManager 2: A Sound API for JavaScripthttp://ajaxian.com/archives/soundmanager-2-a-sound-api-for-javascript

>
> Which is:
> - really crossbrowser
> - suported since years
> - specialized for sounds effects
>
> Regards
>

> On 3/4/07, Thomas Fuchs <t.fu...@wollzelle.com> wrote:
>
>
>
>
>
> > Hi all,
>
> > I've added a new library called sound.js to script.aculo.us, which
> > allows for playing (mp3-based) sound effects. It works without using
> > Flash, by using IE's internal sound playback mechanism (BGSOUND) and
> > falling back on using whatever plugin is used for audio/mpeg on other
> > browsers (in real world terms, this means QuickTime or a QuickTime-
> > compatible plugin).
>

> > Thanks to Jules Gravinese ofhttp://www.webveteran.com/for coming up

> > with the initial idea and code for this.
>
> > Tested and works in IE7, FF2, Safari.
>
> > You can have a look at this by grabbing the latest SVN trunk, and
> > summoning the sound_test.html functional test file.
>
> > Note that this feature isn't meant for any heavy-duty music playback,
> > but just for little sound effects that enhance the user experience.
> > Some examples of where this can be useful:
>
> > - After a long upload, beep to let the user know it's done, even if
> > the browser window is not currently visible
> > - Notification sound for chats
> > - Beep on false password
> > - Warning beep for potential distrous operations
> > - Help messages/audio tutorials
>
> > The current implementation also supports tracks, which means you can
> > play sounds in parallel, or stop what's currently being played and
> > play a new sound. I'm not too sure this functionality is needed
> > (makes only sense for larger sounds, like narrations). I'm open to
> > discussion on this.
>
> > Anyway, it should be considered a bit experimental at this stage, so
> > any reports/bug fixes/enhancements are welcome, as always.
>
> > Best,
> > Thomas
>
> --
> Jean-Philippe Encausse - R&D Jalios SA

> Jp [at] encausse.net -http://www.encausse.com-http://www.jalias.com

[Thomas Fuchs]

There was a var = ... declaration in earlier, so it's just a left-
over from that. :)

Best,
Thomas

[Sébastien Grosjean - ZenCocoon]

I made the functional tests on Opera 9.1 and all seem to work as
expected.

Hope this help,

[Christophe Porteneuve]

jules a écrit :

> The absolute very last thing I wanted out of this is for the user to
> be prompted to install a plugin or run an active-X.

AAMOF, on FF2 with no audio/mpeg plugin (I'm on Debian), I do get
prompted (in a sliding topbar) to install a plugin :-(

--
Christophe Porteneuve a.k.a. TDD
"[They] did not know it was impossible, so they did it." --Mark Twain
Email: t...@tddsworld.com

[Thomas Fuchs]

There probably needs a check added for this, or maybe just the "Win"
check removed-- care to investigate? :)

Best,
Thomas

[jules]

Removing the win check would be the simplest solution. Yeah there's
actually no reason at all to check the platform. We just need to know
if we have a plug-in installed or not. Mac has QT by default. Windows
might, if it's installed.

if IE, use bgsound
if Opera, use embed
if FF & if QT, use embed
if Safari, use embed

That should cover enough bases for now. I don't have access to Debian
for testing. But I think this fix should gracefully degrade for you.

Thomas, can you patch it? I don't know how to make modifications
with Trac/SVN... maybe you can you email me some quicks instructions?
(Mac/
BBEdit preferred or PC/UlraEdit) I looked but I can't find any!

Later we should fall back on Window Media, Real, WinAmp, VLC, etc. I
need to find out which loads and plays audio the quickest and create a
list of plugins ordered by preference.

-Jules

[Thomas Fuchs]

Am 06.03.2007 um 05:47 schrieb jules:

> I don't know how to made modifications
> with Trac/SVN... can you email me some quicks instructions? (Mac/
> BBEdit preferred or PC/UlraEdit)

Google is your friend: http://svnbook.red-bean.com/nightly/en/
svn.basic.html :)

> Later we should fall back on Window Media, Real, WinAmp, VLC, etc. I
> need to find out which loads and plays audio the quickest and create a
> list of plugins ordered by preference.

I don't think this is a good solution. This will just blow up the
code just to support some edge cases we can't test easily. The embed
tag already uses a audio/mpeg mimetype, so other plugins should be
able to pick that up. You'll also get into all sorts of version hell.
I feel spending time on improving the performance and stability of
what's already there is better invested. :)

First thing, we need proper unit tests (we only have the functional
test right now).

I *can* imagine the flash fallback, so we reach a huge majority of
users (namely, the last few % that are composed of FF/Win without
QuickTime).

Best,
Thomas

>
> -Jules
>
> On Mar 5, 6:59 pm, Thomas Fuchs <t.fu...@wollzelle.com> wrote:

--
Thomas Fuchs
wollzelle

http://www.wollzelle.com

questentier on AIM
madrobby on irc.freenode.net

http://www.fluxiom.com :: online digital asset management
http://script.aculo.us :: Web 2.0 JavaScript
http://mir.aculo.us :: Where no web developer has gone before

[jules]

I, and I'm sure others as well, use flashblock. So we'd end up with an
ugly play button floating in the web page if we were to fall back on
it. And then, the browser would have to download a flash movie to play
the sound. It seems kinda clumsy to me, but we'll see how it goes.

I personally don't like flash. OK... I hate flash actually. Until
intel macs came out, flash would cripple the system just to animate
some stupid vector based trendy ill designed eye candy. Blech.

Yes I agree about the bloat. For performance, we should play an empty
sound when sounds.js is loaded. That will warm up/cache whatever audio
playing mechanism we're to use later on.

Well damn it looks like I have a lot of learning to do with SVN...

> http://www.fluxiom.com:: online digital asset managementhttp://script.aculo.us:: Web 2.0 JavaScripthttp://mir.aculo.us:: Where no web developer has gone before

[Mark Holton]

Flash Player has a footprint on something like 98% of machines... pretty safe to fallback with Flash.

(this is really cool by the way, thanks again, Thomas-)

[Mark Holton]

here's a link:
http://www.adobe.com/products/player_census/flashplayer/

[jules]

You're missing the point. We don't WANT to use flash. Why bother with
script.aculo.us at all otherwise? Flash can do animations and ajax-
like functions as well.

When I set out to make a js audio engine, I wanted to do so with no
plugins whatsoever. Which is fine in IE and Opera. However gecko
dropped the support for playing wav natively in the browser... god
only knows why... but they did. So for them, we are forced to use a
plugin.

So, unless we can tell the flash plugin to play an audio file (that
is, not download a swf that plays audio) I don't see it as a good
idea. It's kinda like... giving in. And then it makes people wonder,
'well, why bother using script.aculo.us if we're gonna rely on flash
anyhow?'

On Mar 6, 1:46 pm, "Mark Holton" <holto...@gmail.com> wrote:
> here's a link:http://www.adobe.com/products/player_census/flashplayer/
>

> On 3/6/07, Mark Holton <holto...@gmail.com> wrote:
>
>
>
> > Flash Player has a footprint on something like 98% of machines... pretty
> > safe to fallback with Flash.
>
> > (this is really cool by the way, thanks again, Thomas-)
>

[Thomas Fuchs]

Am 10.03.2007 um 02:16 schrieb jules:

> So, unless we can tell the flash plugin to play an audio file (that
> is, not download a swf that plays audio) I don't see it as a good
> idea. It's kinda like... giving in. And then it makes people wonder,
> 'well, why bother using script.aculo.us if we're gonna rely on flash
> anyhow?'

I don't totally agree as we use flash too, for specialized stuff like
video playing.

For audio playback, especially just little sound effects, i find
having to have an additional flash file and something like UFO a bit
of an overkill, especially when native methods (or straight-forward
plugins) are around that work just fine too.

Just be pragmatic, use what works (in the sense of that the user is
happy and you're happy as a developer*). :)

Best,
Thomas

*I can tell you that messing around with .fla files (think
subversion!) isn't making me happy as a developer

[Kjell Bublitz]

I had the chance to test it now and here are some things i felt missing:

# I can not know the ID that is assigned automatically. A callback
would be great too!
# You continuously append to the DOM but you don't clean it up after
the sound was played.

To accomplish this we need to know when the sound stops. As far as i
know there are no cross-browser/player/whatever ways to have a
callback on the stop or finish event of the sound.

But we do know what files we play and how long there are. So a
possible workaround would be to add a duration option and use it for a
internal timeout to clean it up the DOM after the sound was played and
furthermore add a timed callback .. something like afterFinish. The
automatic cleaning should be optional and the callback should deliver
the ID of the element that was created.

Sound.play('bleep.mp3', {duration: 0.5, autoClean: false,
afterFinish:function( sound_id )
{
new Effect.Highlight('notify'); // after 500ms
$( sound_id ).remove();
}
});

What you think ?

--
Regards, Kjell
www.m3nt0r.de

[Kjell Bublitz]

Disregard the second item. I've just discovered replace: false ;)
But the callback/duration is something i would still like to see.

--
Regards, Kjell
www.m3nt0r.de