We've been experimenting using Autobahn/Websockets to send near
real-time multimedia data from small clients for rendering in HTML5.
This note shares a little bit of our results. Part of the reason for
this is to see if anyone else is doing anything similar.
HTML5 does not yet define a means for real-time continuous audio
transmission. When WebRTC is ratified, it will form the basis for
that kind of capability, but it is likely there will remain browser
incompatibilities. Javascript Media Source extensions are another
promising technology that when delivered will enable browsers to
assemble presentations of media (like audio and video) without
discontinuities or artifacts. The latest spec
shows collaboration between Google, Microsoft and Netflix. However,
this feature is only available on an experimental basis in a few
browsers.
Even without these emerging specifications, the current HTML5 browser
has interesting capabilities for multimedia presentation.
It has:
- websockets: a transport medium
- javascript: an efficient interpreter
- event scheduling: both in Javascript and AudioContext
- audio rendering: Audio elements, and the AudioContext
If a continuous audio broadcast is broken into a series of chunks, a
browser can re-assemble the sequence of audio chunks for playback with
timing that is close to the original. The resulting audio sometimes
has artifacts in the form of occasional clicks between the chunks.
However, it turns out that the quality is remarkably good. The quality is
browser-dependent, and it is interesting to compare the results across browsers.
Testing the capability of HTML5 to reassemble a continuous audio
broadcast required the development of three separate things:
1) a broadcasting application (a program to sample a microphone and
turn it into data for sending via HTTP)
2) a server (for receiving the audio data, potentially transcoding
audio for browsers that need it, and for sending through a websocket
to an HTML5 web-page)
3) a Javascript receiving application (HTML5 + Javascript)
The result of these experiments is the Wazwot iPhone App and its
associated web service.
How does Autobahn Websockets fit in?
Autobahn Websockets provides a Resource interface so that it is easy
to mix Websocket objects into a tree of HTTP Resources. This
capability simplified the construction of the server, since it needs
to handle many broadcasts simultaneously. In our implementation, each
broadcast "channel" is its own resource with its own HTML and
Websocket components.
We originally chose AB because of my familiarity with Twisted and
Python, but also because of some of the Twisted features it embraces.
In particular we needed flow-control (in the form of an
IPushProducer). Real-time audio transmission over a web socket needs
a means to monitor TCP back-pressure and adjust the stream. The
Autobahn/Twisted framework made it pretty easy to implement this
requirement.
Try it out:
Wazwot is basically a research project. It sends continuous audio and
also continuous image frames. If anyone on this list is interested in
trying it out, send me a note and I'll shoot you an IOS app promo code
for evaulation.