Simple WebRTC-SIP gateway

4,358 views
Skip to first unread message

Lorenzo Miniero

unread,
May 15, 2012, 11:16:10 AM5/15/12
to discuss-webrtc
Hi all,

lately I've tried playing a bit with the WebRTC/RTCWEB support in
Chrome, since I'd like to make it available as an alternative means to
access our standards-based conferencing service.

If you attended any recent IETF meeting, you might remember me as one
of the guys running back and forth from one room to another to provide
remote participation support for several sessions through our tool.
Being standards-based, we already make available several access points
(SIP, RTSP, PSTN, etc.) and the idea would be to add a simple WebRTC
access point as well for the meeting in Vancouver, which, besides
being nice for remote participants, I think could greatly help the
ongoing work as a real-world use case.

That said, I've started working on a very simple signalling gateway to
our SIP-based conferencing service, which is based on Asterisk. I've
already managed to bridge a call from a web page to a generic
extension, but not everything seems to work as expected, and so I
thought I'd share what I tried so far in order to get some feedback on
what I may be doing wrong.

For what concerns signalling/negotiation, I'm using the
roap_on_jsep.js code to access the devices and create the
PeerConnection object in a web page. I then send the generated SDP to
my application server, which modifies the SDP a bit before sending it
to Asterisk:

* First of all, it changes the RTP profile from SAVPF to SAVP, since
Asterisk doesn't handle SAVPF.

* Then, the AS cuts the video from the negotiation; in fact, having
both audio and video would be great, but we thought we'd focus on just
audio at first and then add video in a later stage. Besides, Asterisk
at the moment doesn't support VP8 passthrough, which is something we'd
need to add ourselves, together with VP8 support in our videomixer.
While we could have left it in there, we noticed that Chrome seems not
to like the "m=video 0..." stuff Asterisk sends back not being able to
handle it (an exception is triggered in the setRemoteDescription
method) so we chose to just remove it from the signalling manually.

Once the SDP is ready, the AS sends an INVITE to the requested
extension in Asterisk, and sets up the call: only A-law and mu-law
"survive" the negotiation process, but that's ok. Since we're using
Asterisk 1.8, SRTP is supported while ICE is not, although there is
some support for STUN (I had to modify it a bit to make it work using
SRTP as well, though). This means that, before sending the SDP from
Asterisk back to Chrome, the AS needs to manipulate it as well:

* It changes the SAVP Asterisk sends back to SAVPF.

* It adds a session-level "a:ice-lite" attribute to fake (?) an ICE-
Lite support, and a single candidate for audio corresponding to the
publicly reachable address Asterisk negotiated.

* It adds an "a=rtcp:" attribute to explicitly specify the RTCP port
Asterisk uses (I don't think RTCP muxing is supported yet).

At this point, the AS sends this SDP (as ROAP) back to the web page,
which passes it to the PeerConnection to have it processed.


Long story short (I know, too late), this last step seems to go fine,
since the PeerConnection generates an OK message that I send back to
the AS, but that's pretty much as far as it goes. I can see, using
Wireshark, that Chrome starts sending connectivity checks using STUN
to Asterisk, and that the ports are right: Asterisk apparently
correctly handles the Binding Requests Chrome sends, sending a Binding
Success Response back, but instead of starting to see media flow,
Chrome keeps on sending Binding Requests every 500ms, and just ignores
the SRTP packets Asterisk starts sending right away instead.

I'm pretty sure there is something I may have missed or done wrong,
e.g., either in the fake candidate preparation in the SDP or in the
Binding Success Responses Asterisk sends back, that causes the above
mentioned failures, but I can't seem to figure it out. I've read
Hadriel's draft on interworking, but it suggests there are several
things that could affect the success of a call (RTCP being another
potential cause), and I'm not sure what has and what hasn't been
already implemented in Chrome.

Does anyone have a clue on what/where may be the problem? I can
provide both message dumps and wireshark captures if that can help
debug the issue: I left snippets out of this message since it was
already long enough.

For the sake of completeness, the Chrome version I'm using is a Fedora
16 google-chrome-unstable, Version 20.0.1132.3 release 136408.

Thanks!
Lorenzo

Punyabrata Ray

unread,
May 15, 2012, 1:52:09 PM5/15/12
to discuss...@googlegroups.com
Hi Lorenzo,

After discussing this with the developers, your connection issues are most likely due to the fact
that our ICE implementation is not complete yet and is yet to conform to the RFC completely.
We are working hard on completing this.

With respect to the following issue " we noticed that Chrome seems not

to like the "m=video 0..." stuff Asterisk sends back not being able to
handle it (an exception is triggered in the setRemoteDescription
method ", can you open an issue and attach the Wireshark stream
so that we can reproduce this on our end?

Thank you.
- pr

Lorenzo Miniero

unread,
May 16, 2012, 5:43:47 AM5/16/12
to discuss-webrtc
Hi Punyabrata,

thanks for your prompt reply!

I'm aware the ICE implementation may not be complete, just as I'm
aware my side is probably not doing it right either. Knowing what the
Chrome implementation expects right now from the media would help me
align the server side to it, though, so that's why I asked.

About the video=0 proble, by opening an issue do you mean here?
http://code.google.com/p/webrtc/issues/list
If so, I'll open one right away.
Just for the interest of the group, here's the message that causes the
issue (where 1.2.3.4 is the public address of the PBX):

{
"answererSessionID": "jlyHHq6qP5H7Zl6IjY2Jxr9K4BBt8voE",
"messageType": "ANSWER",
"offererSessionID": "104",
"sdp": "v=0\r\no=root 1236077627 1236077627 IN IP4 1.2.3.4\r
\ns=Asterisk PBX 1.8.11.1\r\nc=IN IP4 1.2.3.4\r\nt=0 0\r\na=ice-lite\r
\nm=audio 13416 RTP/SAVPF 0 8 126\r\nc=IN IP4 1.2.3.4\r\na=rtcp:13417
IN IP4 1.2.3.4\r\na=candidate:0 1 udp 2130706432 1.2.3.4 13416 typ
host network_name eth0 generation 0\r\na=rtpmap:0 PCMU/8000\r
\na=rtpmap:8 PCMA/8000\r\na=rtpmap:126 telephone-event/8000\r\na=fmtp:
126 0-16\r\na=ptime:20\r\na=sendrecv\r\na=crypto:1
AES_CM_128_HMAC_SHA1_80 inline:9PAm0Octg6mhwlx7judm02hSOj9DQxx+0oWuFAag
\r\nm=video 0 RTP/AVP 100 101 102\r\n",
"seq": 1
}

As you can see, Asterisk sets the video port to 0 since it doesn't
support VP8 (and you can see the other attributes I mentioned in my
previous post too). This triggers an exception when calling this
method:

this.peerConnection.setRemoteDescription(this.peerConnection.SDP_ANSWER,
new RoapConnection.SessionDescriptionConstructor(msg.sdp));

specifically, a "SYNTAX_ERR: DOM Exception 12" error. Whether the
cause is setRemoteDescription or the SessionDescriptionConstructor
constructor, though, I don't know.

Lorenzo


On 15 Mag, 19:52, Punyabrata Ray <punyabr...@webrtc.org> wrote:
> Hi Lorenzo,
>
> After discussing this with the developers, your connection issues are most
> likely due to the fact
> that our ICE implementation is not complete yet and is yet to conform to
> the RFC completely.
> We are working hard on completing this.
>
> With respect to the following issue " we noticed that Chrome seems not
> to like the "m=video 0..." stuff Asterisk sends back not being able to
> handle it (an exception is triggered in the setRemoteDescription
> method ", can you open an issue and attach the Wireshark stream
> so that we can reproduce this on our end?
>
> Thank you.
> - pr
>

Zack Coder

unread,
May 17, 2012, 7:55:24 AM5/17/12
to discuss...@googlegroups.com
In addition to responding to incoming Bind Requests, you need to issue your own ones (requests, not just responses) directed at the WebRTC client in order to establish bi-directional communications. Don't be surprised to keep getting Binding Requests from Chrome - it will continue issuing them for the duration of the session (and you are expected to do the same).

Lorenzo Miniero

unread,
May 17, 2012, 10:23:12 AM5/17/12
to discuss-webrtc
Hi Zack

actually I didn't work on that since an ICE Lite implementation (what
my current AS acts like, right now) only needs to respond to
connectivity checks, and not generate some of its own. I guess that's
part of the work still under progress in Chrome to fully align and
adhere to the ICE standard, though, as I read in a few other posts, so
that may be where my application fails to comply.

Thanks for pointing that out, I'll try to modify the PBX
implementation to have it generate connectivity checks too and see if
that fixes it.

Lorenzo

Zack Coder

unread,
May 17, 2012, 10:45:47 AM5/17/12
to discuss...@googlegroups.com
Yes, you are right it is not usually expected of ice-lite, but as you said it isn't full compliant yet :)

Lorenzo Miniero

unread,
May 18, 2012, 7:55:32 AM5/18/12
to discuss-webrtc
Unfortunately it didn't work either... both Chrome and Asterisk send
each other Binding Requests and get successful Binding Responses, but
the only one sending media still is Asteris. Chrome apparently ignores
or drops the incoming SRTP packets, since the PeerConnection object
never gets its onaddstream callback invoked anyway: I guess this means
something is wrong on my side besides the previously missing
connectivity checks.

Any clue?

Thanks,
Lorenzo

Mamadou

unread,
May 18, 2012, 9:04:56 AM5/18/12
to discuss-webrtc
Do you have responses to your STUN binding requests?
> > > > > > > mentioned...
>
> read more »

Lorenzo Miniero

unread,
May 18, 2012, 9:54:21 AM5/18/12
to discuss-webrtc
Yes, Chrome replies with a Binding Success Response to every request I
make.
> ...
>
> leggi tutto

Mamadou

unread,
May 18, 2012, 10:58:02 AM5/18/12
to discuss-webrtc
I guess you are also returning success responses with mapped address.

- There are mandatory fields in the ICE candidates (rtp, network_name
username...) which can cause the issue even if you receive STUN 2xx.
- The CNAME in the video part was also mandatory

Would be helpful if you can share with us network trace (wireshark)
with both SIP and STUN pkts.
> > > > > > > > > Long story short (I know, too late), this last...
>
> read more »

Lorenzo Miniero

unread,
May 18, 2012, 12:02:04 PM5/18/12
to discuss-webrtc
Yes, the success responses contain the mapped address. Both requests
and responses look the same (except for the username, of course)
either sent by Chrome or Asterisk (I had to add the "magic" cookie for
that too), so I don't think there's anything missing from that point
of view.

I changed the candidate format as well to make it align to what Chrome
uses, and the username is coherent with what is used in connectivity
checks. My version uses "udp" and not "rtp", though.

For what concerns video, as I anticipated a few posts ago until now
I've cut it out of the negotiation, because of the issue I opened on
the webrtc Issues trac. Today I've started working to add a
passthrough support for VP8 in Asterisk, so as soon as that will be
ready I'll try enabling video and setting the CNAME part as well. For
the same reason I can't share a wireshark trace right now (the code is
not working at the moment) but I'll share one first thing on monday
morning.
> ...
>
> leggi tutto

Mamadou

unread,
May 18, 2012, 12:32:06 PM5/18/12
to discuss-webrtc
"rtp" was not about the transport but the value of the attribute
"name" (first on after "typ")
> > > > > > > > > >        * It...
>
> read more »

Lorenzo Miniero

unread,
May 18, 2012, 2:58:03 PM5/18/12
to discuss-webrtc
If I recall correctly (I don't have access to the dumps here at home)
I don't have any rtp in the candidates line, neither in what Chrome
generates, nor in what I add to the Asterisk SDP (whose format I
copied from the Chrome one anyway). In both cases, I have a " ... typ
host network_name eth0 username ...".

Is my dev Chrome version too old/new? I remember older versions of the
WebRTC-enabled Chrome using "rtp" and "rtcp" in candidates lines, but
they disappeared in more recent ones like the ones I'm using now.

Lorenzo
> > > > > > > > > > > "survive"...
>
> leggi tutto

Lorenzo Miniero

unread,
May 21, 2012, 6:26:22 AM5/21/12
to discuss-webrtc
Hi,

a quick update: IT WORKS! :)

First of all I updated my Chrome dev release as any morning (version
20.0.1132.11, release 137611), but that didn't seem to do anything
special.

Then I modified the Asterisk code to make it generate and respond to
bind requests for the RTCP connection too, since that bit was missing.
At this point I noticed from Wireshark that Chrome had started to send
SRTP packets too: I don't know if it was because of the software
update or the RTCP connection handling connectivity checks, but I saw
them, and that was enough for me.

Audio was still not working, though, and after a bit of debugging I
found out that was because of a problem in how Asterisk currently
handles crypto attributes in the SDP: apparently, Asterisk just takes
the first crypto line it finds and uses that for the negotiation,
despite of the algorythm. The problem is the first line Chrome sends
for audio is related to AES_CM_128_HMAC_SHA1_32, while Asterisk
replies with a AES_CM_128_HMAC_SHA1_80, obviously causing the
authentication to fail. I just had my Application Server remove the
AES_CM_128_HMAC_SHA1_32 line from the SDP before sending it to
Asterisk, and now audio works fine!

Or, to be more precise, I could hear the playback Asterisk was playing
in my browser: I still have not tried if Asterisk can get the frames
Chrome sends just as fine (I'll try this later in the day, I can't see
a reason why that should fail) but that's a very good start
nevertheless.

What puzzles me, though, is that I could hear the audio without
apparently having it in any element in the web page. I have two video
elements in my page: copying the approach the apprtc demo uses, I have
one video element where I put the local MediaStream (FYI, local video
is green in my case, I opened an issue on the webrtc track about that)
and another video element that is supposed to (dis)play the remote
MediaStream as soon as it is available. This remote video element is
never updated, though, and the onaddstream PeerConnection callback
never invoked, and yet I could hear the audio (which was definitely
coming from the webpage since, as soon as I closed it, audio stopped).
Am I wrong in assuming this relation between the media and something
"physical" in the HTML page?

I'll keep you updated on the gateway progress,
Lorenzo
> ...
>
> leggi tutto

Harald Alvestrand

unread,
May 21, 2012, 6:31:00 AM5/21/12
to discuss...@googlegroups.com
On Mon, May 21, 2012 at 12:26 PM, Lorenzo Miniero <lmin...@gmail.com> wrote:
Hi,

a quick update: IT WORKS! :)

Great!
A <video> element will play back audio, too, if it's present in the incoming stream.
(an early version of WebRTC just played back incoming audio without bothering with sending it to the element, but I think that's fixed at this point.)

Lorenzo Miniero

unread,
May 21, 2012, 7:12:32 AM5/21/12
to discuss-webrtc
Hi Harald,

in my case the only video element with a valid url is the one
associated with the local media stream (the remote one is actually
never set up), so, considering I hear the audio anyway, I'm probably
using a slightly older version of the browser. That's probably because
I'm on Linux and using the "google-chrome-unstable" repository on
Fedora, which I guess is not the bleeding-edge version of Chrome, even
if recent enough.

I'll try making video support available in the PBX as well to check
whether that gets displayed too or not.

Lorenzo


On 21 Mag, 12:31, Harald Alvestrand <h...@google.com> wrote:

Harald Alvestrand

unread,
May 21, 2012, 7:42:10 AM5/21/12
to discuss...@googlegroups.com
I checked ... we haven't gotten around to hooking up the sound to the <video> element yet, so what you're seeing is what you should expect.

Sorry for the misinformation!

Lorenzo Miniero

unread,
May 21, 2012, 9:01:20 AM5/21/12
to discuss-webrtc
No problem! It's nice to know my Chrome version is not as old as I
feared :)

Lorenzo Miniero

unread,
May 28, 2012, 7:09:20 AM5/28/12
to discuss-webrtc
Just a quick update. I managed to add a simple VP8 passthrough support
in Asterisk. Actually what it does as of now is accept VP8 as a video
codec, and as a consequence successfully negotiate it: I haven't
tested if the passthrough really works, though. At least this had the
onaddstream callback invoked, and the peer <video> tag element
properly set up, so it's a start.

For now, though, I can see from Wireshark that Chrome doesn't send any
video RTP packet: the STUN connectivity checks go fine, and I can see
RTCP Receiver Reports generated by Chrome, but no RTP packets at all.
I guess this may be because Asterisk is not generating any RTCP
packets by itself, probably because it is not sending RTP packets
either. I'll try to have Asterisk play out a pre-recorded VP8 stream,
to see if this causes it to generate RTCP packets too and maybe fix
the problem. According to the current Chrome implementation, would
this behaviour be enough or is more expected from the peer with
respect to that?

Lorenzo

Lorenzo Miniero

unread,
Jun 13, 2012, 10:42:51 AM6/13/12
to discuss-webrtc
Since I managed to get everything related to audio working just fine
(which has also been integrated in our Meetecho web client, so you
should all be able to attend the next IETF meeting sessions using
WebRTC!), I started focusing on video now.

As I anticipated, I managed to implement a simple passthrough mode for
VP8 in Asterisk, which, according to Wireshark, seems to work. In fact
the version of Chrome I have installed (1171), unlike the ones I had
before, sends video now, so I could finally test it. Attaching the
call to the Asterisk echo test application I can see those video
frames being echoed back, but they are not displayed in the web page.
I tried launching Chrome in debug mode to see if there were any errors
in the log, but could find nothing: Jingle seems to create both the
connections just fine (Connection created). Asterisk also sends RTCP
packets related to video now, but it looks like neither Asterisk nor
Chrome add the CNAME to any of the RTCP packets they send.

Any hint on what could be missing?

Lorenzo

Justin Uberti

unread,
Jun 13, 2012, 4:28:08 PM6/13/12
to discuss...@googlegroups.com
CNAME should be present, but what do require it for?

Lorenzo Miniero

unread,
Jun 14, 2012, 1:53:01 AM6/14/12
to discuss-webrtc
Actually I thought Chrome required it: an older post in this thread
told it was mandatory for the video part (or was it in the SDP only?).
IF that's not the case, I'll try to look at the log for a normal peer
to peer call and see if there's anything the log for my scenario is
missing.


On 13 Giu, 22:28, Justin Uberti <jube...@google.com> wrote:
> CNAME should be present, but what do require it for?
>

Lorenzo Miniero

unread,
Jun 21, 2012, 6:12:08 AM6/21/12
to discuss-webrtc
Apparently the echo test scenario works now: I don't know if it's
because of the browser update (using r1180 now) or because of the
small change I made in the PBX extension, but I could echo both audio
and video back.

I modified the scenario so that the echo test starts immediately after
a beep sound, when you call: this way it works fine. Using the default
echo test scenario Asterisk provides in its demo instead, that is, a
20 seconds audio playback introducing the scenario followed by the
actual echo test, it doesn't. Apparently, the missing incoming video
for those 20 seconds causes the browser to ignore/drop it when it
starts arriving: it may be related to the fact that Asterisk doesn't
send any RTCP packet until it starts sending frames, which for video
only happens when the echo test starts. Probably not a big deal, but I
thought you might be interested in having this information since it
may help you debugging similar scenarios.

Time to work on generating some video on my own, now :)

Lorenzo

Lorenzo Miniero

unread,
Jun 26, 2012, 7:29:35 AM6/26/12
to discuss...@googlegroups.com
Great news, I managed to add support to VP8 to our videomixer! So now both audio and video work fine.

This triggers another question, though: is it possible, using the current MediaStream API, to choose some parameters like framerate/resolution for the video to capture? We use a lower resolution video in our platform since the focus is assumed to be more on the other features (slides, chat, etc.), and so we'd rather have sources generate lower resolusion video streams as well, since we have to transcode/downsize the active ones in order to mix them all. And handling several 640x480@30 videos at the same time can be a burden! If not directly via MediaStream, can this be changed via JSEP at the time?

Thanks,
Lorenzo

Lorenzo Miniero

unread,
Jun 27, 2012, 6:34:22 AM6/27/12
to discuss...@googlegroups.com
Hi Dmitriy,

as I explained in one of my previous posts, it's the signalling gateway application I have deployed at the moment that simply "rewrites" that SAVPF to SAVP before sending it to Asterisk, as Asterisk doesn't currently handle that profile. I do the opposite when providing the browser with the answer. I guess you might try doing the same directly in JavaScript.



Il giorno mercoledì 27 giugno 2012 11:38:29 UTC+2, Dmitriy Shagin ha scritto:
Hi Lorenzo,

I'm trying to call from SIPML5 client to other SIMPL5 client via Asterisk. And I have one question: how can I change RTP/SAVPF profile to profile which Asterisk works with?

Lorenzo Miniero

unread,
Jun 28, 2012, 5:06:19 AM6/28/12
to discuss...@googlegroups.com
Dmitriy,

what headers? it's really as simple as I have described it. You need to change the SDP generated by the browser, specifically the profile to use in each m-line. That said, I don't know what this SIMLP5 thing does about signalling and negotiation: I had to do many more things to get audio and video working with Asterisk, especially on the server side, so just changing the profile will probably not be enough in your case.

Lorenzo


Il giorno mercoledì 27 giugno 2012 14:53:41 UTC+2, Dmitriy Shagin ha scritto:
Lorenzo,

Could you please send me invite messages before your gateway application and after? I just can't understand which headers I should change. 

среда, 27 июня 2012 г., 14:34:22 UTC+4 пользователь Lorenzo Miniero написал:

Mamadou

unread,
Jun 28, 2012, 10:25:48 AM6/28/12
to discuss-webrtc

Justin Uberti

unread,
Jun 29, 2012, 2:56:20 PM6/29/12
to discuss...@googlegroups.com
There is no current way to change the resolution of the encoded streams, but this definitely something we plan to add in the not-too-distant future. We just need to figure out how it should be represented in the API.


--

Lorenzo Miniero

unread,
Jul 3, 2012, 2:07:00 AM7/3/12
to discuss...@googlegroups.com
Thanks for your clarification Justin.
By API I assume you're talking about JSEP instead of MediaStream, right?

Lorenzo Miniero

unread,
Jul 24, 2012, 4:47:08 AM7/24/12
to discuss...@googlegroups.com
Well, I made quite a lot of changes that didn't involve only Asterisk, so summarizing them might be quite a challenge! I guess the best way to understand the required steps is actually going through all the posts: I know it may be boring, but I tried to document as well as possible everything I tried, both successes and failures, so I'm sure it will help you with your efforts as well.

If I really had to summarize, this would be what I had to do:
  • in the web application gateway, handle the SIP side and play with the SDP going back and forth, e.g., fixing the extra space in some SDP lines, changing SAVPF to SAVP and viceversa, adding the required SDP attributes that Asterisk wouldn't put (a:rtcp, a=ice-lite, the candidates and so on), and fixing the crypto lines (removing the SHA1 32 one); I guess this could be done directly in Asterisk, but I wanted to keep things as simple as possible for signalling, and focus on the media challenges instead;
  • in Asterisk, implement connectivity checks by extending the existing support for STUN (correctly handling usernames, and sending STUN requests as soon as one is received, to name one) and adding them for RTCP as well; for audio nothing else was required, since Chrome and Asterisk have at least a codec in common (G.711, not the best one but it works) and we didn't work on adding iSAC support yet;
  • for video, we also needed to add a passthrough implementation of VP8 in Asterisk in order to have it negotiated and the frames handled; we also added support for VP8 in our videomixer (the component we delegate videomixing to) and support for RTCP FIR in Asterisk to request a key frame to active peers.
That said, I don't think you'd make much use of a patch: there are a few hacks here and there that need a cleaner implementation, and besides it's a bit adapted to our needs. Besides, Digium developers have already started working on providing native support for WebRTC in Asterisk:


I don't know whether or not anything is already available or not, but it probably soon will so I guess you'd better wait for that to come out for a more stable and reliable solution.

Hope that helped!
Lorenzo


Il giorno lunedì 23 luglio 2012 00:14:32 UTC+2, Shaheryarkh S. Sheikh ha scritto:
Lorenzo, can you please summarize changes you done in Asterisk source code to make it working with chrome. Actually i am trying to run a simple ECHO test from my chrome browser to Asterisk. The call successfully establishes but there is no media, sent or received by chrome and call eventually dies due to RTP inactivity. Whereas if I use a normal SIP client e.g. QJSimple, it works fine.

I would appreciate if you can give me a patch for asterisk source code to try out.

Thank you.


Muhammad Shahzad

unread,
Jul 24, 2012, 4:55:00 AM7/24/12
to discuss...@googlegroups.com
Thank you very much Lorenzo, I really appreciate it. I will try to follow your guidelines to make it working.

Thank you.


--
 
 
 



--
Muhammad Shahzad
-----------------------------------
CISCO Rich Media Communication Specialist (CRMCS)
CISCO Certified Network Associate (CCNA)
Cell: +92 334 422 40 88
MSN: shari...@hotmail.com
Email: shahe...@googlemail.com
Reply all
Reply to author
Forward
0 new messages