[Q&A] Unexpected ERROR encountered with 3rd-party TURN and cross platform usages !!

[Richard PENG]

Hi,

I have an application trying to exchange data/stream between iOS APP and JS Web via dataconnection/mediaconnection. Everything is good except when external(not turn.skyway.io) TURN server is needed. Several scenarios have been performed to troubleshoot; outlined as the following.

SCENARIO A: options.turn = true

Result is OK.

SCENARIO B: options.turn = false with no options.config at all

SCENARIO C: options.turn = false with options.config set to numb.viagenie.ca, both STUN and TURN(supplied with correct username/credential that have been verified on https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/)

Both results from B & C are FAILED with same symptom.

1) JS Web peer connects to and sends its peer.id to iOS APP peer.

2) iOS APP peer calls JS Web peer.

3) JS Web does get peer.on('call') and call.answer(localStream) and call.on('stream'), but encounters below error right away.

peer.js:739 Uncaught TypeError: Cannot read property 'addIceCandidate' of undefined

  Negotiator.handleCandidate @ peer.js:739

  MediaConnection.handleMessage @ peer.js:351

  Peer._handleMessage @ peer.js:1053

  (anonymous function) @ peer.js:875

  emit @ peer.js:1971

  _socket.onmessage @ peer.js:1369

4) iOS APP peer sees "error Optional(Network message=ICE connection failed. syserr=)" after several seconds.

Q1: Why mediaconnection.pc is undefined and "Optional(Network message=ICE connection failed. syserr=)"?

Q2: With SCENARIO B, why dataconnection can still be established even though no TURN is specified?

Q3: What bothers me even more is that result between peers on the same platform turns out to be OK. If both sides are JS Web peers, everything is good; both iOS APP peers, good as well. Why cross platform usages behave differently?

Thanks a lot.

Cheers,

Richard

[Richard PENG]

Hi,

After turning on the debug mode on JS Web and reading logs below, I have one more question to clarify.

PeerJS:  SkyWay TURN Server is unavailable

PeerJS:  Socket open

PeerJS:  startConnection from DataConnection

PeerJS:  Creating RTCPeerConnection.

PeerJS:  Listening for ICE candidates.

PeerJS:  Listening for `negotiationneeded`

PeerJS:  Listening for data channel

PeerJS:  Listening for remote stream

PeerJS:  `negotiationneeded` triggered

PeerJS:  Created offer.

PeerJS:  Set localDescription: offer for: WEjC8T1TGkiFUmbu

PeerJS:  Generated ICE candidate for: WEjC8T1TGkiFUmbu RTCIceCandidate {candidate: "candidate:2011871526 1 udp 2122260223 192.168.44.1…765 typ host generation 0 ufrag Kkpf network-id 1", sdpMid: "data", sdpMLineIndex: 0}

PeerJS:  Generated ICE candidate for: WEjC8T1TGkiFUmbu RTCIceCandidate {candidate: "candidate:963200470 1 tcp 1518280447 192.168.44.16…ptype active generation 0 ufrag Kkpf network-id 1", sdpMid: "data", sdpMLineIndex: 0}

PeerJS:  Generated ICE candidate for: WEjC8T1TGkiFUmbu RTCIceCandidate {candidate: "candidate:3144124085 1 udp 1686052607 114.136.64.2… rport 64765 generation 0 ufrag Kkpf network-id 1", sdpMid: "data", sdpMLineIndex: 0}

PeerJS:  Generated ICE candidate for: WEjC8T1TGkiFUmbu RTCIceCandidate {candidate: "candidate:3433799847 1 udp 41885439 66.228.45.110 …3 rport 6881 generation 0 ufrag Kkpf network-id 1", sdpMid: "data", sdpMLineIndex: 0}

PeerJS:  ICE candidates gathering complete for: WEjC8T1TGkiFUmbu

PeerJS:  Setting remote description RTCSessionDescription {type: "answer", sdp: "v=0

↵o=- 8354801686297600149 2 IN IP4 127.0.0.1

↵s…id:data

↵a=sctpmap:5000 webrtc-datachannel 1024

↵"}

PeerJS:  Set remoteDescription: ANSWER for: WEjC8T1TGkiFUmbu

PeerJS:  Added ICE candidate for: WEjC8T1TGkiFUmbu

PeerJS:  Data channel connection success

// above is OK on data connection; after this point, mediaconnection is invoked by the other end--iOS APP--with peer.call()

PeerJS:  startConnection from MediaConnection

PeerJS:  MediaConnection created in OFFER

Uncaught TypeError: Cannot read property 'addIceCandidate' of undefined(…)

PeerJS:  startConnection from answer step

PeerJS:  Creating RTCPeerConnection.

PeerJS:  Listening for ICE candidates.

PeerJS:  Listening for `negotiationneeded`

PeerJS:  Listening for data channel

PeerJS:  Listening for remote stream

PeerJS:  Setting remote description RTCSessionDescription {type: "offer", sdp: "v=0

↵o=- 5704071456554484345 2 IN IP4 127.0.0.1

↵s…slabel:ARDAMS

↵a=ssrc:2747919655 label:ARDAMSv0

↵"}

PeerJS:  `negotiationneeded` triggered

PeerJS:  onnegotiationneeded triggered when not stable. Is another connection being established?

PeerJS:  Set remoteDescription: OFFER for: WEjC8T1TGkiFUmbu

PeerJS:  Received remote stream

PeerJS:  Receiving stream MediaStream {id: "ARDAMS", active: true, onactive: null, oninactive: null, onaddtrack: null…}

-- local audio track added

PeerJS:  Created answer.

PeerJS:  Set localDescription: answer for: WEjC8T1TGkiFUmbu

PeerJS:  Generated ICE candidate for: WEjC8T1TGkiFUmbu RTCIceCandidate {candidate: "candidate:2011871526 1 udp 2122260223 192.168.44.1…431 typ host generation 0 ufrag iRsS network-id 1", sdpMid: "audio", sdpMLineIndex: 0}

PeerJS:  Generated ICE candidate for: WEjC8T1TGkiFUmbu RTCIceCandidate {candidate: "candidate:963200470 1 tcp 1518280447 192.168.44.16…ptype active generation 0 ufrag iRsS network-id 1", sdpMid: "audio", sdpMLineIndex: 0}

PeerJS:  Generated ICE candidate for: WEjC8T1TGkiFUmbu RTCIceCandidate {candidate: "candidate:3144124085 1 udp 1686052607 114.136.64.2… rport 64431 generation 0 ufrag iRsS network-id 1", sdpMid: "audio", sdpMLineIndex: 0}

PeerJS:  Generated ICE candidate for: WEjC8T1TGkiFUmbu RTCIceCandidate {candidate: "candidate:3433799847 1 udp 41885439 66.228.45.110 …3 rport 6547 generation 0 ufrag iRsS network-id 1", sdpMid: "audio", sdpMLineIndex: 0}

PeerJS:  ICE candidates gathering complete for: WEjC8T1TGkiFUmbu

PeerJS:  iceConnectionState is disconnected, closing connections to WEjC8T1TGkiFUmbu

PeerJS:  Cleaning up PeerConnection to WEjC8T1TGkiFUmbu

PeerJS:  DataChannel closed for: WEjC8T1TGkiFUmbu

Q4: With error's stacktrace--Negotiator.handleCandidate(this, payload.candidate); on MediaConnection.handleMessage @ peer.js:351--and above execution sequence--Uncaught TypeError encountered prior to "Set localDescription: answer for"--combined, is it valid that candidates gathering starts before setLocalDescription()?

Thanks again for any help you can provide.

Cheers,

Richard

[Richard PENG]

Hi,

One more thing to mention. From logs from iOS APP, it shows that a bunch of candidates were sent via signaling channel before sending its local SDP. Thus,

Q5: As there is no chance to look into the source code of iOS client library, can any of you gentlemen confirms that pc.createOffer() and pc.setLocalDescription() DOES complete sending local SDP before sending candidates inside pc.onicecandidate()?

Many thanks!!

Cheers,

Richard

[Richard PENG]

Hi,

I pretty much can figure out the problematic flow that causes me this trouble, depicted below.

1. iOS APP sends candidates before SDP on the same CONNECTION_ID.

2. JS Web gets the candidate with CONNECTION_ID without being able to find the cached connection. (/lib/peer.js line 275)

3. JS Web thus saves the candidate for later resolution. (/lib/peer.js line 282)

4. iOS APP then sends SDP containing CONNECTION_ID.

5. JS Web gets the offer with CONNECTION_ID and constructs the connection. (/lib/peer.js line 238)

6. JS Web prepares the mediaconnection without setting up RTCPeerConnection as there is no local stream yet. (/lib/negotiator.js line 25) & (/lib/mediaconnection.js line 23)

7. JS Web gets the offer with CONNECTION_ID and tries to process all saved candidates. (/lib/peer.js line 262)

8. JS Web fails to get RTCPeerConnection supposed to be there within the mediaconnection for processing the saved candidate. (/lib/negotiator.js line 342)

According to JSEP IETF draft, would it be mandatory that step 4 takes place before step 1?

Cheers,

Richard

[iwase.yoshimasa]

Hi, 

Thanks writing the detail report. I can't tell every answer as of now but I'll do it as possible as I can.

Q2: With SCENARIO B, why dataconnection can still be established even though no TURN is specified?

Do the both iOS APP and JS App connected to the same network? (e.g. within the same Wifi Access Point)  If so, the apps don't need STUN and TURN server because they have the local host candidates(typically private IP addresses).

Q4: With error's stacktrace--Negotiator.handleCandidate(this, payload.candidate); on MediaConnection.handleMessage @ peer.js:351--and above execution sequence--Uncaught TypeError encountered prior to "Set localDescription: answer for"--combined, is it valid that candidates gathering starts before setLocalDescription()?

The candidate gathering should start after setLocalDescription().

According to JSEP IETF draft, would it be mandatory that step 4 takes place before step 1?

AFAIK, JSEP defines the order of which APIs are called but doesn't define the order of messages sent. Because the order of the messages SENT to each other on some signaling channel can be inverted depending on the network condition, the order must be taken care of by application - in this case SkyWay SDK should care if the out-of-order transmission happens.

I think there might be some bugs on the library. We'd like to reproduce the problem based on the procedure and the condition you wrote but it'll takes some time, sorry.

[Richard PENG]

Hi,

Thanks for the reply. Comments inlined below.

Cheers,

Richard

On Tuesday, November 15, 2016 at 6:56:16 AM UTC+8, iwase.yoshimasa wrote:

Hi, 

Thanks writing the detail report. I can't tell every answer as of now but I'll do it as possible as I can.

Q2: With SCENARIO B, why dataconnection can still be established even though no TURN is specified?

Do the both iOS APP and JS App connected to the same network? (e.g. within the same Wifi Access Point)  If so, the apps don't need STUN and TURN server because they have the local host candidates(typically private IP addresses).

 

No, iOS with 4G and JS with LAN via wifi. However though, I figured out that inside the SDK there is a default setting to stun:stun.skyway.io:3478, which explains why RTCPeerConnection can still be established even with options.turn=false and no options.config.

Q4: With error's stacktrace--Negotiator.handleCandidate(this, payload.candidate); on MediaConnection.handleMessage @ peer.js:351--and above execution sequence--Uncaught TypeError encountered prior to "Set localDescription: answer for"--combined, is it valid that candidates gathering starts before setLocalDescription()?

The candidate gathering should start after setLocalDescription().

Exactly, setLocalDescription() should take place first, followed by candidates gathering; this's also what I think it should be.

According to JSEP IETF draft, would it be mandatory that step 4 takes place before step 1?

AFAIK, JSEP defines the order of which APIs are called but doesn't define the order of messages sent. Because the order of the messages SENT to each other on some signaling channel can be inverted depending on the network condition, the order must be taken care of by application - in this case SkyWay SDK should care if the out-of-order transmission happens.

I've also created an issue here, provided with logs from iOS APP. FYR.

 

I think there might be some bugs on the library. We'd like to reproduce the problem based on the procedure and the condition you wrote but it'll takes some time, sorry.

Thanks again for your efforts spent; look forward to hearing from you gentlemen soon.

[Yusuke NAKA]

Hi, 

Thanks for relay.

Exactly, setLocalDescription() should take place first, followed by candidates gathering; this's also what I think it should be.

1) the order of API calls (ex. setLocalDescription() before candidate gathering starts)

2) the order of messages SENT to the network

3) the order of messages RECEIVED from the network

As you said, JSEP defines 1). But as far as we know, it doesn't define/guarantee 2), 3).

So in this case we think that the receiver-side (SkyWay JS SDK) should care about the out-of-order messages.

SkyWay JS SDK is made considering receive of the out-of-order messages, but there might be some bugs.

Thanks again for your efforts spent; look forward to hearing from you gentlemen soon.

We're sorry but we have to lower a priority for it, because there are no errors occurred in the case of using our SkyWay TURN server.

So we would like to strongly recommend to use our SkyWay TURN server.

This function (using external STUN/TURN servers) is provided mainly for experimental use, so we could not support about the detailed behavior of our SDKs in the case of using it. (We will state it clearly in our documents.)

We really regret to say that we cannot help you.

Your kind understanding would be much appreciated.