Datachannels not supported

[Neil Young]

My node app controls the KMS via 8888. I’m opening the webRtcEndpoint with data channels enabled. At least I hope so:

let kurentoClient = await kurento('ws://localhost:8888/kurento', { failAfter: 5 }) let pipeline = await kurentoClient.create('MediaPipeline') let webRtcEndpoint = await pipeline.create('WebRtcEndpoint', {useDataChannels: true})

Later, before making KMS generating an offer I call

await webRtcEndpoint.connect(webRtcEndpoint) await webRtcEndpoint.createDataChannel() console.log(await webRtcEndpoint.generateOffer()) await webRtcEndpoint.gatherCandidates()

I expected to see the data channel pattern in the offer, but instead I get:

Trace: Error: Data channels are not supported at /home/ubuntu/kms-test/node_modules/kurento-client/lib/KurentoClient.js:365:24 at Object.dispatchCallback [as callback] (/home/ubuntu/kms-test/node_modules/kurento-jsonrpc/lib/index.js:546:9) at processResponse (/home/ubuntu/kms-test/node_modules/kurento-jsonrpc/lib/index.js:667:15) at RpcBuilder.decode (/home/ubuntu/kms-test/node_modules/kurento-jsonrpc/lib/index.js:723:5) at Stream.transportMessage (/home/ubuntu/kms-test/node_modules/kurento-jsonrpc/lib/index.js:208:10) at Stream.emit (events.js:314:20) at Stream.EventEmitter.emit (domain.js:483:12) at drain (/home/ubuntu/kms-test/node_modules/through/index.js:36:16) at Stream.stream.queue.stream.push (/home/ubuntu/kms-test/node_modules/through/index.js:45:5) at WebsocketStream.onMessage (/home/ubuntu/kms-test/node_modules/websocket-stream/index.js:45:15) { code: 40111, data: { type: 'MEDIA_OBJECT_OPERATION_NOT_SUPPORTED' } } at Object.noop (/home/ubuntu/kms-test/node_modules/kurento-client-elements/lib/WebRtcEndpoint.js:35:22) at callback2 (/home/ubuntu/kms-test/node_modules/promisecallback/index.js:27:25) at processTicksAndRejections (internal/process/task_queues.js:97:5) at async test (/home/ubuntu/kms-test/index.js:42:9)

I can’t find the origin of this error in the sources.

[Neil Young]

Uhh, that looks ugly. Here again...

[Neil Young]

Hmm. Anybody having a pointer? Is this scenario not supposed to work at all? I know the sample https://doc-kurento.readthedocs.io/en/stable/tutorials/js/tutorial-helloworld-datachannels.html , but this is a client code sample and the data channel is opened on the peerconnection object there, not on a webrtcendpoint object.

[Juan Navarro]

Hi Neil,

I think you should first establish the SDP Offer/Answer, and then create the DataChannel communication. Could you try to do that?

According to the docs, https://doc-kurento.readthedocs.io/en/latest/_static/client-jsdoc/module-elements.WebRtcEndpoint.html#.createDataChannel

Being supported means that the WebRtcEndpoint has been created with data channel support, the client also supports data channels, and they have been negotiated in the SDP exchange. Otherwise, the method throws an exception, indicating that the operation is not possible.

So I'd expect that SDP negotiation has been done first.

Regards,
Juan

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/50544ec7-56db-441c-87e8-dc4c12e9bef8n%40googlegroups.com.

[Neil Young]

Hi Juan,

Yes I have read this passage too. My test app is very simple, it doesn't even send back an answer to KMS after having received the result of generateOffer.

But I was having an older version of it, which was playing back a canned Chrome offer. And indeed, now the attempt to createDataChannel no longer causes this error, just if I explicitly disable or not enable data channels on creation of webRtcEndpoint.

Should I receive an event on successful open of the channel? I have placed an event handler for the opening, but this is not triggered. Am I right to assume, that this is for the receive channel only?

webRtcEndpoint.on('DataChannelOpen', (event) => {

            console.log('DataChannelOpen')        

})

The documentation I've found doesn't state that so explicitly.

Thank you very much for now. One question drives me around, didn't find an answer: Do you know if a sending data channel is repeated N times if we have the following scenario?

1) One sendonly source sending video and audio to KMS and opening a data channel to send

2) Multiple recvonly sinks are attached to this one source on KMS in order to view the video and listen to the audio. Would all those sinks also receive the data sent by the one source?

Regards

[Neil Young]

For a short moment I thought "OnDataChannelOpened" would be the right event, but it doesn't fire either

[Juan Navarro]

"OnDataChannelOpened" and "DataChannelOpen" are the same exact event, you can use either. The former is deprecated, though, so you should better use the latter one.

I've checked the code and seems that this event should be fired whenever a DataChannel ACK is either sent or received. So I don't think it will trigger immediately, but when the channel is actually established.

Regarding the last question, I'm not sure of your objectives so best thing would be to test and see if it works. Whatever you find, if you feel like doing so, I'd encourage you to describe your scenario and the result of the tests, and I'll probably document your findings so they don't get lost and can be helpful for others who come later to the same or similar issue.

If connecting multiple sinks directly to a single WebRtcEndpoint source doesn't work, there is an alternative that I know for a fact that works: using the Composite element. However, this element seems to be problematic for audio and video; maybe you'd be able to use it only for data. There is a demo app for Java (but it should be easy to translate into other languages) here: https://github.com/Kurento/kurento-demos-java/tree/master/composite-datachannels

(let me know if you want to run that demo and are not familiar with how to run Spring-Boot Java applications)

Regards

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/ca95288c-85de-42d2-a68d-745524a2f9een%40googlegroups.com.

[Neil Young]

Ah, makes sense. What I can say for sure is, that I was able to have one "sendonly" video source and multiple "recvonly" video sinks (no audio). That worked fine supported by my node server: I kept the webrtcEndpoints of the "sendonly" connections and when a "viewer" was connected I looked up the list of "presenters" by that unique key and connected the newly created recvonly webrtcEndpoint with the found webrtcEndpoint. If I remember correctly then the hint for doing it like so came from you :)

I'm just not sure what happens with the datachannel that time. Say the "presenter" (the one with the sendonly connection) has a data channel open and sends into it. It would be lovely, if each connected "viewer" would now get an indication that a data channel is present and would receive, what the "presenter" has to say :)

[Neil Young]

@Juan Navarro: Would it be possible to resume this discussion? Even though I get that "DataChannelOpen" callback now, it doesn't seem to work properly ATM.

Here is the scenario:

- Node JS app, controlling KMS

- WebApp, utilizing Node JS app, providing three "roles": "controller", "publisher", "subscriber"

- "controller" makes "publisher" send video to KMS by sending a "publish" command to the Node app (into a specific signaling server room)

- The Node JS app now triggers KMS to generate an offer, manages between the "publisher" the exchange of SDP and ICE. The publisher sends video

- The same with the subscriber(s): The Node JS app does literally the same for the subscribers, just that there is no new "WebRTCEndpoint" is created, but it is plumbed to the publisher's pipeline.

This scenario works fine for video and most likely also for audio (switched off). I would like to have a 1toN distribution of the video.

Now I thought, it would be great, if the "publisher" could open a data channel and send data to all subscribers.

But this seems to not work.

What I'm doing is:

- I'm creating the publisher's webRTCEndpoint

webRtcEndpoint = await pipeline.create('WebRtcEndpoint', { useDataChannels: true })

- I'm waiting for the MediaStateChanged event of the WebRTCEndpoint, once it goes from DISCONNECTED to CONNECTED. Then I call

await webRtcEndpoint.createDataChannel()

- I'm getting the "DataChannelOpen" callback now

- In the webApp, on the "publisher" part (publisher and subscribers are enabled and have all the things setup for datachannels), I'm getting these notifications, which directly follow the peerConnection callbacks for datachannels

WebRTC notification event: Send data channel opened

WebRTC notification event: Receive data channel opened

WebRTC notification event: Receive data channel closed

- The same happens if I launch a subscriber. 

- In the end "publisher" and "subscriber" have their send datachannels open, but they can't reach each other, because their receive data channel just shortly popped up and disappeared again.

Is there anything special with the receive channel? Setting it up like so:

this.pc.addEventListener('datachannel', this.onPCReceiveChannelCallback)

// Receive data channel setup

onPCReceiveChannelCallback = (event) => {

  this.receiveChannel = event.channel

  this.receiveChannel.binaryType = 'arraybuffer'

  this.receiveChannel.onmessage = this.onPCReceiveMessageCallback

  this.receiveChannel.onopen = this.onPCReceiveChannelStateChange

  this.receiveChannel.onclose = this.onPCReceiveChannelStateChange

}

Please assist

[Neil Young]

For completenes: The OFFER, KMS generates and the ANSWER of the publisher, data channel bolded

OFFER

v=0

o=- 3843980723 3843980723 IN IP4 0.0.0.0

s=Kurento Media Server

c=IN IP4 0.0.0.0

t=0 0

a=group:BUNDLE video0 application0

m=video 1 RTP/SAVPF 97

a=setup:actpass

a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time

a=rtpmap:97 H264/90000

a=fmtp:97 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f

a=rtcp:9 IN IP4 0.0.0.0

a=rtcp-mux

a=recvonly

a=mid:video0

a=rtcp-fb:97 nack

a=rtcp-fb:97 nack pli

a=rtcp-fb:97 ccm fir

a=ssrc:2047874495 cname:user3165847263@host-31c7b05d

a=ice-ufrag:w1EK

a=ice-pwd:3ha/yhO5PUfPVu2Gmj6jN6

a=fingerprint:sha-256 1C:32:12:EE:95:F3:81:02:1F:85:BF:B3:4D:47:48:E6:CC:25:B6:F7:A2:12:36:A4:DB:A4:D3:BC:1B:EB:AA:FB

m=application 1 UDP/DTLS/SCTP webrtc-datachannel

a=setup:actpass

a=sctp-port:5000

a=mid:application0

a=ice-ufrag:w1EK

a=ice-pwd:3ha/yhO5PUfPVu2Gmj6jN6

a=fingerprint:sha-256 1C:32:12:EE:95:F3:81:02:1F:85:BF:B3:4D:47:48:E6:CC:25:B6:F7:A2:12:36:A4:DB:A4:D3:BC:1B:EB:AA:FB

ANSWER

v=0

o=- 4648508826119915713 2 IN IP4 127.0.0.1

s=-

t=0 0

a=group:BUNDLE video0 application0

a=msid-semantic: WMS x2TYIZlCVcO7YUV2I9ZPB41U8WKpFnIqr5Gn

m=video 9 RTP/SAVPF 97

c=IN IP4 0.0.0.0

a=rtcp:9 IN IP4 0.0.0.0

a=ice-ufrag:ryGC

a=ice-pwd:KhYDLnoKU03w78siqmt2BRK5

a=ice-options:trickle

a=fingerprint:sha-256 88:54:FC:EE:27:AB:50:09:8B:54:CA:5C:9E:48:F5:C9:3D:EA:7E:52:B6:8C:39:FA:FA:C1:77:DC:84:CE:74:75

a=setup:active

a=mid:video0

a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time

a=sendonly

a=msid:x2TYIZlCVcO7YUV2I9ZPB41U8WKpFnIqr5Gn 592ab20b-199d-406f-a67d-f46fee67ee8a

a=rtcp-mux

a=rtpmap:97 H264/90000

a=rtcp-fb:97 ccm fir

a=rtcp-fb:97 nack

a=rtcp-fb:97 nack pli

a=fmtp:97 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f

a=ssrc:2123792928 cname:NYvgJxGD7inabBY1

a=ssrc:2123792928 msid:x2TYIZlCVcO7YUV2I9ZPB41U8WKpFnIqr5Gn 592ab20b-199d-406f-a67d-f46fee67ee8a

a=ssrc:2123792928 mslabel:x2TYIZlCVcO7YUV2I9ZPB41U8WKpFnIqr5Gn

a=ssrc:2123792928 label:592ab20b-199d-406f-a67d-f46fee67ee8a

m=application 9 UDP/DTLS/SCTP webrtc-datachannel

c=IN IP4 0.0.0.0

a=ice-ufrag:ryGC

a=ice-pwd:KhYDLnoKU03w78siqmt2BRK5

a=ice-options:trickle

a=fingerprint:sha-256 88:54:FC:EE:27:AB:50:09:8B:54:CA:5C:9E:48:F5:C9:3D:EA:7E:52:B6:8C:39:FA:FA:C1:77:DC:84:CE:74:75

a=setup:active

a=mid:application0

a=sctp-port:5000

[Neil Young]

Oh, wait, I have it: I need to be careful, what channels are to be opened. If I just enable the "sendchannel" on the publisher's webapp (not the receive channel) and I only enable the "receive" channel on the "subscriber's" web app, then the data send by the publisher appears on all N subscribers. Great achievement!!

Thanks. Seems to be a no issue

[Neil Young]

Hi @Juan Navarro,

I would like to ask you again for some pointers. In general data channels are working fine as reported, but only if browsers communicate with KMS.

Here a successful back and forth of OFFER (KMS) and ANSWER (Chrome), in which the connection comes through:

KMS offer

v=0

o=- 3847248857 3847248857 IN IP4 0.0.0.0

s=Kurento Media Server

c=IN IP4 0.0.0.0

t=0 0

a=group:BUNDLE video0 application0

m=video 1 RTP/SAVPF 98 97

b=AS:5000

a=setup:actpass

a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time

a=rtpmap:97 VP8/90000

a=rtpmap:98 H264/90000

a=fmtp:98 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f

a=rtcp:9 IN IP4 0.0.0.0

a=rtcp-mux

a=recvonly

a=mid:video0

a=rtcp-fb:97 nack

a=rtcp-fb:97 nack pli

a=rtcp-fb:97 goog-remb

a=rtcp-fb:97 ccm fir

a=rtcp-fb:98 nack

a=rtcp-fb:98 nack pli

a=rtcp-fb:98 ccm fir

a=ssrc:2705769100 cname:user3007764115@host-7b28121d

a=ice-ufrag:G8Fm

a=ice-pwd:5BHNcNkrqq6yKjg4ujdHWN

a=fingerprint:sha-256 F9:13:D2:08:F1:78:3E:30:76:BA:64:B8:D8:73:BE:41:24:6A:81:05:41:1B:6B:77:0D:4B:87:14:F5:95:EC:D6

m=application 1 UDP/DTLS/SCTP webrtc-datachannel

a=setup:actpass

a=sctp-port:5000

a=mid:application0

a=ice-ufrag:G8Fm

a=ice-pwd:5BHNcNkrqq6yKjg4ujdHWN

a=fingerprint:sha-256 F9:13:D2:08:F1:78:3E:30:76:BA:64:B8:D8:73:BE:41:24:6A:81:05:41:1B:6B:77:0D:4B:87:14:F5:95:EC:D6

Browser answer

v=0

o=- 8828963304090885480 2 IN IP4 127.0.0.1

s=-

t=0 0

a=group:BUNDLE video0 application0

a=msid-semantic: WMS wTMxSlaG4zqFncuadz0nqkljmEcDc6Bv0KiM

m=video 9 RTP/SAVPF 98 97

c=IN IP4 0.0.0.0

a=rtcp:9 IN IP4 0.0.0.0

a=ice-ufrag:SWXG

a=ice-pwd:Dm2rsvmEaG3GdrPAk8NCyVlc

a=ice-options:trickle

a=fingerprint:sha-256 2F:4D:19:7C:C7:A7:2C:B3:D3:F3:55:8D:1A:71:9D:8B:F7:34:17:C8:FD:C6:59:30:86:94:15:61:14:F6:A1:48

a=setup:active

a=mid:video0

a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time

a=sendonly

a=msid:wTMxSlaG4zqFncuadz0nqkljmEcDc6Bv0KiM bc9dd45b-11ff-4d61-b5b1-e22c73bae19f

a=rtcp-mux

a=rtpmap:98 H264/90000

a=rtcp-fb:98 ccm fir

a=rtcp-fb:98 nack

a=rtcp-fb:98 nack pli

a=fmtp:98 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f

a=rtpmap:97 VP8/90000

a=rtcp-fb:97 goog-remb

a=rtcp-fb:97 ccm fir

a=rtcp-fb:97 nack

a=rtcp-fb:97 nack pli

a=ssrc:2875837727 cname:xFuroYp0P4qPdxGf

a=ssrc:2875837727 msid:wTMxSlaG4zqFncuadz0nqkljmEcDc6Bv0KiM bc9dd45b-11ff-4d61-b5b1-e22c73bae19f

a=ssrc:2875837727 mslabel:wTMxSlaG4zqFncuadz0nqkljmEcDc6Bv0KiM

a=ssrc:2875837727 label:bc9dd45b-11ff-4d61-b5b1-e22c73bae19f

m=application 9 UDP/DTLS/SCTP webrtc-datachannel

c=IN IP4 0.0.0.0

a=ice-ufrag:SWXG

a=ice-pwd:Dm2rsvmEaG3GdrPAk8NCyVlc

a=ice-options:trickle

a=fingerprint:sha-256 2F:4D:19:7C:C7:A7:2C:B3:D3:F3:55:8D:1A:71:9D:8B:F7:34:17:C8:FD:C6:59:30:86:94:15:61:14:F6:A1:48

a=setup:active

a=mid:application0

a=sctp-port:5000

Now there is GStreamer WebRTCBin on the other end of KMS. The first problem I had was, that WebRTCBin and KMS don't understand each other, if H.264 is favoured and KMS is the OFFERER. If WebRTCBin OFFERS, the connection can be established. I have not found a reason for it, so I worked around and always have WebRTCBin OFFERING. This works now and I can benefit from some cameras with hardware codecs.

Now I wanted to add data channel support to WebRTCBin and followed their examples. The data channel is announced in the OFFER, but the connection to KMS can again not be established.

WebRTCBin offer

v=0

o=- 5030842061397511384 0 IN IP4 0.0.0.0

s=-

t=0 0

a=ice-options:trickle

a=group:BUNDLE video0 application1

m=video 9 UDP/TLS/RTP/SAVPF 96

c=IN IP4 0.0.0.0

a=setup:actpass

a=ice-ufrag:ZePUzznQsflLBA6FvHgznoDN2XsWsTz8

a=ice-pwd:K+AnDCjgXcBNbSh4EwUDQ9B4/uMOK4u6

a=rtcp-mux

a=rtcp-rsize

a=sendonly

a=rtpmap:96 H264/90000

a=rtcp-fb:96 nack pli

a=framerate:30

a=fmtp:96 packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=J0LAKJWgFAFugHiRNQ==,KM4fIA==

a=ssrc:1834330678 msid:user2798051488@host-b43d348b webrtctransceiver0

a=ssrc:1834330678 cname:user2798051488@host-b43d348b

a=mid:video0

a=fingerprint:sha-256 BF:17:BE:10:CD:AD:B3:35:00:14:3B:9B:69:25:6D:80:C0:CD:4B:16:43:9B:60:7F:12:1A:C7:65:00:42:5F:EE

m=application 0 UDP/DTLS/SCTP webrtc-datachannel

c=IN IP4 0.0.0.0

a=setup:actpass

a=ice-ufrag:ZePUzznQsflLBA6FvHgznoDN2XsWsTz8

a=ice-pwd:K+AnDCjgXcBNbSh4EwUDQ9B4/uMOK4u6

a=bundle-only

a=mid:application1

a=sctp-port:5000

a=fingerprint:sha-256 BF:17:BE:10:CD:AD:B3:35:00:14:3B:9B:69:25:6D:80:C0:CD:4B:16:43:9B:60:7F:12:1A:C7:65:00:42:5F:EE

KMS answer

v=0

o=- 3847251339 3847251339 IN IP4 0.0.0.0

s=Kurento Media Server

c=IN IP4 0.0.0.0

t=0 0

a=ice-options:trickle

a=group:BUNDLE video0

m=video 1 UDP/TLS/RTP/SAVPF 96

a=recvonly

a=mid:video0

a=rtcp:9 IN IP4 0.0.0.0

a=rtpmap:96 H264/90000

a=rtcp-fb:96 nack pli

a=setup:active

a=rtcp-mux

a=framerate:30

a=fmtp:96 packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=J0LAKJWgFAFugHiRNQ==,KM4fIA==

a=ssrc:1684329792 cname:user2830080048@host-f82d9351

a=ice-ufrag:mXv9

a=ice-pwd:7Mypzgr9drqqIXXanzkwAU

a=fingerprint:sha-256 F9:13:D2:08:F1:78:3E:30:76:BA:64:B8:D8:73:BE:41:24:6A:81:05:41:1B:6B:77:0D:4B:87:14:F5:95:EC:D6

m=application 0 UDP/DTLS/SCTP webrtc-datachannel

a=inactive

a=mid:application1

a=ice-ufrag:mXv9

a=ice-pwd:7Mypzgr9drqqIXXanzkwAU

a=fingerprint:sha-256 F9:13:D2:08:F1:78:3E:30:76:BA:64:B8:D8:73:BE:41:24:6A:81:05:41:1B:6B:77:0D:4B:87:14:F5:95:EC:D6

What I see is, that KMS is setting it "inactive", just I can't figure out, why.

Do you have a clue, what to examine?

There are some indications, so for instance the GStreamer issues this error on processing the ANSWER:

0:00:15.313615029  1276 0x7284a4f0 ERROR              webrtcbin gstwebrtcbin.c:4333:_set_description_task:<webrtcbin> media 1 is missing or contains an empty 'setup' attribute

This seems to be a reaction to the "a=inactive". 

The KMS does not reach MediaState:Connected, instead it traces:

2021-11-30T08:55:39.294Z silly: Publisher: IceComponentStateChange state CONNECTING, streamId 1, componentId 1

2021-11-30T08:55:39.294Z silly: Publisher: IceComponentStateChange state CONNECTING, streamId 1, componentId 2

2021-11-30T08:55:46.511Z silly: Publisher: IceComponentStateChange state FAILED, streamId 1, componentId 1

2021-11-30T08:55:46.513Z silly: Publisher: IceComponentStateChange state FAILED, streamId 1, componentId 2

Lost a bit...

[Neil Young]

The only think I see in the KMS logs is:

kmssdpagent.c:1568 create_media_answer() <KmsSdpAgent@0xffff6001bcd0>  Not negotiated media offered with port set to 0

Is this the reaction to "m=application 0"?

[Neil Young]

Well, indeed. If I blindly change "m=application 0" to "m=application 9" then the connection can be established...

What the heck??

[Neil Young]

OK, the rejection of "m=application 0" is obviously in accordance with the RFC https://datatracker.ietf.org/doc/html/rfc3264

However, no browser cares...

[Neil Young]

... and after all: An attempt to send into this channel from the offerer side (webrtcbin) fails. No trace where the data is going. It works peer-to-peer with a browser as counterpart. Not with KMS.

[Neil Young]

OK, if I _omit_ the attempt to create a data channel from my (the python) side it works. I'm just reacting on the incoming data channel from KMS and can send data upstream.

Solved