KMS receives incorrect packets and can't decode video from GStreamer

[Miguel Ángel Exposito Sanchez]

Hi,

I have an app running the following GStreamer pipeline:

webrtcbin name=webrtcbin stun-server=stun://stun.l.google.com:19302

videotestsrc is-live=true ! videoconvert ! video/x-raw, format=NV12 !

videorate ! video/x-raw, framerate=25/1 !

videoscale ! video/x-raw, width=800,height=480 ! 

queue !

x264enc bitrate=4000000 speed-preset=ultrafast tune=zerolatency key-int-max=15 !

video/x-h264,profile=constrained-baseline ! queue max-size-time=100000000 !

h264parse ! 

video/x-h264, stream-format=byte-stream, alignment=nal !

rtph264pay config-interval=-1 aggregate-mode=zero-latency !

application/x-rtp,media=video,encoding-name=H264,clock-rate=90000,payload=96 !

webrtcbin.

On the other side I have a KMS instance and an application server running the one2many-call tutorial (java version). Both are running on the same machine.

If I run my app on the same machine as the server (Linux Desktop) everything works fine.

However I'm getting some odd behavior when I run the app on an Android (7.1) device connected to the same local network as the server:

If the Android device is connected to the LAN through Wi-Fi, then the session starts but I get no video on the web clients. KMS logs complain about invalid packets:

(See on Pastebin: https://pastebin.com/YQgKk7g5)

Connecting to an external server deployed on a cloud provider results in the same behavior.

However, if the Android device is directly connected to the server through USB using the USB Tethering function from Android, everything works well.

I generated DOT diagrams of the good and bad case pipelines but couldn't spot any differences.

A quick Wireshark inspection revealed two things:

- For some reason, ICE chooses TCP over UDP

- The size of the packets out of the Android device greatly differs in both cases, While in the USB tethering mode, the TCP payloads never exceed 1478 bytes

, but via WLAN the packet size can get up to 19 KB

Here's the SDP and ICE data exchanged between both Android device and server in both cases:

USB Tethering (everything works):

SPD offer / Answer: https://pastebin.com/qu9yujFB

ICE candidates: https://pastebin.com/Ra5Ki2RW

Both Android and Server via local WLAN (no video on viewers):

SPD offer / Answer: https://pastebin.com/KF6LE2rM

ICE candidates: https://pastebin.com/Vxdw5Dp4

I've been struggling with this for quite some time now, so any help would be greatly appreciated!

Thanks!

[Neil Young]

Pipeline looks ok. ICE and SDP too. Why not UDP is chosen remains a little secret.

Do you have the UDP range, which is provided as "to be used" open on KMS? Firewall issue?

How are you doing the signaling? You could add some event handlers in your signaling application, like this one in JS, that should be possible in Java too:

webRtcEndpoint.on('NewCandidatePairSelected', (event) => {

this.logger.info(`publisher/${room}: NewCandidatePairSelected, server ${event.candidatePair.localCandidate}`)

this.logger.info(`publisher/${room}: NewCandidatePairSelected, client ${event.candidatePair.remoteCandidate}`)

})

Also there are nice events like MediaStateChanged and ConnectionStateChanged, which allow you to monitor the state of the connection at KMS.

Do you also have a consumer app which could put some light on the scene?

[Miguel Ángel Exposito Sanchez]

Hi Neil,

- No firewall is running on the LAN server. And I triple-checked that ports 40000-57000 are open for both TCP and UDP in the cloud server. Besides, KMS is indeed receiving data according to its logs (https://pastebin.com/YQgKk7g5), but reports broken packets and refuses to decode the incoming video.

- About signaling: my app is based on Qt. I'm using a Qt Websocket against an unmodified one2many-call app from the tutorials.

- Not sure what you mean by a consumer app in this context

- Regarding the *StateChanged events, I'll probably go down that road. However, it looks like some kind of configuration mismatch between the sender and receiver GStreamer pipelines.

- The main difference in SDP between the 'good' and 'bad' cases is the fact that the good case is missing this line present in the bad one:

a=fmtp:96 packetization-mode=1;sprop-parameter-sets=J0IAH41oDIPaEAAAAwAQAAADAyDxB6g=

According to the standard, packetization-mode is assumed as 0 if not explicitly set by an a=fmtp line. Which could help to explain the difference in packet sizes seen in Wireshark, but there's no guarantee that it has anything to do with the issue at hand.

I'd love to understand the logic behind the Android side adding or not this line, or picking TCP over UDP. Is there any GST logging category that can show more info?. I've tried nice*:6 and webrtpbin:6 but got no clues at all.

Any more ideas?

--
You received this message because you are subscribed to a topic in the Google Groups "kurento" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kurento/gFUGaIUcG-w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kurento+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/bd06919a-2d4e-45aa-8b64-7f20519c27b7n%40googlegroups.com.

--

Miguel Ángel Expósito Sánchez Director of R&D E. miguel....@generaldrones.es www.generaldrones.es

La información contenida en este mensaje y los archivos adjuntos es confidencial, cualquier modificación, retransmisión, difusión u otro uso de esta información por personas o entidades distintas a las personas a la que va dirigida está prohibida salvo que esté autorizado expresamente por el emisor del mensaje. Si recibe este mensaje por error, por favor borre el mensaje de cualquier ordenador sin copiarlo ni comunicarlo e informe al emisor del mismo. Aunque GENERAL DRONES, S.L. ha adoptado las precauciones necesarias, recomendamos adopte las medidas oportunas para comprobar que no existen, en este mensaje, elementos que puedan afectar a su equipo informático. GENERAL DRONES, S.L. no aceptará ninguna responsabilidad acerca de los daños o pérdidas que pudieran producirse por este motivo. Asimismo, le informamos que las manifestaciones contenidas en el mensaje son responsabilidad exclusiva y personal de quien lo envía.

The information contained in this message and its attachments are confidential, any modification, broadcasting, diffusion or another use of this information by persons or entities other than the intended recipient is prohibited unless specifically authorized by the sender. If you receive this message by error, please contact the sender and delete the message and its attachments from any computer without making any copy or disclosure. Although GENERAL DRONES, S.L. has taken the necessary precautions, we recommend adopting the appropriate measures to ensure that this message does not contain any elements that could affect your computer. GENERAL DRONES, S.L. will not accept any responsibility for the damages or losses that could take place for this reason. We inform you also that the opinions contained in this message are personal and exclusively responsibility of the sender. 

[Neil Young]

> a=fmtp:96 packetization-mode=1;sprop-parameter-sets=J0IAH41oDIPaEAAAAwAQAAADAyDxB6g=

Negotiating H.264 can be a PITA. Did you try with a VP8 pipeline? That should be straight forward to implement.

My H.264 fmtp lines always are like so: "packetization-mode=1; level-asymmetry-allowed=1; profile-level-id=42e01f"

In my experience especially the profile-level-id is "war deciding" for a proper negotiation. This one describes H.264 constrained baseline, which IMHO is ok for most of the use cases.

My H.264 pipelines therefore usually have this in the tail:

 ! video/x-h264, level=(string)3.1 ! rtph264pay config-interval=1 ! application/x-rtp,media=video,encoding-name=H264,payload=98 ! webrtcbin.

>  Is there any GST logging category that can show more info?. I've tried nice*:6 and webrtpbin:6 but got no clues at all.

KMS logging is very weird, you can try to get more info here: https://doc-kurento.readthedocs.io/en/latest/features/logging.html

> Not sure what you mean by a consumer app in this context

I mean, if there is a "producer" (your Android app) and a media-server (the KMS): Who is watching the video ("consuming")? This part is missing.