Forcing a keyframe to be sent to DataChannel

[Maxim Maxim]

Greetings, everyone. I am using DataChannel to send frames to have more control over broadcast settings. The frames are read with ReadableStream from the stream received with getDisplayMedia. But there is a problem that if nothing happens on the screen, no frames are written to the stream. As a consequence, users connecting when nothing is happening on the speaker's screen don't see the image before the speaker does something and it will be possible to encode the frame as a keyframe.
Hence the question, is it possible to somehow force a frame to be received in order to send it? Perhaps someone has tried or already solved this problem. I would be glad to have your answers. I decided to ask this here since it is tightly related to WebRTC and the WebRTC mechanism itself solves it somehow

[Khun V]

You just need to force a keyframe on the video encoder you're using once a new client joins, there's nothing libwebrtc/WebRTC can help you with. You're using DataChannel for your video, so you have to handle it all yourself - as far as WebRTC is concerned, you're sending some data of unknown nature.

On Sat, Mar 2, 2024 at 2:42 AM Maxim Maxim <light...@gmail.com> wrote:

Greetings, everyone. I am using DataChannel to send frames to have more control over broadcast settings. The frames are read with ReadableStream from the stream received with getDisplayMedia. But there is a problem that if nothing happens on the screen, no frames are written to the stream. As a consequence, users connecting when nothing is happening on the speaker's screen don't see the image before the speaker does something and it will be possible to encode the frame as a keyframe.
Hence the question, is it possible to somehow force a frame to be received in order to send it? Perhaps someone has tried or already solved this problem. I would be glad to have your answers. I decided to ask this here since it is tightly related to WebRTC and the WebRTC mechanism itself solves it somehow

--
This list falls under the WebRTC Code of Conduct - https://webrtc.org/support/code-of-conduct.
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/240f4192-6261-4b63-8352-ce98dc6e673fn%40googlegroups.com.

[guest271314]

You can send black or white frames until your target video starts, possibly including some message stating what's going on using canvas.

[Maxim Maxim]

The thing is, that's what I do. I pass keyFrame: true to the encoder. But the logic of the stream obtained with getDisplayMedia is such that it writes frames there only when something happens on the screen. If there are no changes on the screen, it does not record frames. Calling the encode method with the keyFrame value only indicates that you should force encoding of the frame as a key frame, even though it may be delta. I may be wrong somewhere. If so, please correct me

суббота, 2 марта 2024 г. в 05:08:52 UTC+3, Khun V:

[Nikita Karmatskikh]

In classic WebRTC scenario the problem is solved by sending RTCP PLI message when a new client connects. For your case you can try using VideoEncoder from new WebCodecs API. From docs the method VideoEncoder.encode allows you to force a keyframe

суббота, 2 марта 2024 г. в 20:29:00 UTC+3, guest271314: