Webrtc video is very blurry because of bandwidth estimation (BWE)

69 views
Skip to first unread message

Yuan Ho

unread,
Dec 14, 2021, 11:57:43 AM12/14/21
to discuss-webrtc

We are using webrtc video track to transmit image of a canvas element. The canvas is 1024x768 and updated dynamically once per second so the video is transmitting in 1fps.

However, we found that at the receiver side, the received video is very blurry because the image size is shrinked to 240x180 by webrtc, probably by its BWE (bandwidth estimation) mechanism.

```

let pc = new RTCPeerConnection();

let canvas = document.createElement("canvas");
canvas.width = 1024;
canvas.height = 768;

let stream = canvas.captureStream();

let tracks = stream.getTracks();

tracks.forEach((track) => {
    pc.addTrack(track);
});

let img = document.createElement("img");

img.width = 1024;
img.height = 768;

let ctx = canvas.getContext("2d");

while (true)
{
    updateImage(img);
    ctx.drawImage(img, 0, 0, 1024, 768);

    // sleep 1 second
    sleep(1000);
}

```

The transmitted image size is increased on every 60 frames sent (240x180->360x270->480x360->720x540->1024x768). We reason that the BWE algorithm needs the statistics like frame drop rate of 60 frames to update bandwidth estimation. However, 60 frames is 1 minute under 1fps, which is really slow.

4 updates are needed before the BWE settles on the final image size 1024x768. That is, the user needs to wait 4 minutes to see the video in full resolution, which is unacceptable.

From chrome://webrtc-internals, we can observe the increase of transmitted frame width and height.

SH_557.jpg

We tried setting the framerate parameter of captureStream() to 1 but this does no help.

```

MediaStream = canvas.captureStream(frameRate);

```

We also tried to set RTCRtpSendParameters.degradationPreference and found that its default value is already "maintain-resolution".

Lately, we increased the frame rate to 30fps so that the wait time is cut to 8 seconds in theory. In practice, the user still waits about 20 seconds before seeing full resolution, which is still unacceptable.

Is there a way to bypass webrtc's BWE mechanism, or make it run faster? Can it remember that the bandwidth is large and begin with an estimation of larger bandwidth? Thanks very much.

Reply all
Reply to author
Forward
0 new messages