Websocket sometimes closes unexpectedly

[Kara Isakson]

Hello all,

My team and I use Janus with some of our changes added on top. I'm currently debugging an issue that might be on the Janus end but I am not completely sure.

We sometimes have an issue where our Websocket connection to Janus will randomly close mid-stream. However, this isn't expected by Janus because it doesn't send a disconnection message, and it isn't expected by the browser because we still attempt to send keepalives. We discovered this after finding some cases of Websocket keepalives messages failing to go through from the browser since we did not expect the Websocket to randomly close. Also in these cases, we will continue to receive video for I believe up to a minute, after which I assume Janus cuts off the connection due to not receiving a keepalive message.

Looking in our logs, I see a lot of messages like this, which a teammate flagged as odd: "Websockets choked with queue: 5, trying again" (the number varies, it is often 0 or 1)

That lead me to these two Github pages, but these are resolved and we have these code changes in our codebase:

https://github.com/meetecho/janus-gateway/issues/2081

https://github.com/meetecho/janus-gateway/pull/2107

Has anyone experienced this sort of issue and know what might be happening here? Does this seem like a Janus issue or a browser issue? I can find more relevant logs if needed.

Thanks in advance,

Kara Isakson

[Lorenzo Miniero]

You may want to try master, rather than looking for individual changes. We fixed other things in the WS transport much more recently than the two issues you mentioned.

L.

[Alessandro Toppi]

Do you have intermediate layers (e.g. proxy, load balancer etc.) in your deployment?

If so, try connecting browsers directly to Janus and check if the issue persists.

[Kara Isakson]

Thanks all. Our first step then will be to pull in the latest changes from master and see if that resolves things.

We do have a load balancer in our production environment but not our development environment where I have reproduced this most recently, so it seems like that probably isn't related but it is good to know things like that could affect this.

Cheers,

Kara

[Filipe Costa]

Hey!

I'm facing this problem as well and just want to share my experience.

In our case this is only occurring when the Chrome tab is hidden and only after some time.

We've noticed that sometimes Janus doesn't send the `keepalive` message, which triggers the `timeout` message from the server, closing the connection.

We're using default values for keepalive (25s) and timeout (60s).

After some investigation we've noticed that this is caused by the Chrome Tab Intensive Throttling introduced in Chrome 88.

When a page using Janus is hidden, silent and without using WebRTC for more than 5 minutes, then the `keepalive` `setTimeout` might be intensively throttled so that it'll just be sent once a minute, which will fail to respect the 60s timeout.

In our case we'll increase keepalive and timeout values to 60s / 130s in order to workaround this issue.

it might make sense to change Janus default values as well given this Chrome feature.

Thanks!

Filipe Costa

[Alessandro Toppi]

Thanks for sharing!
ws ping pong could also help in those throttling scenarios.