How should MediaDevice interacts with BFCache

[Mingyu Lei]

Hi MediaDevice/MediaStream owners,

We are from the Chrome BFCache team and we are working on the back/forward cache that freezes the old page when the user navigates away, so that it could be restored if the user performs a history navigation back to that page again. This makes the loading instant and we want to optimize the hit rate of the BFCache.

Currently, the page won't be eligible for BFCache if it has any usage of the MediaDevice, as it will be disabled when the MediaDeviceDispatcherHost is created. This could be over conservative, for example, the page that just calls enumerateDevices() to queries the MediaDevice list and fetch some read-only properties without doing any further operation should be fully compatible with BFCache.

We want to understand what are your thoughts about the correct interaction between MediaDevice and BFCache. The overall principle is that the page in BFCache should not communicate with others and all the behaviors should be the same as if the page is destroyed. We are thinking if it's acceptable to store the page with MediaDevice usage in BFCache, but making sure that the access to any MediaStream is closed (e.g. by setting the enabled property of all the MediaStreamTrack to false, so the MediaStream's active property is false). 

Thanks!

Mingyu

[Mingyu Lei]

+webrt...@chromium.org +guido@

[Guido Urdaneta]

I think it's acceptable to store the page in BFCache even with MediaDeviceDispacherHost usage. 

Wrt MediaStreams, they should be closed by stopping all the tracks (not by setting the enabled property to false). This normally happens automatically when a page is navigated away (we just need to make sure it continues to happen if we store the page in BFCache).

+Harald Alvestrand in case he has more thoughts.

[Harald Alvestrand]

My thought is that pages with open mediastreamtracks shouldn't be stored in the bfcache at all.

Just stopping the tracks while storing the JS state of the page will leave the JS portions of the page thinking that it has open devices, while the device management part of the browser thinks that these devices are closed. Reopening would be highly confusing to the user, and might have security implications.

For pages that just query the MediaDeviceDispatcherHost with enumerateDevices, I don't think there is a problem (but we might want to check if devicechange event needs to be fired when the page's restored from cache).

[Mingyu Lei]

Thanks Guido and Harald for your comments. Please find our inline reply below. In short, we still propose keeping the page with open MediaStreamTracks in BFCache as much as possible.

Wrt MediaStreams, they should be closed by stopping all the tracks (not by setting the enabled property to false). This normally happens automatically when a page is navigated away (we just need to make sure it continues to happen if we store the page in BFCache).

Yes, we agree that we should try to stop all of the tracks in the same manner as it’s done in a (non-BFCached) navigation.

My thought is that pages with open mediastreamtracks shouldn't be stored in the bfcache at all.
Just stopping the tracks while storing the JS state of the page will leave the JS portions of the page thinking that it has open devices, while the device management part of the browser thinks that these devices are closed.

It should be possible to keep the “JS part” and the “device management part” in sync right? One way to achieve it is to close the tracks and notify the JS part through the `ended` event. Even if the event is only fired after the page restoration, it should not create any confusion or inconsistent state.

Blocking BFCache should only be used as a last resort. In this case, it seems keeping the page in BFCache while closing all the tracks is a feasible option, so we want to explore this further and address any concerns before considering blocking BFCache.

Reopening would be highly confusing to the user, and might have security implications.

Indeed, it’s not clear whether reopening (or not reopening) is the “right” behavior due to the differences between users’ expectation (that a new page is loaded when they hit the back button) and the actual state of the page (that it is only hidden and then shown). If the JS side is properly notified that the tracks were closed (due to navigation), it’s able to react accordingly on `pageshow`. So it could be a choice to not reopen the tracks during page restoration.

[Fergal Daly]

On Mon, 6 Nov 2023 at 18:41, Mingyu Lei <le...@chromium.org> wrote:

Thanks Guido and Harald for your comments. Please find our inline reply below. In short, we still propose keeping the page with open MediaStreamTracks in BFCache as much as possible.

Wrt MediaStreams, they should be closed by stopping all the tracks (not by setting the enabled property to false). This normally happens automatically when a page is navigated away (we just need to make sure it continues to happen if we store the page in BFCache).

Yes, we agree that we should try to stop all of the tracks in the same manner as it’s done in a (non-BFCached) navigation.

My thought is that pages with open mediastreamtracks shouldn't be stored in the bfcache at all.
Just stopping the tracks while storing the JS state of the page will leave the JS portions of the page thinking that it has open devices, while the device management part of the browser thinks that these devices are closed.

It should be possible to keep the “JS part” and the “device management part” in sync right? One way to achieve it is to close the tracks and notify the JS part through the `ended` event. Even if the event is only fired after the page restoration, it should not create any confusion or inconsistent state.

Blocking BFCache should only be used as a last resort. In this case, it seems keeping the page in BFCache while closing all the tracks is a feasible option, so we want to explore this further and address any concerns before considering blocking BFCache.

BTW, this is all currently unspecced which is a separate problem. We should aim to spec the interaction as well as update out implementation. I've opened an issue. From past experience, any attempt to spec that the page should not enter BFCache will be opposed unless that is fundamentally incompatible with the API,

F

[Mingyu Lei]

We received a response from Safari in the issue that Fergal created. Their current implementation aligns quite well with our proposal: allow pages with MediaStreamTracks to enter BFCache, close the tracks and fire an 'ended' event without attempting to reopen them on restore.

The only detailed difference is that they are only closing capture tracks, we can discuss this point further.

[Mingyu Lei]

Hi Harald and Guido,

We have created a document that summarizes the scenarios and our proposal, could you help to review it?

Thanks!

Mingyu

[Mingyu Lei]

Hi Web RTC team,

We are working on the first two phases according to the plan, and we have two more questions that need your help to answer.

1. WebRTC feature handle in MediaStreamTrack

In this CL (+chrisha@ in the cc list) a WebScheduledTrackedFeature `kWebRTC` was added when a MediaStreamTrack is live. The CL description says:

this CL modifies the logic to only notify the scheduler once the RTCPeerConnection is supplemented with an
'open' RTCDataChannel or a 'live' MediaStreamTrack, which indicates with much higher likelihood that actual data will be sent.

but the feature is added from MediaStreamTrack regardless of if RTCPeerConnection is in use. Could you help to confirm the behavior? 

We are adding a new feature handle for BFCache that tracks if there is any live MediaStreamTrack, it has the same lifecycle as the existing feature handle. If the WebRTC feature handle indeed needs to be added for each live MediaStreamTrack, shall we combine it with our new handle? Otherwise, we may need to install the WebRTC handle when it's really under the RTC cases.

2. Getting all the live MediaStreamTrack from a page

In the proposal, we discussed the plan to close all the open MediaStreamTrack when navigating away from a page. One possible approach is to observe the navigation event from the MediaStreamWebContentsObserver, and send IPC to each of the MediaStream created from the MediaStreamDispatcherHost to close the tracks. However, it seems currently there is no way to obtain the MediaStream under a particular MediaStreamDispatcherHost. Do you have any suggestions on how to achieve the goal in this plan?

Thanks a lot!

Mingyu