Intent to Implement & Ship: WebRTC Perfect Negotiation APIs

[Henrik Boström]

Contact emails

hb...@chromium.org, gui...@chromium.org

Explainer

"Perfect Negotiation" refers to four API improvements to WebRTC that together make signaling non-racy and more ergonomic. These are:

1. restartIce()

2. setRemoteDescription() with "rollback"

3. setLocalDescription() that implicitly creates the offer or answer

4. Stopping and stopped transceivers

Each of them can be shipped separately (and in any order), but I'm doing a single intent to help with motivation and not repeating discussions in multiple intents.

All related APIs, including motivation for adding them, were presented on the July 2, 2019 W3C WebRTC WG Meeting:

https://docs.google.com/presentation/d/1xcvf0udNeSH7s1FOY7RRqr1dEFvokZjn-MZPjwy3iXQ/edit?usp=sharing

Spec

restartIce()

https://w3c.github.io/webrtc-pc/#dom-rtcpeerconnection-restartice

setRemoteDescription() with "rollback"

https://w3c.github.io/webrtc-pc/#dom-peerconnection-setremotedescription

This refers to the implicit rollback that happens at step 3.1.

setLocalDescription() that implicitly creates the offer or answer

https://github.com/w3c/webrtc-pc/pull/2234

The changes have already been discussed and have consensus, the PR will be reviewed for the WG meeting this week.

Stopping and stopped transceivers

https://w3c.github.io/webrtc-pc/#dom-rtcrtptransceiver-stop

Especially, see the [[Stopping]] and [[Stopped]] internal slots.

Summary

Negotiation refers to how WebRTC endpoints exchange information about what media to send/receive and the APIs associated with negotiation.

Perfect Negotiation refers to recent improvements to the spec that make the negotiation experience free from race conditions where edge cases can cause operations to fail.

1. restartIce()

An ICE restart causes a WebRTC connection to try to reconnect, which you for example need to do if you change WiFi network. This is already shipped in Chrome by passing the {iceRestart:true} argument to createOffer(), but only works if you are in the stable signaling state (assumes you are not already in the middle of negotiation). As such, there are edge cases where {iceRestart:true} will not work, which might only happen due to races.

restartIce() is a race-free version of this API - it checks if you are in the stable signaling state before restarting ICE, and if you aren't, it waits to trigger the ICE restart in the next negotiation instead.

2. setRemoteDescription() with "rollback"

Rollback is a way to cancel an on-going negotiation. This is needed if two WebRTC endpoints attempt to send an offer at the same time, in this case, one of the endpoints have to cancel its local offer by rolling back so that it can accept the incoming offer. This way, it can complete the remote negotiation and the create a follow-up offer once back to stable. Without "rollback" you cannot get out of this situation, and you have to kill the WebRTC session and start over again.

Rollback has existed in the spec for years and shipped by Firefox. Now that the API is improved to happen implicitly at setRemoteDescription(), we want to ship it too.

3. setLocalDescription() that implicitly creates the offer or answer

Shipped today in Chrome (and every other major browser) is createOffer(), createAnswer() and setLocalDescription() which are all asynchronous. The argument passed in to setLocalDescription() is the offer or answer generated by the createOffer() or createAnswer() methods. The change we want to ship for Perfect Negotiation is to create the offer or answer implicitly in the case that setLocalDescription() is called without an argument. This avoids a race that can currently happen if negotiation is started between creating the offer or answer and setLocalDescription(), in which case it will fail because it is in the wrong signaling state.

4. Stopping and stopped transceivers

Stopping a transceiver is the only way to free up resources used by an RTCPeerConnection, this is especially useful for long-lived sessions where people join and leave over time. This is already shipped in Firefox but it was dangerous to use in some edge cases. Now that stopped is negotiated safely, we want to implement it too.

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Risks

Interoperability and Compatibility

Low; all Perfect Negotiation changes where discussed at the July 2, 2019 W3C WebRTC WG Meeting (slides), where public support was given from Chrome and Edge, as well as Firefox who was proposing the changes and already working on them.

All features have been Assigned or Started on the chromium code base by Google and Microsoft contributors.

Edge: In development

Firefox: Shipped or in development

Safari: No signals

Web / Framework developers: No signals

Please include links where possible. Examples include resolutions from relevant standards bodies (e.g. W3C Working Group), tracking bugs, or links to online conversations.

Ergonomics

Yes, these go together with the rest of the WebRTC signaling APIs and are mostly improvements to already-shipped features. Same performance, less prone to error.

Activation

No, the APIs are very similar to the existing APIs, it's typically a matter of invoking 1 method instead of 2.

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

restartIce()

https://cs.chromium.org/chromium/src/third_party/blink/web_tests/external/wpt/webrtc/RTCPeerConnection-restartIce.https.html

https://wpt.fyi/results/webrtc/RTCPeerConnection-restartIce.https.html?label=master&label=experimental

The other APIs will have WPT tests added as part of implementing them. Chrome, Edge and Firefox are contributing to WPT.

Entry on the feature dashboard

restartIce()

https://www.chromestatus.com/feature/5723155500892160

setRemoteDescription() with "rollback"

https://www.chromestatus.com/feature/5079977739419648

setLocalDescription() that implicitly creates the offer or answer

https://www.chromestatus.com/feature/6253672042332160

Stopping and stopped transceivers

https://www.chromestatus.com/feature/5410592384876544

[Philip Jägenstedt]

Thanks for rolling this up into a single intent, it's certainly less work to review and I hope it's less work for you too.

LGTM1, with a bunch of non-blocking questions inline.

Given that pc.createOffer({iceRestart:true}) is racy, do you think web developers should be discouraged from using it? If so, is there a use counter for it, so that we can deprecate/remove once usage is low? Is there MDN documentation that should be updated to warn about the perils?

 

2. setRemoteDescription() with "rollback"

Rollback is a way to cancel an on-going negotiation. This is needed if two WebRTC endpoints attempt to send an offer at the same time, in this case, one of the endpoints have to cancel its local offer by rolling back so that it can accept the incoming offer. This way, it can complete the remote negotiation and the create a follow-up offer once back to stable. Without "rollback" you cannot get out of this situation, and you have to kill the WebRTC session and start over again.

Rollback has existed in the spec for years and shipped by Firefox. Now that the API is improved to happen implicitly at setRemoteDescription(), we want to ship it too.

Was there another way to rollback that Firefox had for years before it happened implicitly in setRemoteDescription()? Or is it the same codepath but some tweaking has now made it non-racy in combination with the other negotiation APIs?

3. setLocalDescription() that implicitly creates the offer or answer

Shipped today in Chrome (and every other major browser) is createOffer(), createAnswer() and setLocalDescription() which are all asynchronous. The argument passed in to setLocalDescription() is the offer or answer generated by the createOffer() or createAnswer() methods. The change we want to ship for Perfect Negotiation is to create the offer or answer implicitly in the case that setLocalDescription() is called without an argument. This avoids a race that can currently happen if negotiation is started between creating the offer or answer and setLocalDescription(), in which case it will fail because it is in the wrong signaling state.

This sounds good. Is it a non-racy equivalent to `pc.setLocalDescription(await pc.createOffer())`? If so, what about the case of `pc.createOffer(options)`, is there a non-racy way to do that?

As with `pc.createOffer({iceRestart:true})`, I wonder if web devs can be warned about the definitely-racy cases here?

[Henrik Boström]

Yes, restartIce() is superior, and I would support deprecating pc.createOffer({iceRestart:true}), though that seems unlikely to happen any time soon. MDN warning about the problems discussed in this slide would be good.

 

2. setRemoteDescription() with "rollback"

Rollback is a way to cancel an on-going negotiation. This is needed if two WebRTC endpoints attempt to send an offer at the same time, in this case, one of the endpoints have to cancel its local offer by rolling back so that it can accept the incoming offer. This way, it can complete the remote negotiation and the create a follow-up offer once back to stable. Without "rollback" you cannot get out of this situation, and you have to kill the WebRTC session and start over again.

Rollback has existed in the spec for years and shipped by Firefox. Now that the API is improved to happen implicitly at setRemoteDescription(), we want to ship it too.

Was there another way to rollback that Firefox had for years before it happened implicitly in setRemoteDescription()? Or is it the same codepath but some tweaking has now made it non-racy in combination with the other negotiation APIs?

Triggering "rollback" was something you did with setLocalDescription(). So, when you get a remote offer and you are in the "has-local-offer" state you would have to do two operations:

async function onRemoteOffer(offer) {

  if (pc.signalingState == 'has-remote-offer') {

    await pc.setLocalDescription({type:"rollback"});

    // Warning: What if something happened in-between SLD(rollback) and SRD(offer)?

  }

  await pc.setRemoteDescription(offer);

}

The change to spec is that performing setRemoteDescription(offer) will implicitly perform the rollback before applying the offer, if a rollback is needed. This makes it impossible for something unexpected to happen in-between rolling back and setting the offer, and relieves the application from writing some boiler-plate logic to take care of this case.

Implementation-wise, the "rollback" steps are still performed the same way, it's just that you queue them internally to happen as part of SRD.

3. setLocalDescription() that implicitly creates the offer or answer

Shipped today in Chrome (and every other major browser) is createOffer(), createAnswer() and setLocalDescription() which are all asynchronous. The argument passed in to setLocalDescription() is the offer or answer generated by the createOffer() or createAnswer() methods. The change we want to ship for Perfect Negotiation is to create the offer or answer implicitly in the case that setLocalDescription() is called without an argument. This avoids a race that can currently happen if negotiation is started between creating the offer or answer and setLocalDescription(), in which case it will fail because it is in the wrong signaling state.

This sounds good. Is it a non-racy equivalent to `pc.setLocalDescription(await pc.createOffer())`? If so, what about the case of `pc.createOffer(options)`, is there a non-racy way to do that?

Yes this is a non-racy version of that, internally the same algorithms are executed, but they are executed sequentially without risk of things happening in-between.

Given the addition of restartIce(), you don't need "options" anymore. The only other option is voiceActivityDetection, which I wonder about... it seems to be a way to remove codecs from the SDP, but you can already control which codecs to offer/accept with the setCodecPreferences() API. I filed Is voiceActivityDetection still needed? on the spec.

As with `pc.createOffer({iceRestart:true})`, I wonder if web devs can be warned about the definitely-racy cases here?

That sounds like a good idea, the problem is called "glare", and the problem is described on this slide.

[Philip Jägenstedt]

+Joe Medley can you advise on how to get the risk of raciness with existing WebRTC APIs documented well on MDN?

Henrik, more inline. Other API owners, please treat as a side discussion :)

Nice!

 

3. setLocalDescription() that implicitly creates the offer or answer

Shipped today in Chrome (and every other major browser) is createOffer(), createAnswer() and setLocalDescription() which are all asynchronous. The argument passed in to setLocalDescription() is the offer or answer generated by the createOffer() or createAnswer() methods. The change we want to ship for Perfect Negotiation is to create the offer or answer implicitly in the case that setLocalDescription() is called without an argument. This avoids a race that can currently happen if negotiation is started between creating the offer or answer and setLocalDescription(), in which case it will fail because it is in the wrong signaling state.

This sounds good. Is it a non-racy equivalent to `pc.setLocalDescription(await pc.createOffer())`? If so, what about the case of `pc.createOffer(options)`, is there a non-racy way to do that?

Yes this is a non-racy version of that, internally the same algorithms are executed, but they are executed sequentially without risk of things happening in-between.

Given the addition of restartIce(), you don't need "options" anymore. The only other option is voiceActivityDetection, which I wonder about... it seems to be a way to remove codecs from the SDP, but you can already control which codecs to offer/accept with the setCodecPreferences() API. I filed Is voiceActivityDetection still needed? on the spec.

Great, thanks for filing! If there's truly not going to be a reason for the offer options dictionary, then removing it as part of a future deprecation of {iceRestart:true} might work. Or perhaps removing createOffer() entirely, as you suggest in the issue.

 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/b75e66d8-2e59-4c1d-b0ae-87dffa17f974%40chromium.org.

[Daniel Bratell]

LGTM2

/Daniel

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYeyMGddy2wctb8Xck71SoV-6wi0boz5FSFtMRRnPQDZ4A%40mail.gmail.com.

--

/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

[jbru...@mozilla.com]

Thanks Henrik! I just posted Firefox's intent to implement here.

[Joe Medley]

I need to understand the problem before I can tell you where raciness should documented. Can someone knowledgeable set up a meeting. 

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

[jbru...@mozilla.com]

Dunno if it helps, but the https://blog.mozilla.org/webrtc/perfect-negotiation-in-webrtc/ blog post chronicles how I discovered each race and links to 3 of the 4 issues filed, each of which hopefully in their initial description captures the raciness of an existing API:

1. https://github.com/w3c/webrtc-pc/issues/2167 pc.createOffer({iceRestart: true}) won't survive rollback
   
2. https://github.com/w3c/webrtc-pc/issues/2166 SLD(rollback) + SRD(offer) may miss ICE candidates
   
3. https://github.com/w3c/webrtc-pc/issues/2165 await setLocalDescription(await createOffer()) may be beat by remote offer
   

The fourth is transceiver.stop() which is a footgun if called at the wrong time on the 1st transceiver on the answerer side. It's covered by two issues:

• https://github.com/w3c/webrtc-pc/issues/2150 firstTransceiver.stop() in "have-remote-offer" stops all BUNDLEd transceivers
  
• https://github.com/w3c/webrtc-pc/issues/2176 firstTransceiver.stop() in "stable" MAY also stop BUNDLEd transceivers
  

But since we're fixing stop()—as opposed to adding better APIs, and Chrome doesn't implement it yet—there's perhaps less to document on that one. OTOH stop() now behaves differently than before, stopping only offers, not answers (requiring a successful negotiation from the offerer side specifically).

Feel free to ping me if you have questions.

.: Jan-Ivar :.

To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.

[Henrik Boström]

Does that help, Joe? Especially Jan-Ivar's excellent blog post. Otherwise we can set something up.

Documentation of existing APIs aside, I'm still waiting for that third LGTM on this intent thread. Chrome's restartIce() is ready to be shipped but blocked on this intent.

[Yoav Weiss]

Love the name, wish we had more "perfect" things in the platform :D

Happy to see this being worked on by multiple vendors and the intent from Firefox folks.

LGTM3

Any word from Safari folks? Can you open a WebKit bug for this, if there isn't one already?

 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5e500566-4ae4-489e-bef1-c77994b97c6f%40chromium.org.

[Henrik Boström]

Thanks, I'll check. I just sent an email to them about this and might see them on VC later today.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Joe Medley]

Maybe. You're implying that there are specific known problems related to specific APIs and that there are no general principles that need communicated.

Is either part of that statement correct?

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5e500566-4ae4-489e-bef1-c77994b97c6f%40chromium.org.

[Henrik Boström]

There are specific known problems to specific APIs that can be worked around if you know about them, but if you don't, you may have failures that only occur due to races.

There are guidelines that could be better communicated to warn developers about this.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Joe Medley]

"There are guidelines that could be better communicated to warn developers about this."

That's why we need to meet.

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5e500566-4ae4-489e-bef1-c77994b97c6f%40chromium.org.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/4e282ad9-07de-4925-a9d8-c50ac374111f%40chromium.org.

[gur...@microsoft.com]

LGTM. Really great to see this land and support it. Just had one clarifying non blocking question:

This is already shipped in Firefox but it was dangerous to use in some edge cases. Now that stopped is negotiated safely, we want to implement it too.

What were the edge cases? Could you elaborate with a few examples so that we could also spend time testing those out thoroughly with this proposal on behalf of Microsoft Edge.