Intent to Ship: Fetch API: keepalive

[Yutaka Hirano]

Contact emails

yhi...@chromium.org, igrigorik@chromium.org

Spec

- https://fetch.spec.whatwg.org/#request-keepalive-flag

- https://fetch.spec.whatwg.org/#dom-request-keepalive

- https://fetch.spec.whatwg.org/#dom-requestinit-keepalive

Summary

By setting the keepalive flag, a developer can make a fetch which will continue working even when a frame is detached. A web developer can use the feature to report events, state updates and analytics with small amount of data even when the page is about to be unloaded. This is useful for analytics and other cases where async delivery of data is required without blocking navigations, and lessens the need for synchronous XHR which is bad for user experience.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/AUAIHVF63SM

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Debuggability

Nothing

Risks

Interoperability and Compatibility

Edge: Positive based on a conversation between Ilya and Todd. Their Beacon implementation is already on top of Fetch and he hinted that it will be quick and easy for them to expose.

 

Firefox: No signals

https://bugzilla.mozilla.org/show_bug.cgi?id=1342484

Safari: They have implemented (and shipped) the property in Request class, but it looks they haven't implemented the feature itself.

https://bugs.webkit.org/show_bug.cgi?id=168865

https://bugs.webkit.org/show_bug.cgi?id=175482

https://bugs.webkit.org/show_bug.cgi?id=175151

We have some restrictions for the feature:

1. It's disabled on requests which need CORS preflight. Making fetch for such a request

    leads to a failure.

2. The feature is disabled on workers.

Ergonomics

Activation

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Web platform tests:
- https://wpt.fyi/fetch/api/request/request-keepalive.html
- https://wpt.fyi/fetch/api/request/request-keepalive-quota.html
- https://wpt.fyi/fetch/api/basic/keepalive.html

Entry on the feature dashboard

https://www.chromestatus.com/features/5760375567941632

[Yoav Weiss]

I'm excited about the feature and the use cases that it will enable. Thanks for working on it! :)

On Wed, Jan 3, 2018 at 12:45 PM Yutaka Hirano <yhi...@chromium.org> wrote:

Contact emails

yhi...@chromium.org, igrigorik@chromium.org

Spec

- https://fetch.spec.whatwg.org/#request-keepalive-flag

- https://fetch.spec.whatwg.org/#dom-request-keepalive

- https://fetch.spec.whatwg.org/#dom-requestinit-keepalive

Summary

By setting the keepalive flag, a developer can make a fetch which will continue working even when a frame is detached. A web developer can use the feature to report events, state updates and analytics with small amount of data even when the page is about to be unloaded. This is useful for analytics and other cases where async delivery of data is required without blocking navigations, and lessens the need for synchronous XHR which is bad for user experience.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/AUAIHVF63SM

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Debuggability

Nothing

Risks

Interoperability and Compatibility

Edge: Positive based on a conversation between Ilya and Todd. Their Beacon implementation is already on top of Fetch and he hinted that it will be quick and easy for them to expose.

 

Firefox: No signals

https://bugzilla.mozilla.org/show_bug.cgi?id=1342484

Safari: They have implemented (and shipped) the property in Request class, but it looks they haven't implemented the feature itself.

https://bugs.webkit.org/show_bug.cgi?id=168865

https://bugs.webkit.org/show_bug.cgi?id=175482

https://bugs.webkit.org/show_bug.cgi?id=175151

That seems like a high compatibility risk. If developers have no way to feature-detect reliable support for the feature, they cannot rely on it as a `sendBeacon` alternative...

Do you know if there are plans to fix the situation?

We have some restrictions for the feature:

1. It's disabled on requests which need CORS preflight. Making fetch for such a request

    leads to a failure.

2. The feature is disabled on workers.

Are those restrictions documented? Are they part of the spec?

If not - could you elaborate on the reasoning for them?

 

Ergonomics

Activation

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Web platform tests:
- https://wpt.fyi/fetch/api/request/request-keepalive.html
- https://wpt.fyi/fetch/api/request/request-keepalive-quota.html
- https://wpt.fyi/fetch/api/basic/keepalive.html

I couldn't find a test that makes sure that the request is in fact kept alive after its frame was detached. Is there one? If not, could you add such a test?

 

Entry on the feature dashboard

https://www.chromestatus.com/features/5760375567941632

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABihn6HeJDdSgfSdcN4zhrYGJSrF4PnodeZMm32JG5_PAbPHMA%40mail.gmail.com.

[Ben Kelly]

On Wed, Jan 3, 2018 at 7:28 AM, Yoav Weiss <yo...@yoav.ws> wrote:

On Wed, Jan 3, 2018 at 12:45 PM Yutaka Hirano <yhi...@chromium.org> wrote:

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Web platform tests:
- https://wpt.fyi/fetch/api/request/request-keepalive.html
- https://wpt.fyi/fetch/api/request/request-keepalive-quota.html
- https://wpt.fyi/fetch/api/basic/keepalive.html

I couldn't find a test that makes sure that the request is in fact kept alive after its frame was detached. Is there one? If not, could you add such a test?

It would also be nice to have a test verifying that the value is reflected in the service worker `FetchEvent.request` properly.

Thanks.

Ben

[Ben Kelly]

I guess maybe also consider how to handle pass-through fetch handler like `fetch(evt.request))`.  If you reject in workers when keepalive is set, does that mean you can't expose `FetchEvent.request.keepalive` yet?  It seems like that should work in the long run, though.

[David Benjamin]

Users generally expect that closing a tab will make a webpage "go away" for some vague notion of "go away". The web's strong discoverability and linkability properties depend on this core expectation. It means clicking on links is relatively safe, compared to, say, running a random executable.

Accordingly, this means that allowing things to outlive a tab does want a bit of extra thought. I expect we wouldn't want to allow 100s of such requests to stay alive indefinitely if the server isn't responding. We probably also wouldn't want to allow such a request to download 100s of GB of data.

Does the specification allow for the browser to impose limitations on these things? They should have an aggressive timeout and whatnot.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

[Daniel Bratell]

The specification talks about 64 KB, but I couldn't, when quickly browsing it, find any other kind of limitation so I too wonder if a malicious site can cause a permanent leakage of limited resources (sockets, memory, ...) until browser restart.

/Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAF8qwaChSeKHvpoWWaXK64bfj5zQ_KPuMNXrYio%3D6E%2Bw%2BdBicg%40mail.gmail.com.

--

/* Opera Software, Linköping, Sweden: CET (UTC+1) */

[Yutaka Hirano]

I don't know.

 

We have some restrictions for the feature:

1. It's disabled on requests which need CORS preflight. Making fetch for such a request

    leads to a failure.

2. The feature is disabled on workers.

Are those restrictions documented? Are they part of the spec?

If not - could you elaborate on the reasoning for them?

Out implementation is not spec conformant on these points. CORS preflight will be supported when the CORS logic is moved out of blink. 

 

 

Ergonomics

Activation

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Web platform tests:
- https://wpt.fyi/fetch/api/request/request-keepalive.html
- https://wpt.fyi/fetch/api/request/request-keepalive-quota.html
- https://wpt.fyi/fetch/api/basic/keepalive.html

I couldn't find a test that makes sure that the request is in fact kept alive after its frame was detached. Is there one? If not, could you add such a test?

That is tested by the last one. 

 

Entry on the feature dashboard

https://www.chromestatus.com/features/5760375567941632

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Yutaka Hirano]

That is tested by https://wpt.fyi/service-workers/service-worker/fetch-event.https.html.

 

Thanks.

Ben

[Yutaka Hirano]

Our implementation imposes 30sec timeout after the frame is destructed, but it's not specified. The spec requires us to reject requests with keepalive set when there are too many keepalive requests with body (see the 8th item of https://fetch.spec.whatwg.org/#http-network-or-cache-fetch).

[Todd Reifsteck]

Microsoft Edge was the first browser to implement fetch with keepalive. It was shipped in Edge 15 in the Windows 10 Creator’s Update in spring 2017.

 

https://github.com/whatwg/fetch/pull/419 is Ilya’s PR that goes over the quotas that Edge implements.

 

If further quotas are added or if quotas are loosened, lets please get them into the public fetch spec to ensure websites can easily work across UAs.

 

-Todd

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABihn6Hq4mcppQAtsJHP3C-BqmED84evjamc8oHVWOSdZU_vCA%40mail.gmail.com.

[Yutaka Hirano]

Thank you for the information, I updated https://www.chromestatus.com/feature/5760375567941632.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Ben Kelly]

Thanks!

How do you handle pass-through service worker that does `evt.respondWith(fetch(evt.request))` for keepalive requests?  Does that fail since keepalive fetch rejects in workers?

 

Just curious.  Thanks again.

Ben

[Yutaka Hirano]

requests with keepalive set works as follows in workers:

 1. The request will be rejected when there are too many inflight bytes (spec conformant).

 2. The request will be canceled when the worker is terminated (not spec conformant).

So, "keepalive fetch rejects in workers" is not correct.

On service workers, evt.respondWith(fetch(evt.request)) will keep the service worker alive for a certain time (regardless of keepalive flag), so it works like keepalive specified.

+horo@, please correct me if I'm wrong.

[Tsuyoshi Horo]

On Fri, Jan 5, 2018 at 4:16 PM, Yutaka Hirano <yhi...@chromium.org> wrote:

requests with keepalive set works as follows in workers:

 1. The request will be rejected when there are too many inflight bytes (spec conformant).

 2. The request will be canceled when the worker is terminated (not spec conformant).

So, "keepalive fetch rejects in workers" is not correct.

On service workers, evt.respondWith(fetch(evt.request)) will keep the service worker alive for a certain time (regardless of keepalive flag), so it works like keepalive specified.

+horo@, please correct me if I'm wrong.

This is correct.

"evt.respondWith(fetch(evt.request))" keeps the service worker alive for 5+ minutes.

See: https://crbug.com/753646#c7

 

On Fri, Jan 5, 2018 at 10:45 AM, Ben Kelly <bke...@mozilla.com> wrote:

On Thu, Jan 4, 2018 at 10:10 AM, Yutaka Hirano <yhi...@chromium.org> wrote:

On Thu, Jan 4, 2018 at 12:15 AM, Ben Kelly <bke...@mozilla.com> wrote:

On Wed, Jan 3, 2018 at 7:28 AM, Yoav Weiss <yo...@yoav.ws> wrote:

On Wed, Jan 3, 2018 at 12:45 PM Yutaka Hirano <yhi...@chromium.org> wrote:

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Web platform tests:
- https://wpt.fyi/fetch/api/request/request-keepalive.html
- https://wpt.fyi/fetch/api/request/request-keepalive-quota.html
- https://wpt.fyi/fetch/api/basic/keepalive.html

I couldn't find a test that makes sure that the request is in fact kept alive after its frame was detached. Is there one? If not, could you add such a test?

It would also be nice to have a test verifying that the value is reflected in the service worker `FetchEvent.request` properly.

That is tested by https://wpt.fyi/service-workers/service-worker/fetch-event.https.html.

Thanks!

How do you handle pass-through service worker that does `evt.respondWith(fetch(evt.request))` for keepalive requests?  Does that fail since keepalive fetch rejects in workers?

 

Just curious.  Thanks again.

Ben

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABihn6GguqC9kBFHXeHGv1_CC_xCbFdqQtu6hH20RgjM_Mjj6g%40mail.gmail.com.

[Ben Kelly]

On Fri, Jan 5, 2018 at 2:16 AM, Yutaka Hirano <yhi...@chromium.org> wrote:

requests with keepalive set works as follows in workers:

 1. The request will be rejected when there are too many inflight bytes (spec conformant).

 2. The request will be canceled when the worker is terminated (not spec conformant).

So, "keepalive fetch rejects in workers" is not correct.

On service workers, evt.respondWith(fetch(evt.request)) will keep the service worker alive for a certain time (regardless of keepalive flag), so it works like keepalive specified.

Thanks!  I misinterpreted the "this feature is disabled in workers" in your first post.  Thanks for clarifying.

Ben

[Yutaka Hirano]

friendly ping :)

[Chris Harrelson]

Hi,

I think there are still two remaining issue blocking this intent:

* Consensus on whether the spec should encode quotas for maximum amount of work, and how much of that must be reflected in our implementation. I'm thinking of the comments made by Daniel and Todd in particular. (Also, independent of this approval process, I think you should get consensus from Chrome networking experts that the limits and quotas in place are sufficient to prevent abuse.)

* Whether feature detection is broken by Safari's current behavior, and what to do about it to ensure a workable rollout.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABihn6Gs2_Dqzs0s%2BiND6rmKYC3zHttCH0Vw9wbs70Lz%2BQbePQ%40mail.gmail.com.

[Yoav Weiss]

On Fri, Jan 12, 2018 at 6:15 PM Chris Harrelson <chri...@chromium.org> wrote:

Hi,

I think there are still two remaining issue blocking this intent:

* Consensus on whether the spec should encode quotas for maximum amount of work, and how much of that must be reflected in our implementation. I'm thinking of the comments made by Daniel and Todd in particular. (Also, independent of this approval process, I think you should get consensus from Chrome networking experts that the limits and quotas in place are sufficient to prevent abuse.)

yhirano@ - Could you open an issue on the fetch spec asking if we should add quotas on the maximum number of requests, on top of the ones we have in place for the maximum number of bytes?

* Whether feature detection is broken by Safari's current behavior, and what to do about it to ensure a workable rollout.

We're discussing this with WebKit engineers, to see if they can complete the feature or place it behind a flag until it is complete.

friendly ping :)

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABihn6HeJDdSgfSdcN4zhrYGJSrF4PnodeZMm32JG5_PAbPHMA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Yutaka Hirano]

On Mon, Jan 15, 2018 at 9:04 PM, Yoav Weiss <yo...@yoav.ws> wrote:

On Fri, Jan 12, 2018 at 6:15 PM Chris Harrelson <chri...@chromium.org> wrote:

Hi,

I think there are still two remaining issue blocking this intent:

* Consensus on whether the spec should encode quotas for maximum amount of work, and how much of that must be reflected in our implementation. I'm thinking of the comments made by Daniel and Todd in particular. (Also, independent of this approval process, I think you should get consensus from Chrome networking experts that the limits and quotas in place are sufficient to prevent abuse.)

yhirano@ - Could you open an issue on the fetch spec asking if we should add quotas on the maximum number of requests, on top of the ones we have in place for the maximum number of bytes?

Done: https://github.com/whatwg/fetch/issues/662

 

friendly ping :)

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABihn6HeJDdSgfSdcN4zhrYGJSrF4PnodeZMm32JG5_PAbPHMA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Anne van Kesteren]

On Fri, Jan 12, 2018 at 6:14 PM, Chris Harrelson <chri...@chromium.org> wrote:
> * Consensus on whether the spec should encode quotas for maximum amount of
> work, and how much of that must be reflected in our implementation. I'm
> thinking of the comments made by Daniel and Todd in particular. (Also,
> independent of this approval process, I think you should get consensus from
> Chrome networking experts that the limits and quotas in place are sufficient
> to prevent abuse.)

FWIW, I'm happy with more restrictions, but note that the concerns
raised are raised a little late, as sendBeacon() already enables all
these things and is shipping today. The keepalive feature is
effectively moving sendBeacon()'s one unique capability to fetch() so
developers don't have to learn two APIs. (Also, if you think it's a
serious bug you might also want to start patching sendBeacon() now and
not wait for any specification clarification...)


--
https://annevankesteren.nl/

[Ilya Grigorik]

I agree with Anne. I understand the motivation and desire to put more safeguards here, but (a) this is extending capability of the existing sendBeacon mechanism across all popular platforms to Fetch, (b) afaik, I'm not aware of any real-world problems caused by current sendBeacon implementation, (c) if we start bending this too far we'll only create incentives for developers to revert back to the tried-and-true 'solutions' of sync XHRs, spinning loops in click handlers, and so on. 

https://github.com/whatwg/fetch/issues/662#issuecomment-357724019

[rsl...@chromium.org]

That's really unfortunate. I agree with David that this doesn't seem good for users if sendBeacon() already affords that capability, and doesn't seem good for our processes if it's the case that we added such capability in violation of users' expectations. These are the same concerns we're discussing in other spaces (for example, HTTP/2 Push), so I don't think these concerns are unique and I don't think the fact that sendBeacon() may be able to violate them to be a good reason to bake in the API.

(b) is the least convincing argument - I think we should be very careful about introducing features that violate users' expectations and ability to reason about how web pages are allowed to use their devices - much like we've had to rethink how we handle APIs like vibration or sensors. We should also make sure to revisit the decisions that may have missed or unintended side-effects for users.

I agree that we run the risk of negative incentives for (c). We should work to resolve those. Interventions that prioritize the user's experience and expectations may be a better solution to appropriately reflect the ecosystem tradeoffs, rather than adding/duplicating the capability.

Keeping sockets open certainly consumes resources on the client (unavoidably so) - and this means that closing tabs/renderers is no longer sufficient to free that up. Similarly, as David points out, it allows users bandwidth to be used in ways that they don't expect and is arguably not reasonable to mitigate. This is part of why there are various concerns with how best to balance features like HTTP/2 Push or background fetching in Service Workers to ensure it matches users expectations and desires.

Regarding sendBeacon(), is https://w3c.github.io/beacon/ the current spec? It seems to refer to keepalive and make reference to the fact that UAs can set limits - so doesn't that mean that sendBeacon() naturally inherits the limits of what we're discussing here, and we should put in safeguards and limits? Have I misunderstood the substance of the concern - I guess I'm pushing back "on a little late" and "existing sendBeacon capabilities" - as it's never too late, and it seems like the capabilities are already defined to be restricted (and should be in Chrome, if we're not already)

[Ilya Grigorik]

As a heads up for folks following along here, we forked and continuing this discussion in https://github.com/whatwg/fetch/issues/662#issuecomment-357740885.

On Mon, Jan 15, 2018 at 8:59 AM, <rsl...@chromium.org> wrote:

Regarding sendBeacon(), is https://w3c.github.io/beacon/ the current spec?

Yep.

 

It seems to refer to keepalive and make reference to the fact that UAs can set limits - so doesn't that mean that sendBeacon() naturally inherits the limits of what we're discussing here, and we should put in safeguards and limits?

In earlier iteration of the spec Beacon API, sendBeacon was defined in terms of Fetch but did not rely on Fetch primitives to enforce limits. Due to developer requests to extend sendBeacon to support other methods (beyond POST), custom headers, etc., we determined that the right path here was to expose the underlying primitives on Fetch and rebase sendBeacon on top of it. This way, those that need more advanced features can use Fetch+keepalive and sendBeacon becomes a convenience API on top of it — both are subject to same restrictions. Hope that makes sense. Short answer: restrictions on Fetch#keepalive should propagate to sendBeacon automatically, as long as the implementation is compliant with latest spec definition.

ig

[Takeshi Yoshino]

I agree that it's bad if renderers can DoS the browser process (e.g. having it hold much memory, many TCP sockets, etc.) as Ryan described.

But availability of either sendBeacon() or fetch()-keepalive: (according to Yutaka's post on GitHub)

- doesn't change the level, way, kind of such DoSes

- do change the duration of such DoSes after the user closes the tab (30 sec)

So, the only problem is mismatch between users' expectation on closing a tab and time to get resources freed, I think.

I think it makes sense to allow Chrome to limit the level of such unexpectedly prolonged DoSes from various points (time, memory size, # of sockets, etc.). But I'm not well convinced that it's something blocks shipping of fetch()-keepalive, or something needs to be un-shipped.

Regarding shipping fetch()-keepalive in addition to sendBeacon(), it:

- doesn't change any of the above (including duration)

- just introduces new customers of it, though it would add extra work for us to take care of in case of e.g. we need to change/un-ship it in the future

So, we should definitely investigate such a pain in taking some necessary actions later:

(a) making them throw/reject on the suggested #-of-req quota excess

(b) un-shipping them completely

I think only possibility of (b) would motivate us to postpone shipping of fetch()-keepalive. Unshipping is not a realistic choice, though, and at this point I don't think we need to take such a choice at all based on my analysis on the severity of the prolonged DoSes.

Then, the only remaining point we need to discuss is whether the debt (a) would be so big for us compared the loss in postponing shipping it. Given that we have some people looking forward to the feature, that the action wouldn't be so painful as we'll be making them throw/reject for maliciously behaving ones, I think we should ship it and address the concerns right after that.

[Ryan Sleevi]

I'm having trouble following this logic, but it's unclear if the proposal is:

1) We shipped sendBeacon() in a way that can cause a browser DoS

2) When reviewing Fetch API, folks noticed it could cause a browser DoS

3) We should ship Fetch API anyways, because no one noticed this when we shipped sendBeacon(), so what's another browser DoS?

?

[Kenji Baheux]

What I'm having trouble to understand is how shipping Fetch Keep Alive makes the situation amy worse? AFAICT, Fetch KeepAlive doesn't provide anything new over sendBeacon within the context of the abuse scenario. Please explain.

What I do know is that by not shipping it we are blocking usage that would make a positive impact for our users.

Should we address the concern? Yes, but not as if it was a P0, launch blocker. It's not, otherwise we should immediately unship sendBeacon and address similar concerns* in other places at a P0 priority. Is there data to support that call?

* AFAIK, DoS can also be done without sendBeacon. The only difference is the "up to 30s bad time after closing a tab" aspect. But, if we are concerned about nefarious actors, they have ways to get in the way of the user gesture to close a tab, making the 30s difference a weak argument to single out sendBeacon. What sorts of mitigations do we have in place for the general case?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACvaWvZbgxOzcXbsd%3DNLwK587Fbpj1Hj5gJMmDXdAiLf%3D0pXjg%40mail.gmail.com.

[Ryan Sleevi]

On Wed, Jan 24, 2018 at 3:35 PM, Kenji Baheux <kenji...@google.com> wrote:

What I'm having trouble to understand is how shipping Fetch Keep Alive makes the situation amy worse? AFAICT, Fetch KeepAlive doesn't provide anything new over sendBeacon within the context of the abuse scenario. Please explain.

So it sounds like you're saying my summary is correct?

What I do know is that by not shipping it we are blocking usage that would make a positive impact for our users.

Because we should be focusing on fixing bugs that cause a browser DoS over shipping new features, if we've got bugs that allow arbitrary content to cause a browser DoS. Our triage process is to put the user first

Should we address the concern? Yes, but not as if it was a P0, launch blocker. It's not, otherwise we should immediately unship sendBeacon and address similar concerns* in other places at a P0 priority. Is there data to support that call?

If we've implemented sendBeacon in a way that can allow web content to cause a browser DoS, I agree, we should unship sendBeacon until such a time as it cannot cause a browser DoS. I don't think this would or should be contoversial.

I would much rather see an owner for the code step up, walk through the scenarios David and I illustrated, and either show they're not possible (e.g. because of some limit that isn't part of the spec but is part of our implementation), or focus on addressing those concerns before continuing the discussion. This is about making sure we ship good code when it's ready, and a known bug that can cause a browser DoS is, I think, at the level of "don't ship".

* AFAIK, DoS can also be done without sendBeacon.

The only difference is the "up to 30s bad time after closing a tab" aspect.

That's a huge difference, as captured on the bug.

But, if we are concerned about nefarious actors, they have ways to get in the way of the user gesture to close a tab, making the 30s difference a weak argument to single out sendBeacon. What sorts of mitigations do we have in place for the general case?

"Classic" loading has all the requests being initiated in the renderer, particularly those untrusted. We further limit the resources we're willing to commit to a renderer, and when that renderer is killed, we free those resources. If we don't free them, that's a memory leak - and we hate those. Within the //net stack, for example, we only commit 16-32K/request, we limit the number of sockets (256), the connections per host (6), the number of pending streams, the window size of H/2 streams, etc - all in the service of making sure that no given request (or set of requests) can cause undue resource allocation that causes a poor user experience.

A sendBeacon within a renderer process means keeping that renderer alive - which certainly doesn't match user expectations for "close this tab, get memory/resources back and site can't keep doing work". We already have limits in the case of service workers precisely to allow us to kill them and bound their resource usage. Fetch with keepalive in the renderer has the same issues - and needs a mitigation strategy, even if it's "implementation defined" (in spec), but certainly in the Chrome code, we shouldn't be enabling without some form of resource bounding. 

This all becomes much, much worse if sendBeacon or keepalive are done in the browser process (e.g. if these have been moved in support of things like CORS in the browser proc or PlzNavigate). In those, the resources are consumed even after the renderer is killed - and we really need to bound that ability, since even killing misbehaving renderers is no longer enough to ensure their damage is limited.

[Kenji Baheux]

On Thu, Jan 25, 2018, 5:50 AM Ryan Sleevi <rsl...@chromium.org> wrote:

On Wed, Jan 24, 2018 at 3:35 PM, Kenji Baheux <kenji...@google.com> wrote:

What I'm having trouble to understand is how shipping Fetch Keep Alive makes the situation amy worse? AFAICT, Fetch KeepAlive doesn't provide anything new over sendBeacon within the context of the abuse scenario. Please explain.

So it sounds like you're saying my summary is correct?

This is the most important point. 

In my world, which could be flawed, I see:

 1. A net positive and no extra harm caused by shipping this feature.

 2. A lack of evidence that shipping Fetch KeepAlive would make things worse.

 3. A lack of evidence and data that this is already a p0 issue which would justify not only blocking this feature but also disabling sendBeacon. Given that, for the purpose of the abuse scenario, these two are interchangeable, I don't see an valid argumentation where we would be blocked here while keeping sendBeacon enabled. If you think this is indeed a P0 concern, I would like to see an intent to (temporarily) unship sendBeacon while we figure out a solution.

What I do know is that by not shipping it we are blocking usage that would make a positive impact for our users.

Because we should be focusing on fixing bugs that cause a browser DoS over shipping new features, if we've got bugs that allow arbitrary content to cause a browser DoS. Our triage process is to put the user first

The triage process has a prioritization step. It seems that we have decided to shortcircuit it here.

[nehte...@gmail.com]

Hello! I use a new email filtering service called BitBounce to better filter my spam. To deliver your email to my inbox, please click the link below and pay the small Credo fee. Thank you! You can sign up or get more info about BitBounce by clicking here. To deliver your email: We’ve never met. I’ll pay your fee. I know you. Add me to your whitelist. Email That Pays BitBounce is a product by: Turing Technology, Inc. BitBounce.com Redwood City, CA

[nehte...@gmail.com]

[Ryan Sleevi]

On Wed, Jan 24, 2018 at 4:16 PM, Kenji Baheux <kenji...@google.com> wrote:

On Thu, Jan 25, 2018, 5:50 AM Ryan Sleevi <rsl...@chromium.org> wrote:

On Wed, Jan 24, 2018 at 3:35 PM, Kenji Baheux <kenji...@google.com> wrote:

What I'm having trouble to understand is how shipping Fetch Keep Alive makes the situation amy worse? AFAICT, Fetch KeepAlive doesn't provide anything new over sendBeacon within the context of the abuse scenario. Please explain.

So it sounds like you're saying my summary is correct?

This is the most important point. 

In my world, which could be flawed, I see:

 1. A net positive and no extra harm caused by shipping this feature.

 2. A lack of evidence that shipping Fetch KeepAlive would make things worse.

 3. A lack of evidence and data that this is already a p0 issue which would justify not only blocking this feature but also disabling sendBeacon. Given that, for the purpose of the abuse scenario, these two are interchangeable, I don't see an valid argumentation where we would be blocked here while keeping sendBeacon enabled. If you think this is indeed a P0 concern, I would like to see an intent to (temporarily) unship sendBeacon while we figure out a solution.

I think we may be heading down an unproductive line, or it may be generating unnecessary hostility or frustration.

My goal is not to block your work, and I understand this feature is seen as important. During the review of the spec, concerns were identified. The reviewers of the spec are not as familiar with the code as you are, and what mitigations may exist, and whether the spec concerns are born out in the implementation.

I'm hoping we can find a collaborative way in which the implementors of sendBeacon and keepAlive in Chrome can review these concerns, and assess whether we have a risk here. I think the "evidence and data" part falls to the implementors, when the spec itself has this issue. Further, in discussions about the spec, it seems there's some cross-browser support for adding limits that are allowed to be implementation defined - so someone who owns the spec work should be proposing changes to do that, and that own the implementation to make sure we're implementing reasonable limits.

Our review process is to try to get actionable feedback, and I fear we've spent more time discussing the relative prioritization than it would take to add some basic limits or review the code for mitigations. I would rather see us ship something non-spec-conforming that ensured our users were safe than ship something spec-conforming and left them at risk, and I'd rather we not ship at all if the former is going to cause significant pain for developers. I don't think it will, the spec is amenable to change, so we should just do the work to do the right thing, and then ship when we're ready.

 

What I do know is that by not shipping it we are blocking usage that would make a positive impact for our users.

Because we should be focusing on fixing bugs that cause a browser DoS over shipping new features, if we've got bugs that allow arbitrary content to cause a browser DoS. Our triage process is to put the user first

The triage process has a prioritization step. It seems that we have decided to shortcircuit it here.

No, I think we're disagreeing about the priority, and that's concerning, because having browser DoS's causable by arbitrary content should be extremely high priority.

The goal of keepalive (and sendBeacon) should be to not end up in a situation like http://www.wired.co.uk/article/google-chrome-crash - an unfortunate bug that allows for trivial crashing of users.

[Kenji Baheux]

On Thu, Jan 25, 2018, 6:35 AM Ryan Sleevi <rsl...@chromium.org> wrote:

On Wed, Jan 24, 2018 at 4:16 PM, Kenji Baheux <kenji...@google.com> wrote:

On Thu, Jan 25, 2018, 5:50 AM Ryan Sleevi <rsl...@chromium.org> wrote:

On Wed, Jan 24, 2018 at 3:35 PM, Kenji Baheux <kenji...@google.com> wrote:

What I'm having trouble to understand is how shipping Fetch Keep Alive makes the situation amy worse? AFAICT, Fetch KeepAlive doesn't provide anything new over sendBeacon within the context of the abuse scenario. Please explain.

So it sounds like you're saying my summary is correct?

This is the most important point. 

In my world, which could be flawed, I see:

 1. A net positive and no extra harm caused by shipping this feature.

 2. A lack of evidence that shipping Fetch KeepAlive would make things worse.

 3. A lack of evidence and data that this is already a p0 issue which would justify not only blocking this feature but also disabling sendBeacon. Given that, for the purpose of the abuse scenario, these two are interchangeable, I don't see an valid argumentation where we would be blocked here while keeping sendBeacon enabled. If you think this is indeed a P0 concern, I would like to see an intent to (temporarily) unship sendBeacon while we figure out a solution.

I think we may be heading down an unproductive line, or it may be generating unnecessary hostility or frustration.

It saddens me that you see this as an unproductive path because this is the crux of the problem. I don't understand the rationale behind the launch blocking, being applied here. Seeing it unadressed is very frustrating to me, because it just seems arbitrary. I will seek other people's opinions, as I really want to understand why I'm the one who is wrong here.

My goal is not to block your work, and I understand this feature is seen as important. During the review of the spec, concerns were identified. The reviewers of the spec are not as familiar with the code as you are, and what mitigations may exist, and whether the spec concerns are born out in the implementation.

I'm hoping we can find a collaborative way in which the implementors of sendBeacon and keepAlive in Chrome can review these concerns, and assess whether we have a risk here. I think the "evidence and data" part falls to the implementors, when the spec itself has this issue. Further, in discussions about the spec, it seems there's some cross-browser support for adding limits that are allowed to be implementation defined - so someone who owns the spec work should be proposing changes to do that, and that own the implementation to make sure we're implementing reasonable limits.

Our review process is to try to get actionable feedback, and I fear we've spent more time discussing the relative prioritization than it would take to add some basic limits or review the code for mitigations.

I've heard that the mitigations discussed so far represented a non trivial amount of work. Maybe, there is an easy short term option.

[nehte...@gmail.com]

[Takeshi Yoshino]

Hi Ryan,

We really appreciate your analysis. In the last post of mine, I wanted to summarize the points we've discovered for each of decisions we should make.

snip

 

On Wed, Jan 24, 2018, 11:56 PM Ryan Sleevi <rsl...@chromium.org> wrote:

I'm having trouble following this logic, but it's unclear if the proposal is:

1) We shipped sendBeacon() in a way that can cause a browser DoS

I wanted to divide "a browser DoS" into:

- a browser DoS while the tab is open

- a browser DoS lasting after the tab is open

I said I estimated the second part is not significant, and also mentioned that it might be conflicting with users' expectation and what we'd like to promise based on Chrome's design principle.

The only point I was wondering is the level of its severity. I didn't intend to and don't object to your concerns.

And, now I understood that you've analyzed this as a huge difference.

 

2) When reviewing Fetch API, folks noticed it could cause a browser DoS

I agree, though I'd like to remark that it's about the 30 sec extra part.

 

3) We should ship Fetch API anyways, because no one noticed this when we shipped sendBeacon(), so what's another browser DoS?

I don't think this claim is so non sense. One possible extreme principle is that we shouldn't add anything to Chrome unless crbug.com becomes empty for bug-marked entries. That's not what we follow. We always need to think about priority.

Given your review, we've started building the right solution for mitigating the browser DoS. Looks the important point we need to clarify and describe to you and the whole team is how much priority we would give to that effort and its timeframe. Sorry that what we've said might impressed you that we've concluded that we don't need to address that at all. That's not. I agree that we need to address that with some priority.

I understood your analysis on the severity of this issue by your last post. Yes, we're disagreeing on this. Let's wait for others' opinion on this as Kenji suggested. I and Yutaka continue the work at GitHub to figure out resolution for the DoS issue.

 

[Yutaka Hirano]

It seems that we are talking about how much resource we (Chrome) would allow for requests with keepalive specified. I think UA can fail a network request when it doesn't have sufficient resource to process the request in general, and I'm open to have such a limitation if it's simple and reasonable. I agree with Anne's comment on the github issue, and would like to have such a limitation as a UA-specific limitation. We might have a specced limitation in the future, but I hope that will not be a huge interoperability issue.

I propose the following.

1. Add two metrics to measure how many requests with keepalive set (i.e., SendBeacon at this moment) are used per-process and per-browser.

2. Set a per-process limit for requests with keepalive specified.

3. Set a global limit for requests with keepalive specified.

The mechanism will be implemented in /services/network (so that it can be shared with the network service).

I will close the github issue as this is a Chrome-specific limitation.

Are you happy with the idea?

[Yoav Weiss]

Would those limits be imposed on in-flight keep-alive requests? Or on keep-alive requests a process can spin up during its lifetime? (I think the former makes more sense)

[Yutaka Hirano]

They are limitations for the number of in-flight keepalive requests.

[Yoav Weiss]

OK, thanks for clarifying! :)

[Ryan Sleevi]

This sounds like "Leave a browser DoS vector reachable, while we gather metrics"

I do not think that sounds like a good plan.

As suggested several times before in various threads on this topic, I think we should apply simple limits and can refine them - potentially raising them - but I don't feel comfortable shipping a known DoS while we measure what we think to be done to prevent that DoS, given that we've identified the ecosystem risk of being conservative and protecting users is low.

That is, I think the order is 3, 2, 1, rather than 1, 2, 3 :)

[Yutaka Hirano]

That is acceptable for me. I will implement the mechanism next week.

Consequently, we need to set limits without data. Are there specific values in your mind?

Also, my understanding is, this feature is actually blocked by 2 & 3, not 1, which means the order is 2 & 3 => (shipping fetch + keepalive) => 1. Is that right?

[Ryan Sleevi]

Yup

[David Benjamin]

I would suggest a very simple conservative limit to start with. Perhaps even: you only get one request, it must complete in N seconds, and we won't read any response body at all, so you don't get to consume any incoming bandwidth beyond a basic acknowledgement. It's a feature that doesn't exist yet, so we won't be breaking sites by setting limits.

I'll certainly admit that's on the extreme side, but I do think you actually want to start around there. The user promise that closing tabs makes them go away is fundamental to everything that we can do in the web platform, so we cannot loosen it lightly. It's like saying that installed apps shouldn't keep using resources after you uninstall them. We promise our users: Dear user, please enjoy all this great content that developers and publishers have made for you. What happens in one site stays in that site, so explore freely without worry. If you ever don't like what you get, we promise that you can always get rid of it by closing the tab.

There is even some precedent for that limit. sendBeacon was launched without these limits, but that's a bug in sendBeacon. It shipped because of <a ping> as precedent, and <a ping> imposes a natural limit of one request! That sendBeacon forgot to carry it over is a bug in sendBeacon we need to fix, not an allowance for keepalive.

It is much easier to add capabilities later than to remove them, which is why it's so critical to get this resolved before launch. If it turns out we guessed too tightly, we'll get developer feedback and can evaluate that feedback against our (much more important, per the Priority of Constituencies) user promises. If we guessed too loosely and are allowing user harm, it will be much more difficult to rein in that abuse because we risk breaking actual sites.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABihn6EHGrvkpjnx-xF4aCAYsjJ1voLo_pEb30i7SZStBPMW6g%40mail.gmail.com.

[Yoav Weiss]

A limit of one in-flight request seems like something that won't be tenable for the use-cases we want to tackle.

I was talking to a third-party provider the other day and they have not switched to sendBeacon due to a bug where the quota was imposed on overall bytes rather than inflight bytes, and the fact that other third parties often ate up their quota.

Setting an unrealistically harsh limit won't break sites, but will make sure that third party providers continue to use sync XHR, which is not a desired outcome.

[Ilya Grigorik]

I'll second Yoav's concern. Enforcing one in-flight request is far too low and will simply drive people away from using this API.

On Thu, Jan 25, 2018 at 7:44 AM, David Benjamin <davi...@chromium.org> wrote:

There is even some precedent for that limit. sendBeacon was launched without these limits, but that's a bug in sendBeacon. It shipped because of <a ping> as precedent, and <a ping> imposes a natural limit of one request! That sendBeacon forgot to carry it over is a bug in sendBeacon we need to fix, not an allowance for keepalive.

sendBeacon was not motivated by <a ping> and that "limit" is not a good reference point. From the introduction section of the spec: 

"Developers opt for issuing blocking requests via synchronous XMLHttpRequest's, inserting no-op busy loops, or using other techniques that block the user agent from executing time-critical operations (e.g. click, unload, and other handlers) and hurt the user experience. The blocking behavior is used to provide improved delivery rate, as it prevents the user agent and the operating system from cancelling the request if the page is unloaded, suspended, or killed by the system."

[Ryan Sleevi]

On Thu, Jan 25, 2018 at 10:52 AM, Yoav Weiss <yo...@yoav.ws> wrote:

A limit of one in-flight request seems like something that won't be tenable for the use-cases we want to tackle.

I was talking to a third-party provider the other day and they have not switched to sendBeacon due to a bug where the quota was imposed on overall bytes rather than inflight bytes, and the fact that other third parties often ate up their quota.

Setting an unrealistically harsh limit won't break sites, but will make sure that third party providers continue to use sync XHR, which is not a desired outcome.

I think, depending on the implementation state today, it may actually be a desirable outcome! As much as we dislike sync XHRs (and rightfully so), those adhere to all the limits - and predictability bits - that David has mentioned and we've raised concerns about.

I agree that it has other downsides that result in poor user experiences on other dimensions, but in the realm of predictability and stability, it actually seems a better option, based on the information shared so far.

[David Benjamin]

On Thu, Jan 25, 2018 at 11:05 AM Ryan Sleevi <rsl...@chromium.org> wrote:

On Thu, Jan 25, 2018 at 10:52 AM, Yoav Weiss <yo...@yoav.ws> wrote:

A limit of one in-flight request seems like something that won't be tenable for the use-cases we want to tackle.

I was talking to a third-party provider the other day and they have not switched to sendBeacon due to a bug where the quota was imposed on overall bytes rather than inflight bytes, and the fact that other third parties often ate up their quota.

Setting an unrealistically harsh limit won't break sites, but will make sure that third party providers continue to use sync XHR, which is not a desired outcome.

I think, depending on the implementation state today, it may actually be a desirable outcome! As much as we dislike sync XHRs (and rightfully so), those adhere to all the limits - and predictability bits - that David has mentioned and we've raised concerns about.

I agree that it has other downsides that result in poor user experiences on other dimensions, but in the realm of predictability and stability, it actually seems a better option, based on the information shared so far.

Agreed. Always always *always* remember that our first priority is to our users. We *cannot* break our promises to our users.

I'm curious about this overall bytes vs. inflight bytes "bug". Do you mean that, if I each 1GB bandwidth via sendBeacon, as long as I only trickle its usage in 1kb at a time, it's okay? Overall bytes sounds like *exactly* the desirable limit. Using up 1GB slow or fast is still using up 1GB of the user's bandwidth. If the user has no way to cancel the request, it doesn't matter how fast you're doing it.

Users don't care what requests on a site are first-party and third-party. If visiting example.com uses up 1kb of bandwidth for the site and 1MB of bandwidth for analytics and ads and whatnot, that site still cost 1MB. It is a desirable that abusive third-party components of the site will eat up the first-party bandwidth. The first-party should consider switching third-party providers in that case if they are not comfortable being a 1MB site.

[David Benjamin]

On Thu, Jan 25, 2018 at 11:28 AM David Benjamin <davi...@chromium.org> wrote:

On Thu, Jan 25, 2018 at 11:05 AM Ryan Sleevi <rsl...@chromium.org> wrote:

On Thu, Jan 25, 2018 at 10:52 AM, Yoav Weiss <yo...@yoav.ws> wrote:

A limit of one in-flight request seems like something that won't be tenable for the use-cases we want to tackle.

I was talking to a third-party provider the other day and they have not switched to sendBeacon due to a bug where the quota was imposed on overall bytes rather than inflight bytes, and the fact that other third parties often ate up their quota.

Setting an unrealistically harsh limit won't break sites, but will make sure that third party providers continue to use sync XHR, which is not a desired outcome.

I think, depending on the implementation state today, it may actually be a desirable outcome! As much as we dislike sync XHRs (and rightfully so), those adhere to all the limits - and predictability bits - that David has mentioned and we've raised concerns about.

I agree that it has other downsides that result in poor user experiences on other dimensions, but in the realm of predictability and stability, it actually seems a better option, based on the information shared so far.

Agreed. Always always *always* remember that our first priority is to our users. We *cannot* break our promises to our users.

I'm curious about this overall bytes vs. inflight bytes "bug". Do you mean that, if I each 1GB bandwidth via sendBeacon, as long as I only trickle its usage in 1kb at a time, it's okay? Overall bytes sounds like *exactly* the desirable limit. Using up 1GB slow or fast is still using up 1GB of the user's bandwidth. If the user has no way to cancel the request, it doesn't matter how fast you're doing it.

(1GB post-tab-close, of course. If you just meant that the site's post-tab-close limit was being consumed by requests that were scheduled and completed pre-tab-close, then, sure, changing that is reasonable.)

[PhistucK]

> Agreed. Always always *always* remember that our first priority is to our users. We *cannot* break our promises to our users.

Not advocating anything, but the web kind of already did break that promise with service workers that can outlive the tab for ten minutes (or more, with continuous postMessage/onmessage? Or was that resolved already?), or so I have understood or read around here recently.

And to chime in with my unrequested opinion :)

Regarding the overall limit versus per in-flight request limit, there should probably be per-in-flight and overall, not just one or the other.

I do not know what is the limit at the moment and why it was not enough for third parties, but I imagine that a few megabytes are fine. There should be a broader limit for overall-browser-send-beacon usage so that the browser will not waste too much bandwidth. And of course, only requests that outlast the tab should count toward the mentioned limits.

I do agree that limits should be placed now (or yesterday). I also agree that placing those limits is pretty much orthogonal to shipping keepalive, since it offers almost nothing new other than a-single-place-for-initiating-HTTP-requests (unless I am missing something). I personally do not really think that initiating HTTP methods other than POST or cross domain increases the attack vector.

In case shipping keepalive delays shipping the limits (in terms of available work force and not by, say, a day), then, yes, I think shipping the limits should be prioritized.

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAF8qwaBSOSyjurCrCxiKJyNDzKZx5yyefa1Ge%2BQPWDNRhGnmqw%40mail.gmail.com.

[Matt Falkenhagen]

On Fri, Jan 26, 2018 at 4:24 AM, PhistucK <phis...@gmail.com> wrote:

> Agreed. Always always *always* remember that our first priority is to our users. We *cannot* break our promises to our users.

Not advocating anything, but the web kind of already did break that promise with service workers that can outlive the tab for ten minutes (or more, with continuous postMessage/onmessage? Or was that resolved already?), or so I have understood or read around here recently.

You are right that there is some delay, but service workers are designed to go away once they aren't needed (e.g., tab is closed and they aren't getting legitimate events like push notifications). The specification allows the UA to terminate service workers at any time.

Service worker shouldn't be able to keep themselves alive indefinitely. The recursive self postMessage was a bug fixed in: https://bugs.chromium.org/p/chromium/issues/detail?id=647943 and https://bugs.chromium.org/p/chromium/issues/detail?id=648836

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABc02_Kr%2B2PmHf%3Db2RQFrcKhhh1sP9vGGCwfx3BcgiW8ssurPA%40mail.gmail.com.

[Ilya Grigorik]

On Thu, Jan 25, 2018 at 8:28 AM, David Benjamin <davi...@chromium.org> wrote:

On Thu, Jan 25, 2018 at 11:05 AM Ryan Sleevi <rsl...@chromium.org> wrote:

On Thu, Jan 25, 2018 at 10:52 AM, Yoav Weiss <yo...@yoav.ws> wrote:

A limit of one in-flight request seems like something that won't be tenable for the use-cases we want to tackle.

I was talking to a third-party provider the other day and they have not switched to sendBeacon due to a bug where the quota was imposed on overall bytes rather than inflight bytes, and the fact that other third parties often ate up their quota.

Setting an unrealistically harsh limit won't break sites, but will make sure that third party providers continue to use sync XHR, which is not a desired outcome.

I think, depending on the implementation state today, it may actually be a desirable outcome! As much as we dislike sync XHRs (and rightfully so), those adhere to all the limits - and predictability bits - that David has mentioned and we've raised concerns about.

I agree that it has other downsides that result in poor user experiences on other dimensions, but in the realm of predictability and stability, it actually seems a better option, based on the information shared so far.

Agreed. Always always *always* remember that our first priority is to our users. We *cannot* break our promises to our users.

Indeed. I believe we're both arguing for the user here; let's not create any false tensions here. Sync XHRs are not good for the user, and neither is incurring the predictability concerns raised in the thread. 

 

I'm curious about this overall bytes vs. inflight bytes "bug". Do you mean that, if I each 1GB bandwidth via sendBeacon, as long as I only trickle its usage in 1kb at a time, it's okay?

No, and it's not possible in both existing sendBeacon implementation, nor the updated model / Fetch:

• See step 6 in Beacon: https://w3c.github.io/beacon/#sec-processing-model
• See step 4.5.8 in Fetch: https://fetch.spec.whatwg.org/#http-network-or-cache-fetch
• "The above limit ensures that requests that are allowed to outlive the environment settings object and contain a body, have a bounded size and are not allowed to stay alive indefinitely."  

ig

[Yutaka Hirano]

On Fri, Jan 26, 2018 at 8:28 AM, Matt Falkenhagen <fal...@chromium.org> wrote:

On Fri, Jan 26, 2018 at 4:24 AM, PhistucK <phis...@gmail.com> wrote:

> Agreed. Always always *always* remember that our first priority is to our users. We *cannot* break our promises to our users.

Not advocating anything, but the web kind of already did break that promise with service workers that can outlive the tab for ten minutes (or more, with continuous postMessage/onmessage? Or was that resolved already?), or so I have understood or read around here recently.

You are right that there is some delay, but service workers are designed to go away once they aren't needed (e.g., tab is closed and they aren't getting legitimate events like push notifications). The specification allows the UA to terminate service workers at any time.

I believe the browser can abort a network connection due to resource exhaustion, so I don't see a fundamental difference between them.

[Yutaka Hirano]

Status update:

We had a meeting on this topic, and agreed on the following:

 A. Have a relatively loose limit on the number of SendBeacon connections (as it's already shipped).

 B. Have a relatively tight limit on the number of fetch() + keepalive connections (as there is no users yet).

 C. Update both values and unify them in the future based on UMA data and developer feedback.

For A, I set 20 (per process) and 256 (global) for all requests with keepalive set.

For C, I added histograms.

I will do B next week and will resume this thread after that.

Thanks,

[Yutaka Hirano]

On Sat, Feb 3, 2018 at 12:43 AM, Yutaka Hirano <yhi...@chromium.org> wrote:

Status update:

We had a meeting on this topic, and agreed on the following:

 A. Have a relatively loose limit on the number of SendBeacon connections (as it's already shipped).

 B. Have a relatively tight limit on the number of fetch() + keepalive connections (as there is no users yet).

 C. Update both values and unify them in the future based on UMA data and developer feedback.

For A, I set 20 (per process) and 256 (global) for all requests with keepalive set.

For C, I added histograms.

I will do B next week and will resume this thread after that.

Regarding B, we've agreed to start with 10 (per process) for fetch() + keepalive based on some UMA data.

The change is landing, and now I would ask for the approval for shipping.

Thanks!

[Chris Harrelson]

LGTM1

[Ojan Vafai]

LGTM2

LGTM1

 

Thanks,

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAF8qwaBSOSyjurCrCxiKJyNDzKZx5yyefa1Ge%2BQPWDNRhGnmqw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABc02_Kr%2B2PmHf%3Db2RQFrcKhhh1sP9vGGCwfx3BcgiW8ssurPA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw_Taax9P5Nwz%3D7d-7s5FvhQdrDCNfRBPSHYbZc%2BGM3M-w%40mail.gmail.com.

[Yoav Weiss]

LGTM3

[yhi...@chromium.org]

> We have some restrictions for the feature:
> 1. It's disabled on requests which need CORS preflight. Making fetch for such a request

As Out-of-Renderer CORS (OOR-CORS) is rolling out, we are ready to remove this restriction.

I'm going to support CORS preflight on M81.

leads to a failure.On Wednesday, January 3, 2018 at 8:45:40 PM UTC+9, Yutaka Hirano wrote:

Contact emails

yhi...@chromium.org, igrigorik@chromium.org

Spec

- https://fetch.spec.whatwg.org/#request-keepalive-flag

- https://fetch.spec.whatwg.org/#dom-request-keepalive

- https://fetch.spec.whatwg.org/#dom-requestinit-keepalive

Summary

By setting the keepalive flag, a developer can make a fetch which will continue working even when a frame is detached. A web developer can use the feature to report events, state updates and analytics with small amount of data even when the page is about to be unloaded. This is useful for analytics and other cases where async delivery of data is required without blocking navigations, and lessens the need for synchronous XHR which is bad for user experience.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/AUAIHVF63SM

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Debuggability

Nothing

Risks

Interoperability and Compatibility

Edge: Positive based on a conversation between Ilya and Todd. Their Beacon implementation is already on top of Fetch and he hinted that it will be quick and easy for them to expose.

 

Firefox: No signals

https://bugzilla.mozilla.org/show_bug.cgi?id=1342484

Safari: They have implemented (and shipped) the property in Request class, but it looks they haven't implemented the feature itself.

https://bugs.webkit.org/show_bug.cgi?id=168865

https://bugs.webkit.org/show_bug.cgi?id=175482

https://bugs.webkit.org/show_bug.cgi?id=175151

We have some restrictions for the feature:

1. It's disabled on requests which need CORS preflight. Making fetch for such a request

    leads to a failure.

2. The feature is disabled on workers.

Ergonomics

Activation

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Web platform tests:
- https://wpt.fyi/fetch/api/request/request-keepalive.html
- https://wpt.fyi/fetch/api/request/request-keepalive-quota.html
- https://wpt.fyi/fetch/api/basic/keepalive.html

Entry on the feature dashboard

https://www.chromestatus.com/features/5760375567941632

[Yoav Weiss]

On Wed, Jan 15, 2020 at 5:59 PM <yhi...@chromium.org> wrote:

> We have some restrictions for the feature:
> 1. It's disabled on requests which need CORS preflight. Making fetch for such a request

As Out-of-Renderer CORS (OOR-CORS) is rolling out, we are ready to remove this restriction.

I'm going to support CORS preflight on M81.

That's great news! :)

 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/0fb00d1e-ad24-48c6-8139-1befd229dbf4%40chromium.org.