Intent to Ship: NetInfo API extension for network quality

[Tarun Bansal]

Contact emails

tba...@chromium.org, be...@chromium.org, igri...@chromium.org

 

Spec

We shipped NetInfo in Chrome Android M38. This i2s is for new extensions that provide network quality signals to developers: downlink, rtt, effectiveType. Definitions:

• https://wicg.github.io/netinfo/#-dfn-networkinformation-dfn-interface

• https://wicg.github.io/netinfo/#effective-connection-types

 

Summary

The goal is to expose network performance information to developers, as perceived by the UA, in a format that’s easy to consume and act upon: UA monitors latency and throughput of recent requests and provides estimates for RTT, throughput, and effective connection type that developers should optimize for -- e.g. if the recently observed latency and/or throughput is low, the effective connection type will be mapped to a “low” value like 2G or 3G, regardless of the underlying network technology in use.

Notes from BlinkOn, and followup discussions: https://github.com/WICG/netinfo/issues/46#issuecomment-276804272

Note: we have been using these signals internally in Chrome to trigger various interventions where effective connection type is mapped to 2g or slow-2g. This API exposes these signals to developers, allowing them to adapt and optimize their content to avoid the need for such interventions.

 

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/d/msg/blink-dev/TS9zT_u2M4k/ydZK5WpTBwAJ

 

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Currently this is supported only on Chrome OS and Android.

 

 

Debuggability

The network quality estimates are also exposed via chrome://net-internals. Developers can use net-internals, in a local environment to better understand the behavior of the estimation process: open Events tab and look for “network_quality_estimator” records. Developers can also override the network quality estimate using the Chrome switch “force-effective-connection-type” which can be set from chrome://flags or as a command line switch.

 

Interoperability and Compatibility Risk

Low. This is a new extension to the NetInfo API.

 

Is this feature fully tested by web-platform-tests?

Yes, will be. Implementation in progress: https://crbug.com/719108.

 

 

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=723068

 

Entry on the feature dashboard

https://www.chromestatus.com/feature/5108786398232576

 

 

[Johnny Stenback]

Any signals from other browser vendors on this one?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/e208082e-8b52-4dce-9559-6f3945b1fd5f%40chromium.org.

[tba...@google.com]

There have not been any signals from other vendors yet.

[Ben Kelly]

On Thu, May 18, 2017 at 1:24 PM, tbansal via blink-dev <blin...@chromium.org> wrote:

There have not been any signals from other vendors yet.

FWIW mozilla had a discussion about it on our mailing list last year:

https://groups.google.com/forum/#!msg/mozilla.dev.platform/lCZmhCDGHPY/hq3CgPkFDwAJ

I would say there is a fair amount of concern.  I believe we are collecting telemetry to possibly remove NetworkInformation due to privacy concerns.

Ben

[Ilya Grigorik]

Hey Ben.

I'd appreciate any feedback on https://wicg.github.io/netinfo/#privacy-considerations -- that content was the result of a very similar discussion on the earlier i2s for downlinkMax [1]. With regards to other points: multi path is something we considered [2], identifying metered connections would be great but really hard in practice [3], and NQE (I believe) should go a long way towards making the data more actionable. 

[1] https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/tU_Hqqytx8g/xok7kFumAwAJ

[2] https://github.com/WICG/netinfo/pull/29

[3] https://github.com/WICG/netinfo/issues/41

[Ben Kelly]

On Fri, May 19, 2017 at 11:11 AM, Ilya Grigorik <igri...@google.com> wrote:

On Thu, May 18, 2017 at 10:48 AM, Ben Kelly <bke...@mozilla.com> wrote:

On Thu, May 18, 2017 at 1:24 PM, tbansal via blink-dev <blin...@chromium.org> wrote:

There have not been any signals from other vendors yet.

FWIW mozilla had a discussion about it on our mailing list last year:

https://groups.google.com/forum/#!msg/mozilla.dev.platform/lCZmhCDGHPY/hq3CgPkFDwAJ

I would say there is a fair amount of concern.  I believe we are collecting telemetry to possibly remove NetworkInformation due to privacy concerns.

I'd appreciate any feedback on https://wicg.github.io/netinfo/#privacy-considerations -- that content was the result of a very similar discussion on the earlier i2s for downlinkMax [1]. With regards to other points: multi path is something we considered [2], identifying metered connections would be great but really hard in practice [3], and NQE (I believe) should go a long way towards making the data more actionable. 

I asked the folks with concerns to take a look and reply here.  Since I don't personally have an objection I don't want to try to represent their views here.

Thanks.

Ben

[mtho...@mozilla.com]

On Saturday, May 20, 2017 at 2:06:36 AM UTC+10, Ben Kelly wrote:

I asked the folks with concerns to take a look and reply here.  Since I don't personally have an objection I don't want to try to represent their views here.

There were two classes of concerns.  Privacy, and fit-for-purpose.

It seems like the authors of the spec have addressed the privacy concerns with the usual "sure, the house is on fire, but it wasn't *my* firebomb that started it" argument.  In this case, it's a pretty compelling argument: this API doesn't expose anything that a site can't learn in various other ways [1].

The arguments about whether the API is actually any good were a bigger concern.  As with anything we do, the cost-benefit analysis needs to be performed.  What we have here is of questionable value.

This API presents a downlink estimate, which is increasingly meaningless other than as a fingerprinting input.  It's a value that approximates what an attacker might independently measure, but it's a different value.  In exchange for this modest increase in fingerprinting surface is information that is very difficult to use.  The use case that I think is intended is deciding on quality metrics: pushing a bigger video or less degraded images.  This metric isn't suitable for those cases, because what you really need is an *end-to-end* estimate of throughput.  The theoretical speed of the local link is rarely a good proxy for that.  The best - and arguably only - way to arrive at an end-to-end estimate is to do what video sites already do: use the link then measure and adapt.  The browser can't realistically do any different.

The API presents a connection type, which is equally meaningless.  In the discussion we had, it was observed that "is this link free" was the question that this was aimed at answering.  The actual information that can be extracted from a type is marginal.  Again, it does expose information that sites might not have otherwise had.

The RTT measure is underspecified.  Using "recently observed round-trip times on the client", doesn't mean anything.  Is this to the origin only or does it include other origins?  Is this measured at the TCP layer, TLS layer, or HTTP layer?  Is this a minimum, or is it going to be subject to noise induced by packet loss?

I really don't understand the logic behind exposing an "effective network type" and who benefits from it.  If it is a simple table lookup, a site could easily perform the same calculation.  But see the above discussion about the value of the primitives that are input to that calculation.

Finally, the API doesn't acknowledge the possibility that there might be multiple active interfaces.  The API adds "mixed" as a type, but that is likely to be the only value you will ever see on some classes of device.

--Martin

[1] Regarding the privacy considerations, the WebRTC example isn't relevant.  I'd remove it.  It also overstates capabilities, see https://tools.ietf.org/html/draft-ietf-rtcweb-ip-handling for the latest.

[Yoav Weiss]

On Mon, May 22, 2017 at 4:01 AM <mtho...@mozilla.com> wrote:

On Saturday, May 20, 2017 at 2:06:36 AM UTC+10, Ben Kelly wrote:

I asked the folks with concerns to take a look and reply here.  Since I don't personally have an objection I don't want to try to represent their views here.

There were two classes of concerns.  Privacy, and fit-for-purpose.

It seems like the authors of the spec have addressed the privacy concerns with the usual "sure, the house is on fire, but it wasn't *my* firebomb that started it" argument.  In this case, it's a pretty compelling argument: this API doesn't expose anything that a site can't learn in various other ways [1].

The arguments about whether the API is actually any good were a bigger concern.  As with anything we do, the cost-benefit analysis needs to be performed.  What we have here is of questionable value.

This API presents a downlink estimate, which is increasingly meaningless other than as a fingerprinting input.  It's a value that approximates what an attacker might independently measure, but it's a different value.  In exchange for this modest increase in fingerprinting surface is information that is very difficult to use.  The use case that I think is intended is deciding on quality metrics: pushing a bigger video or less degraded images. 

That's not the only use case in mind for this. The use-case this enables is not just quality adaptation, but also content adaptation: e.g. deciding that video is not the right medium due to BW constraints, and presenting images or text instead.

 

This metric isn't suitable for those cases, because what you really need is an *end-to-end* estimate of throughput.

That *is* what the metric provides "based on recently observed throughput on the client".

 

  The theoretical speed of the local link is rarely a good proxy for that.  The best - and arguably only - way to arrive at an end-to-end estimate is to do what video sites already do: use the link then measure and adapt. 

That's feasible for video quality adaptation, but not for images, alternative content, etc.

 

The browser can't realistically do any different.

The API presents a connection type, which is equally meaningless.  In the discussion we had, it was observed that "is this link free" was the question that this was aimed at answering.  The actual information that can be extracted from a type is marginal.  Again, it does expose information that sites might not have otherwise had.

The RTT measure is underspecified.  Using "recently observed round-trip times on the client", doesn't mean anything.  Is this to the origin only or does it include other origins?  Is this measured at the TCP layer, TLS layer, or HTTP layer?  Is this a minimum, or is it going to be subject to noise induced by packet loss?

I agree this would benefit from having a tighter definition. (as would the "observed throughput" definition)

 

I really don't understand the logic behind exposing an "effective network type" and who benefits from it.  If it is a simple table lookup, a site could easily perform the same calculation.  But see the above discussion about the value of the primitives that are input to that calculation.

Sites can do the calculation themselves, but in the context of content-adaptation and a NetInfo related header, this will enable content to vary on a single value, resulting in improved cacheability.

 

Finally, the API doesn't acknowledge the possibility that there might be multiple active interfaces.  The API adds "mixed" as a type, but that is likely to be the only value you will ever see on some classes of device.

--Martin

[1] Regarding the privacy considerations, the WebRTC example isn't relevant.  I'd remove it.  It also overstates capabilities, see https://tools.ietf.org/html/draft-ietf-rtcweb-ip-handling for the latest.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6744a5fc-da9f-43d3-8263-2d7413cf652e%40chromium.org.

[Philip Jägenstedt]

Looks like Mozilla folks are in the loop. Have you tried to reach out to Microsoft and Apple folks in some public place? In addition to poking people on GitHub, an option is to file bugs at https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/new/ and https://webkit.org/new-bug.

I found https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/110956/ but that was probably closed due to no information.

The discussion with Mozilla is still ongoing. If you haven't already done so, can you file a request for review at https://github.com/w3ctag/design-reviews/issues?

There's no hard and fast rule for what blocks shipping and for how long, so whenever you think that all that can be said has been said, if there's still no conclusion, please poke this thread.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/58390de8-9059-4ea6-9eb5-af28f6cee7e7%40chromium.org.

[Martin Thomson]

On Mon, May 22, 2017 at 9:01 PM, Yoav Weiss <yo...@yoav.ws> wrote:
>> The theoretical speed of the local link is rarely a good proxy for that.
>> The best - and arguably only - way to arrive at an end-to-end estimate is to
>> do what video sites already do: use the link then measure and adapt.
>
>
> That's feasible for video quality adaptation, but not for images,
> alternative content, etc.

How do you think that the browser will produce this estimate, if not
using the same technique? Some sites are beefy enough that they get
out of slow start, but many source content from other origins in ways
that ensure that the estimate is guaranteed to be bad.

[Boris Zbarsky]

On 5/22/17 7:01 AM, Yoav Weiss wrote:
> That *is* what the metric provides

> <https://wicg.github.io/netinfo/#dom-networkinformation-downlink> "based

> on recently observed throughput on the client".

There were two distinct concerns brought up in the Mozilla discussion,
which I would like to keep somewhat separate:

1) Security/privacy/fingerprinting concerns.

2) "This API does not address the use cases it's aiming to address"
concerns.

The "downlink" and "effectiveType" and "rtt" attributes are good steps
forward in terms of addressing use cases that were presented during the
discussion. The "type" and "downlinkMax" attributes are at best
misleading except _maybe_ for the case of a cell phone talking to a cell
tower as the first hop; they are not helpful for the use cases that were
presented. (As it happens, they also provide information that the
server can't extract itself, even post-facto, which makes them a bigger
problem in terms of the privacy concerns, but my main concern with them
is that they are just bad API.)

Sadly, the spec document itself does not describe what use cases it's
trying to address (it does the "here's the solution we picked" thing
instead), but as I recall the metered vs non-metered use case was
something people were concerned about and is unaddressed. This is
independent of what happens with "type" and "downlinkMax", since those
don't address that use case anyway.

-Boris

[tba...@google.com]

The current implementation in Chromium computes throughput across

requests (irrespective of their origin).  The algorithm is a bit

heuristic -- A throughput observation is taken when the network is

predicted to be saturated. This prediction can be made on the basis

of number of requests in flight, or by looking at changes in the packet

loss rate etc.

[Ilya Grigorik]

On Sun, May 21, 2017 at 10:01 PM, <mtho...@mozilla.com> wrote:

The arguments about whether the API is actually any good were a bigger concern.  As with anything we do, the cost-benefit analysis needs to be performed.  What we have here is of questionable value.

This API presents a downlink estimate, which is increasingly meaningless other than as a fingerprinting input.  It's a value that approximates what an attacker might independently measure, but it's a different value.  In exchange for this modest increase in fingerprinting surface is information that is very difficult to use.  The use case that I think is intended is deciding on quality metrics: pushing a bigger video or less degraded images.  This metric isn't suitable for those cases, because what you really need is an *end-to-end* estimate of throughput. 

This is a rehash of the earlier discussion in [1]. The tl;dr is: knowing that you're on a downlinkMax ~2G connection is a strong signal regardless of end-to-end measurements -- you're constrained by the last hop and much of the world coming online still falls into this bucket. Similarly, in absence of useful end-to-end measurement (e.g., after the interface has been quiet for a while), you still need to fallback to downlinkMax.

[1] https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/tU_Hqqytx8g/xok7kFumAwAJ

 

The theoretical speed of the local link is rarely a good proxy for that.  The best - and arguably only - way to arrive at an end-to-end estimate is to do what video sites already do: use the link then measure and adapt.  The browser can't realistically do any different.

That's exactly what this i2s is providing. 

On Mon, May 22, 2017 at 7:01 AM, Yoav Weiss <yo...@yoav.ws> wrote:

On Mon, May 22, 2017 at 4:01 AM <mtho...@mozilla.com> wrote:

On Saturday, May 20, 2017 at 2:06:36 AM UTC+10, Ben Kelly wrote:

The RTT measure is underspecified.  Using "recently observed round-trip times on the client", doesn't mean anything.  Is this to the origin only or does it include other origins?  Is this measured at the TCP layer, TLS layer, or HTTP layer?  Is this a minimum, or is it going to be subject to noise induced by packet loss?

I agree this would benefit from having a tighter definition. (as would the "observed throughput" definition)

Updated in https://github.com/WICG/netinfo/issues/56 -- if you have other suggestions, please chime in there!

On Mon, May 22, 2017 at 11:38 AM, Philip Jägenstedt <foo...@chromium.org> wrote:

Looks like Mozilla folks are in the loop. Have you tried to reach out to Microsoft and Apple folks in some public place? In addition to poking people on GitHub, an option is to file bugs at https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/new/ and https://webkit.org/new-bug.

I found https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/110956/ but that was probably closed due to no information.

Edge uservoice request has been open for a while. Opened new bug on Webkit tracker. 

 

If you haven't already done so, can you file a request for review at https://github.com/w3ctag/design-reviews/issues?

From: https://github.com/w3ctag/design-reviews/issues/86

"In terms of API design, this seems reasonable. We understand that there are a number of trade offs in this design space, and that there is some risk here. However, we feel that publishing something in this space is generally good, because it meets a significant need in the platform -- even if it is not initially perfect. We'd encourage you to continue to gather information from developers and discuss it with other implementers."

On Mon, May 22, 2017 at 9:14 PM, Boris Zbarsky <bzba...@mit.edu> wrote:

2)  "This API does not address the use cases it's aiming to address" concerns.

The "downlink" and "effectiveType" and "rtt" attributes are good steps forward in terms of addressing use cases that were presented during the discussion.  The "type" and "downlinkMax" attributes are at best misleading except _maybe_ for the case of a cell phone talking to a cell tower as the first hop; they are not helpful for the use cases that were presented. 

Right. The key observation (and motivation for this API) to keep in mind: that condition holds for significant fraction of the billions coming online.

 

Sadly, the spec document itself does not describe what use cases it's trying to address (it does the "here's the solution we picked" thing instead), but as I recall the metered vs non-metered use case was something people were concerned about and is unaddressed. 

There is an ongoing discussion about this in https://github.com/WICG/netinfo/issues/41. I don't think we need to block this i2s on that, and I'd appreciate your input on that thread.

---------

Stepping back, a few points: 

1. Chrome for Android shipped support for downlinkMax in M48. 
2. This i2s uses (1) as a building block to enable network quality signals.
3. We're still trying to figure out if and how we can report "metered" signals -- this doesn't block (2).

Given that most of our users coming online are on slow (~2G~3G) connections, we need to enable developers to make the right tradeoffs and optimization decisions. The list of interventions that we (Chrome) trigger on such connections continues to grow—and is based on the same signals we're exposing here—and we need to give developers the signals and tools to do the right thing, such that we can stop adding and forcing these interventions on their behalf. Also, as a side note, I don't think this is a pain shared equally by all browsers, which is reflected in varying levels (read, lack of.. for the most part) of engagement in this space.

If there are still gaps in the spec definitions for the signals we're proposing here, I'm happy to iterate on that -- please file bugs on GH! So far, I believe we've addressed all the issues raised above.

On Mon, May 22, 2017 at 11:38 AM, Philip Jägenstedt <foo...@chromium.org> wrote:

There's no hard and fast rule for what blocks shipping and for how long, so whenever you think that all that can be said has been said, if there's still no conclusion, please poke this thread.

Poking it now :-). Personally, I'd like push for a higher sense of urgency here for reasons stated above.

[Rick Byers]

LGTM1.  

I agree this is urgent for our user base, and likely less so for other engines.  We've already shipped the most contentious bit here, and this intent is about addressing the legitimate feedback that downlink is often insufficient.

On May 26, 2017 10:48 AM, "'Ilya Grigorik' via blink-dev" <blin...@chromium.org> wrote:

On Sun, May 21, 2017 at 10:01 PM, <mtho...@mozilla.com> wrote:

The arguments about whether the API is actually any good were a bigger concern.  As with anything we do, the cost-benefit analysis needs to be performed.  What we have here is of questionable value.

This API presents a downlink estimate, which is increasingly meaningless other than as a fingerprinting input.  It's a value that approximates what an attacker might independently measure, but it's a different value.  In exchange for this modest increase in fingerprinting surface is information that is very difficult to use.  The use case that I think is intended is deciding on quality metrics: pushing a bigger video or less degraded images.  This metric isn't suitable for those cases, because what you really need is an *end-to-end* estimate of throughput. 

This is a rehash of the earlier discussion in [1]. The tl;dr is: knowing that you're on a downlinkMax ~2G connection is a strong signal regardless of end-to-end measurements -- you're constrained by the last hop and much of the world coming online still falls into this bucket. Similarly, in absence of useful end-to-end measurement (e.g., after the interface has been quiet for a while), you still need to fallback to downlinkMax.

[1] https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/tU_Hqqytx8g/xok7kFumAwAJ

 

The theoretical speed of the local link is rarely a good proxy for that.  The best - and arguably only - way to arrive at an end-to-end estimate is to do what video sites already do: use the link then measure and adapt.  The browser can't realistically do any different.

That's exactly what this i2s is providing. 

On Mon, May 22, 2017 at 7:01 AM, Yoav Weiss <yo...@yoav.ws> wrote:

On Mon, May 22, 2017 at 4:01 AM <mtho...@mozilla.com> wrote:

On Saturday, May 20, 2017 at 2:06:36 AM UTC+10, Ben Kelly wrote:

The RTT measure is underspecified.  Using "recently observed round-trip times on the client", doesn't mean anything.  Is this to the origin only or does it include other origins?  Is this measured at the TCP layer, TLS layer, or HTTP layer?  Is this a minimum, or is it going to be subject to noise induced by packet loss?

I agree this would benefit from having a tighter definition. (as would the "observed throughput" definition)

Updated in https://github.com/WICG/netinfo/issues/56 -- if you have other suggestions, please chime in there!

On Mon, May 22, 2017 at 11:38 AM, Philip Jägenstedt <foolip@chromium.org> wrote:

Looks like Mozilla folks are in the loop. Have you tried to reach out to Microsoft and Apple folks in some public place? In addition to poking people on GitHub, an option is to file bugs at https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/new/ and https://webkit.org/new-bug.

I found https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/110956/ but that was probably closed due to no information.

Edge uservoice request has been open for a while. Opened new bug on Webkit tracker. 

 

If you haven't already done so, can you file a request for review at https://github.com/w3ctag/design-reviews/issues?

From: https://github.com/w3ctag/design-reviews/issues/86

"In terms of API design, this seems reasonable. We understand that there are a number of trade offs in this design space, and that there is some risk here. However, we feel that publishing something in this space is generally good, because it meets a significant need in the platform -- even if it is not initially perfect. We'd encourage you to continue to gather information from developers and discuss it with other implementers."

On Mon, May 22, 2017 at 9:14 PM, Boris Zbarsky <bzba...@mit.edu> wrote:

2)  "This API does not address the use cases it's aiming to address" concerns.

The "downlink" and "effectiveType" and "rtt" attributes are good steps forward in terms of addressing use cases that were presented during the discussion.  The "type" and "downlinkMax" attributes are at best misleading except _maybe_ for the case of a cell phone talking to a cell tower as the first hop; they are not helpful for the use cases that were presented. 

Right. The key observation (and motivation for this API) to keep in mind: that condition holds for significant fraction of the billions coming online.

 

Sadly, the spec document itself does not describe what use cases it's trying to address (it does the "here's the solution we picked" thing instead), but as I recall the metered vs non-metered use case was something people were concerned about and is unaddressed. 

There is an ongoing discussion about this in https://github.com/WICG/netinfo/issues/41. I don't think we need to block this i2s on that, and I'd appreciate your input on that thread.

---------

Stepping back, a few points: 

1. Chrome for Android shipped support for downlinkMax in M48. 
2. This i2s uses (1) as a building block to enable network quality signals.
3. We're still trying to figure out if and how we can report "metered" signals -- this doesn't block (2).

Given that most of our users coming online are on slow (~2G~3G) connections, we need to enable developers to make the right tradeoffs and optimization decisions. The list of interventions that we (Chrome) trigger on such connections continues to grow—and is based on the same signals we're exposing here—and we need to give developers the signals and tools to do the right thing, such that we can stop adding and forcing these interventions on their behalf. Also, as a side note, I don't think this is a pain shared equally by all browsers, which is reflected in varying levels (read, lack of.. for the most part) of engagement in this space.

If there are still gaps in the spec definitions for the signals we're proposing here, I'm happy to iterate on that -- please file bugs on GH! So far, I believe we've addressed all the issues raised above.

On Mon, May 22, 2017 at 11:38 AM, Philip Jägenstedt <foolip@chromium.org> wrote:

There's no hard and fast rule for what blocks shipping and for how long, so whenever you think that all that can be said has been said, if there's still no conclusion, please poke this thread.

Poking it now :-). Personally, I'd like push for a higher sense of urgency here for reasons stated above.

On Tue, May 23, 2017 at 1:42 PM, <tba...@google.com> wrote:

On Monday, May 22, 2017 at 4:45:04 PM UTC-7, Martin Thomson wrote:

On Mon, May 22, 2017 at 9:01 PM, Yoav Weiss <yo...@yoav.ws> wrote:
>>   The theoretical speed of the local link is rarely a good proxy for that.
>> The best - and arguably only - way to arrive at an end-to-end estimate is to
>> do what video sites already do: use the link then measure and adapt.
>
>
> That's feasible for video quality adaptation, but not for images,
> alternative content, etc.

How do you think that the browser will produce this estimate, if not
using the same technique?  Some sites are beefy enough that they get
out of slow start, but many source content from other origins in ways
that ensure that the estimate is guaranteed to be bad.

The current implementation in Chromium computes throughput across

requests (irrespective of their origin).  The algorithm is a bit

heuristic -- A throughput observation is taken when the network is

predicted to be saturated. This prediction can be made on the basis

of number of requests in flight, or by looking at changes in the packet

loss rate etc.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXXVKq50-wgiQZeSi5WigTBFBZFTN_hvERsg0WDu5xcRr7HKA%40mail.gmail.com.

[Boris Zbarsky]

On 5/26/17 10:48 AM, 'Ilya Grigorik' via blink-dev wrote:
> Right. The key observation (and motivation for this API) to keep in
> mind: that condition holds for significant fraction of the billions
> coming online.

Not really.

For the "cell phone talking to a cell tower" case, this API always
reports "cellular" as the connectionType. That tells you absolutely
nothing; it might be 2g or it might be 4g.

So even for the one use case when the first hop is at least likely to be
the constraint on the connection, this API lumps in the cases when it is
with the ones when it's not. And of course whether the first link is
"ethernet" or "wifi" is really no one's business and no use on its own
as a piece of information.

So I still don't see any valid use cases being presented for
connectionType if we're really talking about the network performance end
of this.

The downlinkMax attribute _does_ report more fine-grained information,
of course. But I still see no need for distinguishing between the
various kinds of "wifi", or the various forms of "ethernet". There
might be something to say about the different "bluetooth"s. But is
there a practical reason to distinguish between CDMA and 1xRTT, say?

I, personally, would be much happier if we had a few buckets for the
downlinkMax, insteaad of a huge table of lots of possible values. For
the use cases I've seen described that would be sufficient.

> There is an ongoing discussion about this
> in https://github.com/WICG/netinfo/issues/41. I don't think we need to
> block this i2s on that

I'm not trying to block the intent to ship. I'm just saying that I will
oppose such an intent to ship in _Firefox_, because I think this is a
bad API in its current form. Whether you care about that is, of course,
up to you.

This is about as much time as I have to spend on this at the moment,
honestly.

> and I'd appreciate your input on that thread.

Please feel free to repost whatever you want from my mail in that
thread. I'm not going to get involved in yet another interminable
github discussion where people are more or less talking past each other,
sorry. I have much more productive ways to spend my time.

> Given that most of our users coming online are on slow (~2G~3G)
> connections, we need to enable developers to make the right tradeoffs
> and optimization decisions. The list of interventions that we (Chrome)
> trigger on such connections continues to grow—and is based on the same
> signals we're exposing here

I don't have a problem with a limited bucketing into "slow" and "fast"
connections, possibly even at a few levels. This API is going _way_
beyond that, for no good reason I can see.

-Boris

[Boris Zbarsky]

On 5/28/17 10:47 PM, Boris Zbarsky wrote:
> I'm not trying to block the intent to ship. I'm just saying that I will
> oppose such an intent to ship in _Firefox_, because I think this is a
> bad API in its current form.

And to be clear, on the Firefox end I would insist that we not ship or
neuter (e.g. have it always report the same value) the connectionType,
and bucket downlinkMax.

The other parts of the API are much more reasonable, as I said above;
the look pretty good to me at first glance. Again, this is for your
information only, and only to the extent that you are trying to
determine what the actual positions of other UAs are; past that whatever
I have to say isn't really a part of your intent process anyway.

-Boris

[Boris Zbarsky]

On 5/28/17 10:47 PM, Boris Zbarsky wrote:

> I'm not trying to block the intent to ship. I'm just saying that I will
> oppose such an intent to ship in _Firefox_, because I think this is a
> bad API in its current form. Whether you care about that is, of course,
> up to you.

I just realized this part was ambiguous. In the last sentence, "that"
refers to what Firefox ships, not the quality of the API. I'm quite
certain you care about the quality of the API.

-Boris

[Philip Jägenstedt]

On Mon, May 22, 2017 at 5:38 PM Philip Jägenstedt <foo...@chromium.org> wrote:

Have you tried to reach out to Microsoft and Apple folks in some public place? In addition to poking people on GitHub, an option is to file bugs at https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/new/ and https://webkit.org/new-bug.

Ping for this bit. 

[Philip Jägenstedt]

First the things that are new for this intent, i.e. navigator.connection.effectiveType/downlink/rtt:

The wording for downlink & rtt was recently clarified. Martin, do you think that is now clear enough to allow for an independent implementation?

The launch bug has been marked as needing privacy review, and I've filed https://github.com/WICG/netinfo/issues/58, pardon if that's just retreading old ground.

I'll check back here later this week, to allow Martin some time to comment.

As for effectiveType, it simply depends on those two values and doesn't expose any additional information. That presumably makes it "a simple table lookup", but also makes it fairly harmless. It's plausible to me that it could be used to decide whether to autoplay (muted) videos, for example, or any other binary choice.

Regarding navigator.connection.type/downlinkMax, then, which are not part of this intent.

The original Intent to Ship: Network Information API was for navigator.connection.type and a related event. (Also referred to as connectionType in this thread because the enum is called ConnectionType.) On the compat risk, that intent said "Firefox shipped on Android and FFOS in 31. Desktop support coming." It is in Gecko's IDL and I've confirmed that Firefox for Android returns "wifi" for navigator.connection.type in a test, and that it's still not enabled on Desktop.

downlinkMax was added in a later intent, and then there appears to have been interest for FFOS and +Marcos Caceres was mentioned and included in the thread.

It looks to me like the 3 new attributes address some of the shortcomings of the 2 existing attributes, and are orthogonal in their definitions rather than building upon the old ones.

It looks like neither Edge nor Safari have shipped navigator.connection on Desktop or Mobile, and WebKit doesn't have it in their IDL. Working out the fate of the 2 existing attributes would be good, but AFAICT only loosely connected to this intent.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/107225a9-7f49-00ef-f4ff-2f5adc5a6167%40mit.edu.

[Jochen Eisinger]

The chromstatus entry says no signal for other web browsers, but it looks like the signals are at least "concerned". It also lists positive signals from web developers - can you point at some statements I could read to better understand the use cases?

It looks a bit like a service could already use existing signals (mobile UA, IP from range that belongs to a mobile carrier) deduce a fair bit of information, so I share Mozilla's privacy concerns here.

On the other hand, I wonder whether e.g. an ad network that wants to push a large video would refrain from doing so if they had a signal that there's limited bandwidth, or whether we should rather go into the direction of feature policies and tell a site that it doesn't get to download more than xxx KB of media, and block it if it tries to?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYdw2M%2BZmCY8rVYe%3DbfqJaOxmETYBKttYeN7Y%3D7YD6LNUQ%40mail.gmail.com.

[Yoav Weiss]

On Tue, May 30, 2017 at 3:14 PM Jochen Eisinger <joc...@chromium.org> wrote:

The chromstatus entry says no signal for other web browsers, but it looks like the signals are at least "concerned". It also lists positive signals from web developers - can you point at some statements I could read to better understand the use cases?

I wrote about the use-cases here: https://blog.yoav.ws/adapting_without_assumptions/

Akamai already performs network-related adaptations (using various heuristics), but network information from the browser would be very useful to that end.

Facebook was also involved in the API discussions, so it seems like they are supportive.

+bmaurer - who may have more to say on Facebook's support.

 

It looks a bit like a service could already use existing signals (mobile UA, IP from range that belongs to a mobile carrier) deduce a fair bit of information, so I share Mozilla's privacy concerns here.

On the other hand, I wonder whether e.g. an ad network that wants to push a large video would refrain from doing so if they had a signal that there's limited bandwidth, or whether we should rather go into the direction of feature policies and tell a site that it doesn't get to download more than xxx KB of media, and block it if it tries to?

I don't think it's either/or. In the use-case you're describing it is legitimate for the ad network to serve a video when there's enough bandwidth to download it, but they should avoid it when there isn't.

To that end, you need to provide the ad network with a network signal, otherwise they will not be able to adapt their content.

Then we can add a "don't download heavy media in low BW conditions" policy on top of that, and "punish" the ad network when it downloads too much content.

[Philip Jägenstedt]

It's later in the week now.

Ilya has filed https://bugs.webkit.org/show_bug.cgi?id=172643 and I've just filed https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/12189241/

LGTM2, with the caveat that it's not an expression of confidence that there's not a privacy problem here, I will leave that to privacy review and https://github.com/WICG/netinfo/issues/58.

On Mon, May 29, 2017 at 4:47 PM Philip Jägenstedt <foo...@chromium.org> wrote:

[Ilya Grigorik]

On Sun, May 28, 2017 at 7:47 PM, Boris Zbarsky <bzba...@mit.edu> wrote:

The downlinkMax attribute _does_ report more fine-grained information, of course.  But I still see no need for distinguishing between the various kinds of "wifi", or the various forms of "ethernet".  There might be something to say about the different "bluetooth"s.  But is there a practical reason to distinguish between CDMA and 1xRTT, say?

I, personally, would be much happier if we had a few buckets for the downlinkMax, insteaad of a huge table of lots of possible values.  For the use cases I've seen described that would be sufficient.

Thanks, opened GH issues to track this: https://github.com/WICG/netinfo/issues/60 - note: I don't believe this is blocking for this i2s.

On Tue, May 30, 2017 at 6:40 AM, Yoav Weiss <yo...@yoav.ws> wrote:

On Tue, May 30, 2017 at 3:14 PM Jochen Eisinger <joc...@chromium.org> wrote:

The chromstatus entry says no signal for other web browsers, but it looks like the signals are at least "concerned". It also lists positive signals from web developers - can you point at some statements I could read to better understand the use cases?

Facebook was also involved in the API discussions, so it seems like they are supportive.

There was also a lot of interest in it when we discussed it at one of the breakouts at last BlinkOn -- see notes; not sure if it was recorded.

On Wed, May 31, 2017 at 7:10 AM, Philip Jägenstedt <foo...@chromium.org> wrote:

downlinkMax was added in a later intent, and then there appears to have been interest for FFOS and +Marcos Caceres was mentioned and included in the thread.

It looks to me like the 3 new attributes address some of the shortcomings of the 2 existing attributes, and are orthogonal in their definitions rather than building upon the old ones.

Not entirely. In absence of recent (end-to-end) network activity data effectiveType does fallback to downlinkMax value of the first network hop. I think the underlying question in some of the above discussions is: the ECT attributes we're discussing in this i2s improve the signal we expose to developers by taking into account both the end-to-end and first hop properties of the client and, as such, do we still need to expose the first hop as a standalone signal? 

My personal take is.. In the short~medium term, yes: NetInfo.type usage is ~0.9% [1] and NetInfo.downlinkMax ~0.5% [2]. In the longer term, maybe not: if we can drive adoption of ECT attributes, perhaps we can sunset type and downlinkMax down the road.

[1] https://www.chromestatus.com/metrics/feature/timeline/popularity/946

[2] https://www.chromestatus.com/metrics/feature/timeline/popularity/947

LGTM2, with the caveat that it's not an expression of confidence that there's not a privacy problem here, I will leave that to privacy review and https://github.com/WICG/netinfo/issues/58.

Followed up on the issue. If anyone else has suggestions or questions, please chime in on the GH thread.

[Boris Zbarsky]

On 6/2/17 12:57 PM, Ilya Grigorik wrote:
> Thanks, opened GH issues to track this:
> https://github.com/WICG/netinfo/issues/60

Thank you.

> - note: I don't believe this is blocking for this i2s.

Agreed; this i2s is about the new bits; that issue is about the old bits.

-Boris

[Jochen Eisinger]

Rick pinged me on the intents spreadsheet, however, I still feel I don't have all the information I'd need to approve this.

We've shipped a version of the API that other browser vendors are actively opposed to (issue 60), and the extension to the surface isn't ultimately answering the question whether your connection is metered or not, but exposes cross origin information (issue 58).

Have we considered exposing whether the user configured the OS to consider the connection as metered, and whether the browser will apply interventions for slow networks or not?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6d04148e-6f4f-5d79-329e-09e4a807d7b0%40mit.edu.

[Marcos Caceres]

On June 5, 2017 at 4:48:54 PM, Jochen Eisinger (joc...@chromium.org) wrote:
> Have we considered exposing whether the user configured the OS to consider
> the connection as metered, and whether the browser will apply interventions
> for slow networks or not?

(Editor's hat on...)

I'm not a networking expert, so I don't have any opinion about the new
attributes that have been added to the spec (apart that I trust Ilya,
Yoav, et al., who are experts, to do the due diligence on the use
cases).

Regarding "metered", those of us that worked on the spec have
considered it in the past. However, people wanted some kind of
solution whereby you could magically determined if the connection was
tethered. We should just stop and give up on trying to find that magic
solution.

So for metered, we should just agree to use the explicit signal from
the OS (like you can set in Windows 10 on a connection-by-connection
basis).

Again, for the spec, I've been primarily interested in solving the use
cases from:
https://www.w3.org/TR/netinfo-usecases/

So people can build those kinds of UIs into their apps. Opinions will
vary if people find the above UI controls useful. I personally found
them very useful in places like Canada and in Australia, where I've
been capped to 4gb of data per month.

I'll pre-emptively note that blanket "don't allow App X to use
connection Y" don't work well - because often you need fine grained
control over what gets downloaded over a cellular connection (e.g.,
for a the Audible app, I want to be able to refresh my library and
browse "What's New" in the store, but not actually allow d/l books
over a metered connection).

</rant>

Marcos

[Yoav Weiss]

On Mon, Jun 5, 2017 at 9:19 AM Marcos Caceres <mcac...@mozilla.com> wrote:

On June 5, 2017 at 4:48:54 PM, Jochen Eisinger (joc...@chromium.org) wrote:
> Have we considered exposing whether the user configured the OS to consider
> the connection as metered, and whether the browser will apply interventions
> for slow networks or not?

(Editor's hat on...)

I'm not a networking expert, so I don't have any opinion about the new
attributes that have been added to the spec (apart that I trust Ilya,
Yoav, et al., who are experts, to do the due diligence on the use
cases).

Regarding "metered", those of us that worked on the spec have
considered it in the past. However, people wanted some kind of
solution whereby you could magically determined if the connection was
tethered. We should just stop and give up on trying to find that magic
solution.

So for metered, we should just agree to use the explicit signal from
the OS (like you can set in Windows 10 on a connection-by-connection
basis).

I'd be wary about exposing that as a "metered" signal, because we need to make it explicit that a lack of signal here does not mean the connection is not metered, simply that the user hasn't explicitly set it as such in their OS (which I suspect most users won't).

I'm not saying that we should wait for a "magic" solution here, as it's not likely to come, but we should give the "metered" signal some more thought (e.g. maybe the user-set OS connection signal should be folded into the existing "Save-Data" client hint?)

As the "metered" signal is orthogonal to the bandwidth ones from a content-adaptation perspective, I don't think that we should block the current intent on decision on that front.

[Marcos Caceres]

On June 5, 2017 at 5:52:04 PM, Yoav Weiss (yo...@yoav.ws) wrote:

> On Mon, Jun 5, 2017 at 9:19 AM Marcos Caceres wrote:
> I'd be wary about exposing that as a "metered" signal, because we need to
> make it explicit that a lack of signal here does not mean the connection is
> not metered, simply that the user hasn't explicitly set it as such in their
> OS (which I suspect most users won't).
>
> I'm not saying that we should wait for a "magic" solution here, as it's not
> likely to come, but we should give the "metered" signal some more thought
> (e.g. maybe the user-set OS connection signal should be folded into the
> existing "Save-Data" client hint?)
>
> As the "metered" signal is orthogonal to the bandwidth ones from a
> content-adaptation perspective, I don't think that we should block the
> current intent on decision on that front.

We should probably move this back to the repo at this point - this is
now moving back into a standards discussion.

I've filed: https://github.com/WICG/netinfo/issues/61

[Ilya Grigorik]

On Sun, Jun 4, 2017 at 11:48 PM, Jochen Eisinger <joc...@chromium.org> wrote:

Rick pinged me on the intents spreadsheet, however, I still feel I don't have all the information I'd need to approve this.

We've shipped a version of the API that other browser vendors are actively opposed to (issue 60), and the extension to the surface isn't ultimately answering the question whether your connection is metered or not,

For good reasons and not for lack of trying.. I've tried to capture the latest status here:

https://github.com/WICG/netinfo/issues/41#issuecomment-306221378

I don't believe this i2s should block on the above.

 

exposes cross origin information (issue 58).

Responded to this in https://github.com/WICG/netinfo/issues/58#issuecomment-305835868.

---

To untangle some of the threads: 

1. This i2s is to enable and ship ECT signals: effectiveType, rtt, downlink. 
• I don't believe there are any outstanding objections to these, based on discussion above.
  
  
2. We've opened a new bug to revisit connection.type + downlinkMax bucketing
• Discussion: https://github.com/WICG/netinfo/issues/60
• Shipping desktop support is blocked on resolution of above discussion.
  
  
3. There is demand for "is connection expensive" signal...
• This is the least mature and under active discussion: https://github.com/WICG/netinfo/issues/41

Anything else I'm missing? I don't believe (1) should be blocked by (2) or (3).

ig

[Philip Jägenstedt]

On Fri, Jun 2, 2017 at 6:58 PM Ilya Grigorik <igri...@google.com> wrote:

On Wed, May 31, 2017 at 7:10 AM, Philip Jägenstedt <foo...@chromium.org> wrote:

downlinkMax was added in a later intent, and then there appears to have been interest for FFOS and +Marcos Caceres was mentioned and included in the thread.

It looks to me like the 3 new attributes address some of the shortcomings of the 2 existing attributes, and are orthogonal in their definitions rather than building upon the old ones.

Not entirely. In absence of recent (end-to-end) network activity data effectiveType does fallback to downlinkMax value of the first network hop. I think the underlying question in some of the above discussions is: the ECT attributes we're discussing in this i2s improve the signal we expose to developers by taking into account both the end-to-end and first hop properties of the client and, as such, do we still need to expose the first hop as a standalone signal? 

My personal take is.. In the short~medium term, yes: NetInfo.type usage is ~0.9% [1] and NetInfo.downlinkMax ~0.5% [2]. In the longer term, maybe not: if we can drive adoption of ECT attributes, perhaps we can sunset type and downlinkMax down the road.

[1] https://www.chromestatus.com/metrics/feature/timeline/popularity/946

[2] https://www.chromestatus.com/metrics/feature/timeline/popularity/947

 

Following up on the orthogonality here. In the code it looks like downlink, rtt and effectiveType all fall back to other values when "not observing," e.g. rtt will be 0. (Filed https://github.com/WICG/netinfo/issues/63)

In practice, would this happen every time the phone has been turned off for 5 minutes, or in some much rarer circumstance?

It seems reasonable to have this fallback, so that users of the API can just use the values and have things work well enough. Absent any randomization, it will mean that it's possible to tell when the device hasn't been recently active at all, is that worth spelling out in https://wicg.github.io/netinfo/#privacy-considerations?

[Chris Harrelson]

LGTM3

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYdRAdaZZ6uV_M454eECejknAtqp%3D494_wTQsSZjxZQE7w%40mail.gmail.com.

[Rick Byers]

Thanks Chris.

Note that we discussed this in the API Owners meeting today - notes for anyone interested are at http://bit.ly/api-owners-weekly (though we've only been using these for the rare case where there's interesting discussion/debate, almost all intents have been resolved directly on the list lately).

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw_fYaAVf--gsSvMXsge1hkx2ntzznJ3KH1vL9rVJ2eDpQ%40mail.gmail.com.