Intent to Ship: Restrict "private network requests" for subresources to secure contexts.

[Titouan Rigoudy]

Contact emails

tit...@chromium.org, cl...@chromium.org, mk...@chromium.org, va...@chromium.org

Explainer

https://github.com/WICG/cors-rfc1918/blob/master/explainer.md

Specification

https://wicg.github.io/cors-rfc1918/

API spec

Yes

Design docs

https://docs.google.com/document/d/1tIl8F0i31_z6gttaaPmRDynHBqWsLaV_94ngrpg5Oeg/edit#heading=h.lnbc74amxd9e
https://docs.google.com/document/d/15r4PQdHJa3SunkGg0RDgm6MlcCdWSZLJj9Cq2awSLvo/edit

Summary

Requires that private network requests for subresources may only be initiated from a secure context. "Private network requests" are those initiated from a public network, targeting a private network. Examples include internet to intranet requests and intranet loopbacks. This is a first step towards fully implementing CORS-RFC1918: https://chromestatus.com/feature/5733828735795200

Blink component

Blink

TAG review

https://github.com/w3ctag/design-reviews/issues/572

TAG review status

Pending

Risks

Interoperability and Compatibility

< 1% of page views are currently loading at least one resource from a loopback address, and this change would affect them in one way or another: - https://www.chromestatus.com/metrics/feature/timeline/popularity/3689 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3691 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3693 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3695 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3697

Note that the above metrics almost certainly overestimate the affected page loads as some frame hierarchies currently confuse Blink: https://crbug.com/1136028. Previous estimates based on less precise metrics suggested that this was much less widespread.

In any case, we are waiting for more detailed metrics gathered in M87 to finalize our plan to ship the feature. We have an enterprise opt-out through enterprise policies. This is a bit unfortunate, as enterprises are exactly the sort of things that have large local networks full of interesting resources, but necessary.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/143) https://bugzilla.mozilla.org/show_bug.cgi?id=1481298

WebKit: No signal

Web developers: Negative (https://bugs.webkit.org/show_bug.cgi?id=171934) Developers running local servers generally want those servers to keep running without alteration. The conversation on a tangentally-related WebKit bug is representative.

Activation

Developers of non-secure sites that rely upon local servers will need to upgrade to HTTPS. This might cause some complications, as mixed-content checks will begin to apply. Chrome carves out HTTP access to loopback (as perhttps://w3c.github.io/webappsec-secure-contexts/#localhost), which is a release valve for folks who don't want to go through the effort of securely-distributing certs for local servers.

See also the discussion with Plex, which embed videos served from the private network into their public webapp: https://github.com/WICG/cors-rfc1918/issues/23. We are fairly confident that other such issues can be resolved on a case-by-case basis.

Security

This change should be security-positive.

Debuggability

When blocking a request due to a secure-context restriction, we'll add a helpful message to the console. The devtools network panel will show information about the source and remote address spaces at play.

Is this feature fully tested by web-platform-tests?

No. Further tests are on their way. Full coverage will require a change to WPTs, RFC currently under review: https://github.com/web-platform-tests/rfcs/pull/72

Tracking bug

https://crbug.com/986744

Launch bug

https://crbug.com/1129801

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5436853517811712

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/EeGg7TxW6U4/m/7ZvqAqHLAwAJ
Ready for Trial: https://groups.google.com/a/chromium.org/g/blink-dev/c/EeGg7TxW6U4/m/7ZvqAqHLAwAJ

This intent message was generated by Chrome Platform Status.

[yo...@yoav.ws]

On Friday, November 20, 2020 at 10:47:51 PM UTC+1 Titouan Rigoudy wrote:

Contact emails

tit...@chromium.org, cl...@chromium.org, mk...@chromium.org, va...@chromium.org

Explainer

https://github.com/WICG/cors-rfc1918/blob/master/explainer.md

Specification

https://wicg.github.io/cors-rfc1918/

API spec

Yes

Design docs

https://docs.google.com/document/d/1tIl8F0i31_z6gttaaPmRDynHBqWsLaV_94ngrpg5Oeg/edit#heading=h.lnbc74amxd9e
https://docs.google.com/document/d/15r4PQdHJa3SunkGg0RDgm6MlcCdWSZLJj9Cq2awSLvo/edit

Summary

Requires that private network requests for subresources may only be initiated from a secure context. "Private network requests" are those initiated from a public network, targeting a private network. Examples include internet to intranet requests and intranet loopbacks.

To clarify, the network requests to those private networks will be done over non-secure HTTP, and will not be considered mixed content?

 

This is a first step towards fully implementing CORS-RFC1918: https://chromestatus.com/feature/5733828735795200

Blink component

Blink

TAG review

https://github.com/w3ctag/design-reviews/issues/572

TAG review status

Pending

Risks

Interoperability and Compatibility

< 1% of page views are currently loading at least one resource from a loopback address, and this change would affect them in one way or another: - https://www.chromestatus.com/metrics/feature/timeline/popularity/3689 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3691 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3693 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3695 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3697

Note that the above metrics almost certainly overestimate the affected page loads as some frame hierarchies currently confuse Blink: https://crbug.com/1136028. Previous estimates based on less precise metrics suggested that this was much less widespread.

In any case, we are waiting for more detailed metrics gathered in M87 to finalize our plan to ship the feature. We have an enterprise opt-out through enterprise policies. This is a bit unfortunate, as enterprises are exactly the sort of things that have large local networks full of interesting resources, but necessary.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/143) https://bugzilla.mozilla.org/show_bug.cgi?id=1481298

WebKit: No signal

Web developers: Negative (https://bugs.webkit.org/show_bug.cgi?id=171934) Developers running local servers generally want those servers to keep running without alteration. The conversation on a tangentally-related WebKit bug is representative.

My read of that thread is that folks would want to keep the possibility to access either localhost or local servers, but that the restriction of that access to secure contexts would not be a huge blocker for their use-cases. Am I understanding correctly?

[Titouan Rigoudy]

Hi Yoav,

Thanks for the reply, see my answers below.

On Mon, Nov 23, 2020 at 4:21 AM yo...@yoav.ws <yo...@yoav.ws> wrote:

On Friday, November 20, 2020 at 10:47:51 PM UTC+1 Titouan Rigoudy wrote:

Contact emails

tit...@chromium.org, cl...@chromium.org, mk...@chromium.org, va...@chromium.org

Explainer

https://github.com/WICG/cors-rfc1918/blob/master/explainer.md

Specification

https://wicg.github.io/cors-rfc1918/

API spec

Yes

Design docs

https://docs.google.com/document/d/1tIl8F0i31_z6gttaaPmRDynHBqWsLaV_94ngrpg5Oeg/edit#heading=h.lnbc74amxd9e
https://docs.google.com/document/d/15r4PQdHJa3SunkGg0RDgm6MlcCdWSZLJj9Cq2awSLvo/edit

Summary

Requires that private network requests for subresources may only be initiated from a secure context. "Private network requests" are those initiated from a public network, targeting a private network. Examples include internet to intranet requests and intranet loopbacks.

To clarify, the network requests to those private networks will be done over non-secure HTTP, and will not be considered mixed content?

We are not relaxing mixed content restrictions for this launch. There was an existing carveout for http://localhost, so public and private https websites will still be able to make requests to http://localhost without running into mixed content. However, public https websites are not able to make requests to private websites served over http due to mixed content. Fixing this requires serving the private website with https, and thus having proper certificates in place.

 

This is a first step towards fully implementing CORS-RFC1918: https://chromestatus.com/feature/5733828735795200

Blink component

Blink

TAG review

https://github.com/w3ctag/design-reviews/issues/572

TAG review status

Pending

Risks

Interoperability and Compatibility

< 1% of page views are currently loading at least one resource from a loopback address, and this change would affect them in one way or another: - https://www.chromestatus.com/metrics/feature/timeline/popularity/3689 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3691 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3693 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3695 - https://www.chromestatus.com/metrics/feature/timeline/popularity/3697

Note that the above metrics almost certainly overestimate the affected page loads as some frame hierarchies currently confuse Blink: https://crbug.com/1136028. Previous estimates based on less precise metrics suggested that this was much less widespread.

In any case, we are waiting for more detailed metrics gathered in M87 to finalize our plan to ship the feature. We have an enterprise opt-out through enterprise policies. This is a bit unfortunate, as enterprises are exactly the sort of things that have large local networks full of interesting resources, but necessary.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/143) https://bugzilla.mozilla.org/show_bug.cgi?id=1481298

WebKit: No signal

Web developers: Negative (https://bugs.webkit.org/show_bug.cgi?id=171934) Developers running local servers generally want those servers to keep running without alteration. The conversation on a tangentally-related WebKit bug is representative.

My read of that thread is that folks would want to keep the possibility to access either localhost or local servers, but that the restriction of that access to secure contexts would not be a huge blocker for their use-cases. Am I understanding correctly?

I believe so. I think this was written with a full CORS-RFC1918 launch in mind, not just the secure context restriction part. I should have double-checked this entry was still up-to-date, sorry!

[yo...@yoav.ws]

When do you expect to have more accurate metrics?

1% of page views is a lot...

Also - do you know what the distribution of that usage over sites is? 

Seems like the metrics are too recent to have landed in an HA run, which is a shame, as that would've made looking at what the responsible sites are easier.

[Titouan Rigoudy]

Hi Yoav,

Great questions! We expect to have more accurate metrics once we fix inheritance issues, for which we are targeting M89.

I think an explanation would help. The problem right now is that the numbers are dominated by requests made from the "unknown" address space, which account for ~1% of page visits [1]. Requests made from non-unknown address spaces account for ~0.08% of page visits [2]. We have decided not to block requests coming from the "unknown" address space as part of this launch. So far this means that only ~0.08% of page visits would be affected.

We suspect that the metric imbalance is due to a bug in the address space calculation logic. Iframes whose content is not loaded from the network (`about:blank`, `about:srcdoc`, `data:`, `blob:`, `filesystem:` and `javascript:`) are incorrectly classified as coming from the "unknown" address space instead of inheriting an address space from their parent/navigation initiator/what have you. Say a private website embeds an `about:blank` iframe and then `document.write()`s some HTML requesting an image from its own origin, we will classify that as a request from "unknown" to "private", when it should be "private" to "private" and thus unaffected by this launch.

This intuition is bolstered by the fact that the number of private network requests coming from non-unknown address spaces is an order of magnitude lower. It seems unlikely that 92% of private network requests on Windows are coming from iframes behaving as described in the scenario above, though in fairness it is possible.

All in all this means we suspect that once we fix address space inheritance, we will see that most of these requests were incorrectly classified as problematic. Some might also turn out to be bad ads or malicious injected scripts trying to poke at users' private networks, which we would like to block.

If that does not turn out to be the case, we will re-evaluate the decision to ship anyway.

As for the distribution: we have UKM integration which has highlighted a few external websites that seem to be relying on this feature. So far we have identified no more than a handful legitimate cases. We are waiting for 87 to roll out to 100% of stable to gather more complete data. In the meantime, we have engaged with DevRel to start reaching out to help affected websites work around the upcoming restriction.

Cheers,

Titouan

[1] https://uma.googleplex.com/p/chrome/timeline_v2/?sid=9316046e83ef01ba2d820fa3431af4aa

[2] https://uma.googleplex.com/p/chrome/timeline_v2/?sid=85970ee3fc0871a8716879e323f86131

[Daniel Bratell]

I think having the whole compatibility picture, including the current dark matter in the "unknown" address space, is required before judging the impact the web will see from this. Especially since something that is more or less a bug fix (change "unknown" to something more correct) could otherwise expand the usage beyond what is tolerable.

My hope is that your suspicions are correct, but I think we should see what the data says after your improvements to the classifications.

/Daniel

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPATO9fggPNQwczgSKoHX9C58PijAPoUQOR9ETJzY9PXJdT-kw%40mail.gmail.com.

[Titouan Rigoudy]

Hi Daniel,

That's fair. We are aiming to fix our metrics by M89. Once that's rolled out, we'll analyze the metrics some more and I'll get back to this thread with analysis results.

Cheers,

Titouan

[yo...@yoav.ws]

The scenario that concerns me here is the following:

http://vendor.example.com is currently used to control a certain home appliance, which it does over plain-text HTTP.

Once that restriction will be in effect, it would need to upgrade the site to https://vendor.example.com, which the vendor can easily do, but would also need to upgrade the appliance to have a local address and provide a cert for it, which presumably is hard and would require work from the local IT person (AKA the customer).

Is the above a reasonable scenario? Did I understand it correctly?

Do we know of appliances that fit this model? Do we know of ones that won't be able to upgrade?

[Titouan Rigoudy]

Hi Yoav,

That is a very reasonable scenario indeed, and one that we have considered already in the case of Plex [1]. In their case there is a reasonable workaround, since their local appliance also serves a copy of the webapp. In general there are a couple strategies a service can use to work around the upcoming restriction: flipping the embedding, navigating instead of fetching subresources, or using webtransport to sidestep the PKI.

We have started analyzing UKM data to identify other similarly-impacted services. So far we have only noticed a couple more seemingly-legitimate usecases. We are planning to reach out to all affected parties through DevRel. We also put out a web.dev article [2] requesting feedback from services that find themselves in a position like you described.

Cheers,

Titouan

[1] https://github.com/WICG/cors-rfc1918/issues/23

[2] https://web.dev/cors-rfc1918-feedback/

[Chris Harrelson]

What about router configuration pages? My ASUS home router, for example, is accessed by a non-secure-only URL at "router.asus.com" (which is of course intercepted locally by the router). Since the top level URL in that case is non-secure, this intent does not apply to it, right? Is it generally the case that route configuration pages are like mine, and won't be affected by this intent?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPATO9dVhgetAgvXPmpbCC%3DLQ7vQU_NvMiqY3rvNfHh7TSXP8w%40mail.gmail.com.

[Titouan Rigoudy]

Hi Chris,

What matters is not the domain name of the website but rather the IP address from which it was loaded. If the router serves its own configuration website (responds to the DNS request with its own private IP address), then the website should not make cross-origin / cross-network requests. Thus it should be unaffected by this intent.

In my experience, router configuration pages are indeed like yours: the router serves its own admin interface.

Cheers,

Titouan

[Chris Harrelson]

On Thu, Dec 10, 2020 at 1:29 AM 'Titouan Rigoudy' via blink-dev <blin...@chromium.org> wrote:

Hi Chris,

What matters is not the domain name of the website but rather the IP address from which it was loaded. If the router serves its own configuration website (responds to the DNS request with its own private IP address), then the website should not make cross-origin / cross-network requests. Thus it should be unaffected by this intent.

Makes sense, thanks for the clarification.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPATO9f48xehxkD3B4KyGo8QZ1UKjsGSVDCftj48dtOSefe3Sg%40mail.gmail.com.

[Chris Harrelson]

OK thanks. The API owners reviewed this intent today, and while we saw no problems other than compat data, will wait for the M89 data you mentioned before making a decision.

[Titouan Rigoudy]

Hi Chris,

Thanks for the update. I'll come back with better data in hand sometime in early March, then :)

Cheers,

Titouan

[Lutz Vahl]

Hi Chris,

Thanks for the info! While we're waiting for the metrics to roll out we crafted a plan how to ship the change as soon as we got the approval. Part of the plan is to raise a deprecation issue in DevTools letting the developers know that requests will get blocked soon. As this will help us bring the use counters down we're thinking about adding such a warning already in M90 (Stable release 2021-04-13). 

Would this be ok even if the I2S is not approved by then as the API owners didn't see any other problems than the compat data?

Please let me know if you've any concerns.

Cheers,

 Lutz

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPATO9fFaUWuatrCCROsrR1JJ1tPbvqAiL7hFyFGY75z4daSnA%40mail.gmail.com.

[Chris Harrelson]

Hi,

Adding a deprecation message that the feature will be removed in M91 might be fine... Are you pretty sure that the data will show an actually-impacted fraction of page loads much less than 1%? What does the data on canary and dev indicate?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAH0ixBPxqGd4KSizo6r3hNSpjKMYQe08q3Et8WsJDhYpNfNKew%40mail.gmail.com.

[Lutz Vahl]

Hi Chris,

The new metrics are currently only available on canary and we know from earlier analysis that canary data was not reflecting the same distribution as dev/stable. Therefore we're not sure yet! We'll monitor the metrics as soon as they hit additional channels. But we're thinking about adding the warning in M90 (CL adding it needs to land before 2021-02-25) to let the affected user know about the upcoming change. This will help us to spread the information about the upcoming change and bring the use counter down. Is this fine with you?

The plan is to enable the feature in M92 controlled via finch. Doing so we can monitor the metrics during the rollout and react in case of issues.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw86TgADuT7H%3D3qM3LnkELreJFbg9MmYKoK23TZDhT1vKw%40mail.gmail.com.

[Chris Harrelson]

Hi,

On Tue, Jan 12, 2021 at 9:51 AM Lutz Vahl <va...@chromium.org> wrote:

Hi Chris,

The new metrics are currently only available on canary and we know from earlier analysis that canary data was not reflecting the same distribution as dev/stable.

Right, I get that part. But does the canary data at least look positive?

 

Therefore we're not sure yet! We'll monitor the metrics as soon as they hit additional channels. But we're thinking about adding the warning in M90 (CL adding it needs to land before 2021-02-25) to let the affected user know about the upcoming change. This will help us to spread the information about the upcoming change and bring the use counter down. Is this fine with you?

The problem is that we don't approve deprecations without removal timelines, so we need at least some indication that we're likely to hit the timeline. That's why I was asking about the Canary channel. If the data from Canary looks promising, then your plan sounds good to me.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAH0ixBOawYEzd3b2BRGrn4iAXhqq4mj5OPWUV%3DF-dsk-Ayemrw%40mail.gmail.com.

[Chris Harrelson]

The API owners met today, and we're ok with adding the deprecation message targeting M92 removal. Once the UseCounters come in please come back to this thread for final approval. If the UseCounter is too high, we'll need to adjust or remove the deprecation message.

[Titouan Rigoudy]

Hi Chris,

> Right, I get that part. But does the canary data at least look positive?

Not so far, but we are hopeful that the story will improve by next week. We carried out our plan to fix the known issue with address space calculation and inheritance, but that did not yield the expected improvements in canary data when it started trickling in this week. Turns out, there were more than one bug at play.

Our metrics were:

 1) incorrectly classifying provisional frames' first navigations as originating from the unknown address space

 2) lumping together subresource fetches and navigation fetches

Based on our tests, we are more confident in the correctness of the metrics for subresource fetches only, and would like to start blocking those first. Yesterday I landed a patch in M89 to split up the metrics between subresources and navigations, so that we can evaluate compat risk independently.

> The API owners met today, and we're ok with adding the deprecation message targeting M92 removal. Once the UseCounters come in please come back to this thread for final approval. If the UseCounter is too high, we'll need to adjust or remove the deprecation message.

That's great! I will come back with canary data about subresources in particular when the data is available. I think it would make sense to start displaying a deprecation warning for those only, and start blocking those first. Navigations (including child frame navigations) can be evaluated separately, once metric issues are sorted out?

Cheers,

Titouan

[Titouan Rigoudy]

Hi friendly API Owners,

I come bearing good news. Canary metric data for subresource fetches looks much better in M89: https://uma.googleplex.com/p/chrome/timeline_v2/?sid=3640110376dac998bbe14d7b9ff5898a

We are down to ~0.1% of page visits that would be affected by this launch, were they to not migrate away from this behavior by M92. Uses of the unknown address space are down to ~0.001% of page visits - these can be excluded from the launch anyway.

I would also like to double check that scoping down the launch to only subresource requests sounds reasonable? Navigations can also be used for CSRF attacks, but the metrics code needs a little more work before we can rely on it for launch decisions.

Cheers,

Titouan

[Daniel Bratell]

Really good to have gotten rid of the dark matter in the metrics! The range initially was about 0.08 - 1.0%, and now it's about 0.1% (the Canary data you link is Google internal so I have not checked anything myself).

The number 0.1% is still high if it means a service/device/page would be broken, but you don't expect that to be the case come final shipping in the summer? Sorry if I'm asking something that has already been answered, but I can't remember an answer. You were going to reach out to possibly affected services/device makers?

And it's perfect that is is scoped to subresources since that is the part we have been evaluating.

/Daniel

[Titouan Rigoudy]

Hi Daniel,

There are a couple reasons I hope that launching this will be OK anyway:

1. Some of the uses we record are malicious: breaking these pages is our stated goal.
2. We are gathering data to identify the biggest users of this feature, and will reach out to them directly through DevRel.
3. We have communicated about this publicly through web.dev, and will show a deprecation warning in DevTools starting in M90.

#2 and #3 should help us bring the use counters down, but #1 means we will never fall below a certain threshold.

Cheers,

Titouan

[Yoav Weiss]

Any news on the data front?