Intent to Ship: CSP: `report-to` directive.

[Andy Paicu]

Contact emails

mk...@chromium.org,andy...@chromium.org

Spec

https://w3c.github.io/webappsec-csp/#directive-report-to

Summary
The `report-to` directive wires CSP violation reports up to the Reporting API (https://wicg.github.io/reporting/), and deprecates the existing `report-uri` directive.

`Content-Security-Policy: script-src 'self'; report-to default` will queue violation reports into the reporting group named "default", and the browser will deliver them in a batch, along with other reports from other APIs.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/d/msg/blink-dev/Xro1jnasnY0/rUbiEtbDBQAJ

Interoperability risk

Firefox: No public signals

Edge: No public signals

Safari: No public signals

Web developers: No signals

Folks generally seem enthusiastic about the reporting API, but I don't think any vendors in WebAppSec WG have taken a close look at `report-to` yes. Hopefully this intent will give them reason to do so. :)

Testing, however, is going to be difficult. Since part of the benefit of this feature is that reports are delivered in browser-controlled batches, out of band, it's going to be difficult to write web platform tests that verify much of anything. `setTimeout(verifyReportReceived, 10000000)` doesn't sound exciting. This isn't unique to CSP, however; any feature that uses this mechanism is going to have similar troubles. We'll have to put our heads together to see what we can do to ensure interop.

Compatibility risk

The `report-to` directive is specified as overriding the `report-uri` directive, so developers can support both older and newer browsers by using both in a policy. This means that we should be able to both support a gradual migration from one to the other, and if we decide that this was a terrible idea and should throw the whole thing away, we'll be able to do so without breaking anyone.

Ongoing technical constraints

None

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=726634

Link to entry on Chrome Platform Status

ttps://www.chromestatus.com/features/5826576096690176

Requesting approval to ship?

Yes

--

Regards,

Andy Paicu

[Ian Clelland]

On Wed, Sep 12, 2018 at 9:01 AM 'Andy Paicu' via blink-dev <blin...@chromium.org> wrote:

Testing, however, is going to be difficult. Since part of the benefit of this feature is that reports are delivered in browser-controlled batches, out of band, it's going to be difficult to write web platform tests that verify much of anything. `setTimeout(verifyReportReceived, 10000000)` doesn't sound exciting. This isn't unique to CSP, however; any feature that uses this mechanism is going to have similar troubles. We'll have to put our heads together to see what we can do to ensure interop.

For feature policy reporting, we've so far elected to test through the ReportingObserver API, rather than the Reporting API. I'm not sure if CSP reports are observable in JS though, so that may not be an option. (And it doesn't let you test the endpoint-group selection specifically either)

Two other possibilities I can think of for this would be:

 - Run a reporting server as part of the WPT test runner, but have a way to verify all of the delivered reports, out-of-band, *after* a test run in order to mark each test as passed or failed.

 - Spec and write a WebDriver protocol extension to request the generated reports from the browser without waiting for them to be sent. +paulmeyer has been working on getting WebDriver extensions to work with WPT, so he may have a better idea about how much work that would be.

[Douglas Creager]

Testing, however, is going to be difficult. Since part of the benefit of this feature is that reports are delivered in browser-controlled batches, out of band, it's going to be difficult to write web platform tests that verify much of anything. `setTimeout(verifyReportReceived, 10000000)` doesn't sound exciting. This isn't unique to CSP, however; any feature that uses this mechanism is going to have similar troubles. We'll have to put our heads together to see what we can do to ensure interop. 

 

[snip] 

- Run a reporting server as part of the WPT test runner, but have a way to verify all of the delivered reports, out-of-band, *after* a test run in order to mark each test as passed or failed.

There is some WPT logic that we use to test Network Error Logging that you can repurpose for this.  It runs a collector that stores received reports into the wptserve stash, and provides a helper JavaScript function for verifying that a particular report was received.  There were a couple of wrinkles we had to deal with:

• For timeouts, we mention in the README that the browser needs to be configured to deliver reports "immediately" instead of waiting for a batch to queue up, as it would normally do.  Chrome's test harness drops the delivery interval down to 100 ms, which is good enough.
• Reporting policies are global, and not local to a browser context, so you'll need to ensure that any test cases that exercise the Reporting API and have conflicting policies are serialized.  To handle this for NEL, we're using a simple spin-lock.

[Andy Paicu]

Ah perhaps I should have been more careful with what I copy from the I2I.

We ended up using existing CSP reporting testing scripts (https://github.com/web-platform-tests/wpt/blob/master/content-security-policy/support/report.py) that use the wpt stash to store and then check the reports and that seems to work just fine for chrome since the test harness as mentioned above has a lowered delivery interval.

So I think the testing is not a concern anymore.

Regards,

Andy Paicu

--

Regards,

Andy Paicu

[Mike West]

LGTM1.

The reporting API is the right long-term mechanism for things like CSP violation reports, and `report-to` has been in the spec for quite some time now without concern.

I would like to see tests for the ReportingObserver integration, though. I didn't see those checks when skimming through the existing WPT. Can we make sure that's in place before enabling by default? Thanks! 

-mike

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACnmqYiasx3RkTxFRtoLuXFw967TSp-cvpACR034gRnZGqEv5A%40mail.gmail.com.

[Philip Jägenstedt]

LGTM2

Thanks for your work on the test, Andy. Do you think this view will go all-green for Chrome after shipping?

https://wpt.fyi/results/content-security-policy/reporting-api?label=experimental

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3Dc6XL0ZankH0%2Bs8JaW9J66kbUoD%2Bom9-yhdX3HbUHBipA%40mail.gmail.com.

[Rick Byers]

LGTM3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYd93jjOGuRqcgh789DEctt1U7Uzxk4xugGkfw%3DRDxtVVw%40mail.gmail.com.

[Andy Paicu]

The directive is relying on the report API which is currently behind a flag and on a 1% Finch experiment. The flag is currently disabled by default so all positive tests are failing. 

I have a virtual test suite https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/VirtualTestSuites?l=616 that runs them with the flag enabled so once the Reporting API is fully shipped and enabled by default the tests should pass.

Regards,

Andy Paicu

--

Regards,

Andy Paicu