HTTP caching behavior in secure contexts

[mi...@bocoup.com]

Hello, all. I'm working to port Chromium's service worker tests from
LayoutTests to the Web Platform Tests project [1]. I've come across one such
test that has resisted my best efforts, and I thought someone here might be
able to help me understand what is going wrong.

The test in question is defined across two files, see [2] and [3]. My work-in
progress patch is at [4].

At a high-level, the test takes the following steps:

1. The script attempts to register a new service worker, which triggers a
request for the associated JavaScript file
2. The server responds with the file, specifying `Cache-Control: maxage=3600`
3. The script attempts to register the service worker again, and the test
asserts that no request is made (which is observable in this case by
checking if the script was re-evaluated)

The assertion in step 3 passes when run under Chromium's infrastructure.
Notably, that involves accessing the test from http://127.0.0.1:8000 .

That assertion fails when run under WPT's infrastrcture because another request
is in fact issued, despite the `Cache-Control` header in the previous response.
In contrast to the Chromium infrastructure, the `wptserve` application makes
the test available on the domain https://web-platform.test:8443` (this is
facilitated by a hostfile mapping to 127.0.0.1).

I'm wondering if Chromium has some internal heuristic that is influenced by the
protocol, port, and/or domain. If so, is that defined by some standard? Or is
it a specification-allowed optimization?

Thanks!

Mike

[1] Tracking bug here: https://crbug.com/688116
[2] https://chromium.googlesource.com/chromium/src.git/+/0f5b86386accbd17d06a26475df674a2e1df8bf2/third_party/WebKit/LayoutTests/http/tests/serviceworker/update-served-from-cache.html
[3] https://chromium.googlesource.com/chromium/src.git/+/0f5b86386accbd17d06a26475df674a2e1df8bf2/third_party/WebKit/LayoutTests/http/tests/serviceworker/resources/update-served-from-cache-worker.php
[4] https://codereview.chromium.org/2904063003/

[Matt Menke]

Does WPT use self-signed certs, along with ignore-certificate errors?  This may be due to that.  If WPT uses HTTP instead of HTTPS, what happens?

--
You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+unsubscribe@chromium.org.
To post to this group, send email to net...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/2d93c296-c433-4f4f-92f7-ef7d1f40fdfd%40chromium.org.

[Marijn Kruisselbrink]

That sounds plausible yes. Service workers requiring secure context means they can't run on http, but for example another wpt test specifically for http cache behavior has the same behavior. It passes on http but fails on https with nothing getting cached.

[Mike Pennisi]

> Does WPT use self-signed certs, along with ignore-certificate errors?  This
> may be due to that.

Yes, WPT uses self-signed certs, and I am running Chromium with that flag.

> If WPT uses HTTP instead of HTTPS, what happens?

The test passes in that case. Service workers are only available in secure
contexts, which may be provided over HTTP via `localhost` or `127.0.0.1`.
Unfortunately, WPT cannot support tests that require any specific domain name
because it is deployed to the web.

It sounds as though this is expected behavior, which makes me believe it is
allowed by the relevant specifications as a host-defined optimization. Can you
confirm that? If so, I will proceed by labeling this test as
"Chromium-specific" and aborting my attempts to contribute it to WPT.

[Matt Menke]

Caching is always best-effort, and never required by spec.  We don't cache when responses use self-signed certs because we don't trust the responses, and don't want to poison the cache with non-authoritative responses (Dislaimer:  I work on neither the cache nor SSL/certs, I just like to pretend I know things).

[Mike Pennisi]

Understood. Thanks, Matt!

[Marijn Kruisselbrink]

Makes sense. It's unfortunate though that that means we can't have cross-browser tests like this service worker one. It essentially means we won't be able to test things like the upcoming useCache enum/flag, and similar features. We should probably at least file a web-platform-tests bug to track this issue (of spec features that are currently impossible to test).

[Matt Menke]

May be worth figuring out how browser tests resolve this issue.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CA%2BOSsVY3wWXALp3FbriOoiHdMTkyoHNy_V-5teObHy1QS9Lr7w%40mail.gmail.com.

[Mike Pennisi]

> Makes sense. It's unfortunate though that that means we can't have
> cross-browser tests like this service worker one. It essentially means we
> won't be able to test things like the upcoming useCache enum/flag, and
> similar features. We should probably at least file a web-platform-tests bug
> to track this issue (of spec features that are currently impossible to test).

I think this is a limitation of standards-based testing generally, not of the
specific WPT architecture. Even if WPT supported the conditions that would
satisfy this test in Chromium, WPT couldn't accept the test because, as Matt
said, "Caching is always best-effort, and never required by spec." As long as
that's the case, it seems that any tests regarding caching behavior will
necessarily be vendor-specific. I think that if we wanted to assert consistent
behavior from a feature like `useCache`/`updateViaCache`, we'd have to start by
standardizing its effect all the way through HTTP..

[Ryan Sleevi]

Small clarification - when using --ignore-certificate-errors, we don't cache. We also don't cache information from 'bad' certificates that the user clicked through.

For purposes of testing, while reducing some of the terrible footgun approach, we recently introduced --ignore-certificate-errors-spki-list (which requires an explicit --user-data-dir), which provides a more realistic testing flag. In this case, if a set of explicitly denoted hashes are sent by the server (regardless of whether or not they signature verify, since one of the errors you may be trying to ignore is 'invalid signature'), then it will be treated 'as if' it was good.

Because of the significant security risk of this flag (bypassing cert verification entirely), the --user-data-dir should be explicitly specified and all the normal caveats apply that this is 'dangerous'. It is, however, slightly less dangerous than installing a local trust anchor, which is the other way you can bypass this error (and which, IIRC, WPT also supports)

So there are options for testing this, in differing degrees of hermeticism and security, and with the hopefully obvious statement that always use an ephemeral profile directory specified by --user-data-dir because any traffic it sends won't necessarily be encrypted to the desired level.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CAEK7mvp1yM_j5DvLvHJBqnDftxRiKkf0drKOdMvnd9ruZhGeOg%40mail.gmail.com.

[Mike Pennisi]

Thank you for walking me through all that, Ryan! As per my previous message,
though, I still think we have to omit this test from WPT. If my interpretation
is correct, it comes down to asserting a "SHOULD." This is perfectly valid
expectation for Chromium to hold for itself but not appropriate for a test
suite derived strictly from standards.

I may be wrong though. I've CC'd Anne van Kesteren since he has a pretty strong
vision for WPT.

[Domenic Denicola]

Just saw this thread. Anne is on vacation from the next month or so, but my opinion is that WPT is for tests that are useful to all browsers, not only for tests that must pass in all browsers.

My guess is that caching tests would be useful to all browsers, especially with a note in their head (e.g. a HTML comment) stating that they're testing a should and don't necessarily need to pass.

One thing that might make them un-useful to browsers is if the way that these "should"s were implemented caused test flakiness. (E.g., if a bunch of other tests run on the same domain, the cache might get evicted mid-test or something.) In that case, yeah, they wouldn't be a good candidate for WPT. I'm not sure whether that's the case or not.

> it, send an email to net-dev+u...@chromium.org.

>
> To post to this group,
> send email to net...@chromium.org.
>
> To view this
> discussion on the web
> visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/2d93c296-c433-4f4f-92f7-ef7d1f40fdfd%40chromium.org.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
> You received this message because you are subscribed
> to the Google Groups "net-dev" group.
>
> To unsubscribe from this group and stop receiving

> emails from it, send an email to net-dev+u...@chromium.org.

[Anne van Kesteren]

On Mon, Jun 12, 2017 at 6:47 PM, Domenic Denicola <d...@domenic.me> wrote:
> Just saw this thread. Anne is on vacation from the next month or so, but my opinion is that WPT is for tests that are useful to all browsers, not only for tests that must pass in all browsers.
>
> My guess is that caching tests would be useful to all browsers, especially with a note in their head (e.g. a HTML comment) stating that they're testing a should and don't necessarily need to pass.
>
> One thing that might make them un-useful to browsers is if the way that these "should"s were implemented caused test flakiness. (E.g., if a bunch of other tests run on the same domain, the cache might get evicted mid-test or something.) In that case, yeah, they wouldn't be a good candidate for WPT. I'm not sure whether that's the case or not.

Note that mnot wrote a whole bunch of cache tests already and put them
under fetch/http-cache/. Definitely supportive of testing that.


--
https://annevankesteren.nl/