BrowserTests for Origin Trials using an EmbeddedTestServer

[Sam LeDoux]

Hi all,

I am working on setting up an Origin Trial for StorageAccessHeaders, and we are facing some challenges with the current behavior for browser tests with OT tokens. 

We are trying to set up a persistent origin trial for the feature, and in order to properly test it, we need to be able to check the content of a header on follow-up requests to the same domain after opting the origin into the trial. The current practice for testing OT tokens in Chrome seems to rely on using a URLLoaderInterceptor to intercept requests to the enabled domain and creating a request header with the token's value (example). Since our feature involves code in URLRequestHTTPJob, we cannot properly test the behavior of StorageAccessHeaders when the load is intercepted since the StorageAccessHeaders code will not be called in that case. 

Is there a recommended solution to this situation? We didn't see any similar cases in the documentation or the codebase. 

The current solution we've been testing locally has been to use an EmbeddedTestServer with a preselected port value that we also use when generating the token in the command line by passing something like `https://example.test:<port>` to generate_token.py. This works on our local machine, but we expect it would be unreliable and prone to flakeyness since the preselected port may not always be available.

A possibly stronger solution we thought of would be the capability to dynamically generate an OT token from within a browser test - that way we could create a token with a port value that the EmbeddedTestServer has already determined to be available at the time of test. If there is a way to do this already we would love to learn about it as well as other possible solutions that may exist. 

Best,

Sam

[Jason Chase]

Hi Sam,

Apologies for the delay in responding.

This is a known issue, tracked in https://issues.chromium.org/40860522, for making it easier for tests to use OT tokens. In previous occurrences, teams have been able to use the URLLoaderInterceptor as a workaround, so finding a solution hasn't been prioritized.

My preferred solution would be to support variable port numbers in the validation of OT tokens (https://issues.chromium.org/40116616). As noted in that issue, it's not only Chrome automated tests that have the problem of variable port numbers, so this would benefit more than just Chrome developers. I actually have this implemented in a local CL. This stalled in discussion of whether it was a good idea to relax the validation. I could (and should) restart that discussion to reach some conclusion.

Dynamically generated tokens have also been considered. The challenges with that approach are mostly technical/code health, but I assume manageable. The approach of dynamically generating tokens should also address the long-term problem of having OT tokens embedded in source, which will eventually expire.

When do you need a solution for this? That may be a deciding factor in the recommended path.

Thanks,

Jason

--
You received this message because you are subscribed to the Google Groups "experimentation-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to experimentation...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/experimentation-dev/2392849b-5213-4cd4-be14-f561af515f35n%40chromium.org.

[Sam LeDoux]

Hi all,

Updating this thread as we are experiencing some friction due to this issue. Some browser tests of ours were recently disabled because they relied on flakey workarounds that were necessary due to the lack of support for testing with OT tokens on test servers with dynamic port numbers. We would really appreciate it if a solution for this issue could take a high priority so that we can write reliable browser tests for the origin trial we are working on. 

Best,

Sam

On Mon, Aug 19, 2024 at 9:32 AM Sam LeDoux <sle...@google.com> wrote:

Jason,

Thank you for the response, and for pointing to those issue threads.

All of the solutions you linked to would work for us, so we'd be happy if any of those land. 

In terms of a timeline, we're trying to land our code as soon as possible because Storage Access Headers are a top priority for our team, even if our current testing solution is flaky. Although we're not necessarily fully blocked on this, we'd really appreciate having a quick working solution to the port issues with testing OT tokens as it would help decrease flakiness in the code we land. 

Thank you again for looking into this.

Best, 

Sam