WPTs with nontrivial server-side functionality: options?

[David Van Cleve]

My team's working on implementing a new web API [1] that lets clients execute certain cryptographic operations alongside outgoing requests. Handling these operations' outputs (e.g. to verify in tests that the client is behaving correctly) involves some nontrivial but not horribly complex server-side work: parse the request's binary format; execute some reasonably basic but low-level elliptic curve operations; serialize a response. We currently only have a C++ implementation for this server-side logic.

We're considering a couple options for integration with layout tests or WPTs:

1. ship a C++ binary in blink_web_tests_support_data, like imagediff currently is; in tests, use a simple Python wrapper that parses request headers, passes them to the binary, and puts the binary's response in a response header

2. reimplement the server-side functionality in Python (probably 500-1kloc, depending on whether we can use external dependencies)

3. completely mock out the client-side and server-side cryptographic functionality (not preferred unless unavoidable; seems fiddly and less realistic)

We have a couple questions:

- What are your thoughts about the feasibility of approach #1? Adding a new binary dependency in the vein of imagediff seems simple enough, but I'm a little concerned about the gymnastics of calling into the binary during the tests' execution.

- For #2, it would simplify the Python implementation to be able to use a couple dependencies outside the standard library. Would this be possible? (Does the answer vary between WPTs and Blink layout tests?)

We'd also naturally appreciate any general thoughts or advice. Thanks!

David, Charlie, Steven (Trust Tokens team)

[1]  Trust Tokens: https://github.com/wicg/trust-token-api

[Robert Ma]

First of all, do you expect, or hope other browser vendors will implement and test this API? (Hopefully the answer is yes.)

Assuming yes, then approach 1 has another twist: how do we share this C++ program with other browser vendors? Note that even though we could implement logic in WPT to download and use prebuilt binaries (and we are probably going to do that on wpt.fyi at least), other browser vendors would almost certainly want to be able to build this binary from source, ideally without having to checkout the whole Chromium repo or use gn/ninja.

Regarding approach 2, 500-1k LoC sounds reasonable. Do you have a list of dependencies you wish to use? On a high level, we have two preferences: 1. the dependencies should ideally be Py-2/3 compatible as we still need to support old platforms for a while (in fact, Py2 is still our primary version and we're working towards Py3 compatibility); 2. the dependencies are preferably pure Python (no native extensions). If the dependencies meet these two requirements, they'd be trivial to add; otherwise, we'll need further discussions on the tradeoff.

--
You received this message because you are subscribed to the Google Groups "ecosystem-infra" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ecosystem-inf...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/ecosystem-infra/CAMeJurfQnggynHMQCQHge2fduvR2zPq0GWZYsy14RBJ7WVr0aQ%40mail.gmail.com.

[David Van Cleve]

Thanks! Sounds like approach 2 is the way to go. We'll get back in touch once we have an idea of the Python dependencies.

Just to double-check, would it be possible to add Python dependencies for Blink layout tests, too? We currently have a couple of testRunner methods for clearing persistent state related to the feature that we're planning on eventually porting to WPT automation; this will become a little more urgent if it's only possible to implement the end-to-end tests as WPTs.

> First of all, do you expect, or hope other browser vendors will implement and test this API? (Hopefully the answer is yes.)

We do! Yes, providing a Python file definitely seems simpler. 

> Regarding approach 2, 500-1k LoC sounds reasonable. Do you have a list of dependencies you wish to use?

Nothing final yet, but it looks like `cbor` and `cryptography` (or similar) would probably suffice, and those, for example, are both py2-and-3 compatible. 

[Robert Ma]

cbor looks good, but cryptography is actually a rather complex package (with a bunch of dependencies) with native extensions... Again, pure Python dependencies would be great, but I understand crypto is hard without openssl.

In Chromium, we'd need something like this added to run_web_tests: https://source.chromium.org/chromium/chromium/tools/depot_tools/+/master:gsutil.vpython;l=34;drc=8b52ca7aa7965de75410cd784e5dae4cceef59e7

And the upstream WPT also needs some non-trivial setup. I'm currently working on the QUIC server that also involves native extensions, and I'll have a better idea of the scope soon (maybe in a month).

[David Van Cleve]

Thanks. I didn't realize that about `cryptography`. The functionality we need to implement doesn't actually have any openssl dependencies per se; we'd likely be able to do without python packages with openssl dependencies if they require additional effort to support.

[David Van Cleve]

Hi Robert, I just glanced back at this thread and saw your last message again. How hard did it end up being to add Python code with native dependencies for the QUIC WPT support?

[Robert Ma]

Hi David, we did reach the consensus to add native dependencies after some discussions. The main challenge is that every time you want to upgrade these Python packages with native components, you'd need to coordinate with infra folks of various browser vendors (at least Chrome Operations to upload the new versions to cipd, and potentially Firefox/WebKit infra as well if they also implement this feature and need to run the tests), so the WPT community generally want to minimize such dependencies. 

[James Woo]

Hi all, it has been a while since the last update on this thread. Steven and I are trying to gauge the complexity of Option #2 in the original post. Does WPT infrastructure currently have any Python dependencies that support cryptographic operations? Do they use native extensions? For example, if `cryptography` is available for WPTs, that could greatly simplify the aforementioned Python server implementation.