Trust Token API / fallback nonstandard origin trial plan

265 views
Skip to first unread message

David Van Cleve

unread,
May 19, 2020, 2:12:31 PM5/19/20
to blink-api-owners-discuss, Steven Valdez, Charles Harrison
We're working on origin trialling the Trust Token API and would like your approval for a nonstandard origin trial plan.

Background
- The API provides the ability to specify cryptographic operations ("issuance," "redemption," and "signing") to execute alongside outgoing requests. The three operations are specified by varying values of the same parameters object; fetch, XHR, and the iframe element's interfaces expand to accept instances of this object.
- We expect signing and redemption will be executed by third-party scripts in a wide variety of first-party contexts, and it would be infeasible to have each of these first-party sites enroll in the trial. 
- The origin trials team is working on launching third party origin trials, which allow a script served from an origin enrolled in an origin trial to provide the origin trial's token, but it's not certain that this will launch by M84.

Our plan
Our preferred plan is to use a third party origin trial, which will allow us to gate all three operations behind the origin trial: the third-party scripts executing redemption and signing will be able to provide origin trial tokens in order to execute the operations. 

However, assuming third party origin trials don't launch in M84, we'd like to be able to fall back to using a standard origin trial to gate usage of the feature.

The API's signing operation uses the output of redemption, which in turn uses the output of issuance, so it's possible to restrict usage of signing and redemption implicitly by gating only issuance behind the origin trial. Since issuance will be executed by first-party script in a relatively small number of contexts, we can require an origin trial token in order to execute issuance. If we see (aggregate) usage exceeding the origin trial limit, we'll use an associated base::Feature as a kill switch, globally removing the API surface.

This plan is nonstandard because it would mean that the API surface for signing and redemption is exposed whenever the base::Feature is enabled, both in contexts with origin trial tokens and without, with usage only implicitly limited. 

Our request
We'd like your approval since this trial plan changes the stable API surface whenever the base::Feature is enabled (which will be for the duration of the origin trial, but will likely be for a small minority of users).

We've cleared this with the origin trials team as a fallback and feel that, while it isn't structurally ideal, monitoring UseCounters and having a base::Feature kill switch will suffice to prevent burn-in. 


David Van Cleve, Steven Valdez, Charlie Harrison

dav...@chromium.org

unread,
May 20, 2020, 5:32:33 PM5/20/20
to blink-api-owners-discuss, sva...@chromium.org, cshar...@chromium.org, dav...@chromium.org
Friendly ping. I wasn't sure if this was the right destination for this kind of exception request; please let me know if elsewhere (blink-dev?) would be better.

Rick Byers

unread,
May 21, 2020, 4:00:14 PM5/21/20
to dav...@chromium.org, blink-api-owners-discuss, sva...@chromium.org, Charles, Jason Chase
Sorry for the delay David. I agree 3p OT is the right solution here, but also that what you propose is a viable fallback that will be nearly equivalent if we believe there's little chance of someone taking a dependency on any of the API without access to the issuance API. Trust Tokens also doesn't seem like the sort of thing we're going to see a huge number of developers interested in trying to use as soon as possible, so that further reduces the risk of burn-in in my mind.

I'd say that unless any other API owner objects here in the next 24 hours, you should consider this modified OT plan acceptable as a stop-gap until the 3P OT infrastructure is available.

Thanks for reaching out.

Rick



--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/790ba961-5df6-4574-af3c-4f1b41cfbc8b%40chromium.org.

Rick Byers

unread,
May 21, 2020, 4:07:17 PM5/21/20
to dav...@chromium.org, blink-api-owners-discuss, sva...@chromium.org, Charles, Jason Chase
I meant to mention that I see the Signed Exchange OT as having a similar precedent here, which we debated at some length. There we required only the origin serving the exchange to opt-in to the OT, not the origin of the signed document itself. Technically this meant one origin could be exposed to the effects of a trial enabled by another origin. 3P OTs are going to give us that property officially, and so I don't think your proposal is too fundamentally different. Of course you need to think through whether sites might change behavior in some way based purely on the existence of the API surface, but I assume the risk in the short-run would be low.

Rick

David Van Cleve

unread,
May 21, 2020, 4:21:18 PM5/21/20
to Rick Byers, blink-api-owners-discuss, Steven Valdez, Charles, Jason Chase
Thanks, Rick! 

dav...@chromium.org

unread,
Sep 2, 2020, 7:47:27 PM9/2/20
to blink-api-owners-discuss, rby...@chromium.org, sva...@chromium.org, cshar...@chromium.org, cha...@chromium.org, dav...@chromium.org
We'd like to request an increase from the standard “0.5% of page loads” origin trial usage target, to 5%. We feel this won’t lead to burn-in because the API is intended to be a less useful (but privacy-better) long-term partial substitute for an existing web feature (third-party cookies): the raison d’etre of the API in the current world is prototyping and experimentation.

The main goal of this origin trial is to understand whether trust tokens are useful for fraud detection. A key challenge is distinguishing the effect of sparsity---if we enable Trust Tokens for 1% of clients, models trained on observed tokens will see a roughly 1% sample---from the effect  of the inherent coarseness of tokens’ encoded information. 

Our experimentation partners have evaluated existing models analogous to potential trust token-based models on sparse subsamples of their training data; they’ve found that roughly a 10% sample would suffice to retain most of the models’ quality. This translates to estimated UseCounter values of under 5%.^ 

The primary argument for keeping origin trial usage low is the ability to avoid large developers depending on the feature. We don’t expect developers to develop a production reliance on tokens’ embedded information during this origin trial, because trust tokens are harder to use and contain far less information than third-party cookies (the benefit being that they’re much better for privacy). Further, since Trust Tokens is independently gated by a base::Feature, origin trial participants will still see the API unavailable for a substantial majority of their users.



^ We currently have the feature enabled for 3% of clients; this number is calibrated conservatively to lead to feature usage below the standard 0.5% UseCounter target. Extrapolating, increasing the rollout to 10% of clients would lead to UseCounter values below a couple percentage points; 5% is an upper bound.
Thanks, Rick! 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

Yoav Weiss

unread,
Sep 3, 2020, 3:29:42 AM9/3/20
to dav...@chromium.org, blink-api-owners-discuss, Rick Byers, sva...@chromium.org, cshar...@chromium.org, Jason Chase
That sounds reasonable to me. LGTM1

Thanks, Rick! 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/2796bc9d-05bc-4a7a-8c5f-6fa7b9ec84b6o%40chromium.org.

Alex Russell

unread,
Sep 3, 2020, 3:40:17 PM9/3/20
to Yoav Weiss, dav...@chromium.org, blink-api-owners-discuss, Rick Byers, sva...@chromium.org, Charles Harrison, Jason Chase, blink-dev
Also sounds reasonable to me, but would like to make sure we follow-up at a reasonable cadence to watch usage and make sure we aren't breaching limits. A report back to a blink-dev thread once a month or so sounds reasonable. LGTM2 with that caveat.

Also want memorialise the decision on blink-dev@ for visibility, so cc-ing here. Specifically, we discussed in today's API OWNERS meeting and are setting a new precedent for a one-off policy exception. We agreed that this should require 3 LGTMs to move forward.

Regards 


Mike West

unread,
Sep 3, 2020, 3:41:20 PM9/3/20
to Alex Russell, Yoav Weiss, David Van Cleve, blink-api-owners-discuss, Rick Byers, sva...@chromium.org, Charles Harrison, Jason Chase, blink-dev
I agree with Alex's caveats. LGTM3 on that basis.

-mike


David Van Cleve

unread,
Mar 18, 2021, 4:02:18 PM3/18/21
to blink-api-owners-discuss, Mike West, Yoav Weiss, David Van Cleve, blink-api-owners-discuss, rby...@chromium.org, sva...@chromium.org, Charles Harrison, Jason Chase, blink-dev, Alex Russell

API owners,

We request your approval to move the Trust Token API origin trial to a rollout limiting burn-in with a 5% holdback, superseding the origin trial's current fixed usage target.

When initially estimating what subsample would suffice to provide generalizable experiment results, our experimentation partners used a particular training procedure to train models on subsampled data and found that a 10% subsample led to reasonable quality. However, this process involved some assumptions in its choice of training procedure used: in the event, these did not correspond well to the approach used to evaluate live data from the Trust Token API origin trial rollout.

In the time since our prior request for a higher, but still strictly limited, alternative origin trial usage target, third-party origin trial procedure design work has proposed an alternate rollout model aiming to avoid burn-in by using a small base::Feature holdback, accompanied by periodic cliffs involving either backwards-incompatible changes or disabling the feature. Rather than requesting a specific further limited increase to the origin trial usage target for this trial, we’d like to switch to the alternate model of limiting burn-in with a persistent 5% holdback and periodic backwards-incompatible changes. We’d additionally like to emphasize again that we believe the Trust Token API has a particularly low risk of burn-in, because trust tokens are intended to be a less useful, but privacy-better, alternative to third party cookies, a currently available alternative.

Mike West

unread,
Mar 22, 2021, 3:50:26 AM3/22/21
to David Van Cleve, blink-api-owners-discuss, Yoav Weiss, David Van Cleve, rby...@chromium.org, sva...@chromium.org, Charles Harrison, Jason Chase, blink-dev, Alex Russell
Hey David, thanks for coming back with this question. I'll start by noting that this request to shift away from a fixed usage target, and towards a holdback model will (just like the original request to exceed the typical limit) require 3 LGTMs. I'm splitting it into a distinct thread to make the new discussion a little easier to follow.

Trust Tokens are somewhat unique. They're a central part of the abuse-mitigation-at-scale story for the cookieless world we're working towards, and it's quite important that we get them right. Experimentation with a number of different partners and diverse architectures seems more important for this API than for most. At the same time, Trust Tokens are strictly less-capable* than the status quo cookies they aim to replace. If they're not successful at addressing the use case for which they're designed, then it seems quite likely that folks will simply continue using cookies.

With this in mind, I think it's reasonable to be more flexible here than usual. Shifting the Origin Trial away from a page-view cap towards a cap on the percentage of Chrome's users is something I can be comfortable with. Burn-in will likely be limited by a holdback percentage on the one hand, and periodic (hopefully well-documented and promulgated?) shifts in the API surface on both the client and server side on the other. Still, a jump to 95% of the user base is substantial, and similar-enough to shipping that I'd like to see a little more public justification. What does it enable that, say, 50% wouldn't?

I'd also like to understand the state of the user-facing messaging around this feature. Have y'all wired Trust Tokens up to chrome://settings/siteData? Do we give users control over their usage (via content settings or similar)? Is the developer-facing story still NetLog-based, or has work on DevTools integration made progress? As we move towards broader and broader applicability, these questions seem more important than they do for tightly-scoped trials.

-mike

* Trust Tokens themselves are less-capable than cookies, but the platform-provided tokens wrapped up in the existing origin trial are a signal that cookies can't directly replicate, and I'm more concerned about this part of the trial than the rest. What are y'all's thoughts on burn-in there?

David Van Cleve

unread,
Mar 22, 2021, 7:17:39 PM3/22/21
to Mike West, blink-api-owners-discuss, Charles Harrison, blink-dev, Simon Zünd
Thanks for the thorough review and for the high-level favorable consideration towards the idea of moving to a holdback. 

Sizing: Rather than working with integrators to produce a specific estimate for "sufficiently large" sizing greater than the current 10% base::Feature rollout (targeting ≤5% usage), we felt it would be simpler to move to a different rollout model. Switching to a rollout where the feature is near-universally enabled (modulo the holdback and environment-specific feature support limitations, like older browser versions and, for now, WebView) hopefully simplifies analysis by substantially mooting questions about adequate sizing.


Controls: The "Privacy and security > Clear browsing data" control clears included sites' associated Trust Tokens state (code pointer; lengthier description in the design doc's Clearing data and browser settings section).


Developer support: szuend@ (cced) has been busy landing very neat DevTools support; you can find instructions and some screenshots in our web.dev article. I think the DevTools support is probably mature enough to be a sufficient aid for debugging new JS code interacting with an existing token issuance service, or for (say) a site to debug integrating JS library code that uses the API. The NetLog integration still comes in handy when developing a new issuance service, since it provides visibility into the browser's guts to help diagnose specific server-side implementation issues.


Mike West

unread,
Mar 24, 2021, 6:13:42 AM3/24/21
to David Van Cleve, blink-api-owners-discuss, Charles Harrison, blink-dev, Simon Zünd
On Tue, Mar 23, 2021 at 12:17 AM David Van Cleve <dav...@chromium.org> wrote:
Sizing: Rather than working with integrators to produce a specific estimate for "sufficiently large" sizing greater than the current 10% base::Feature rollout (targeting ≤5% usage), we felt it would be simpler to move to a different rollout model. Switching to a rollout where the feature is near-universally enabled (modulo the holdback and environment-specific feature support limitations, like older browser versions and, for now, WebView) hopefully simplifies analysis by substantially mooting questions about adequate sizing.

I agree that the proposed model is simpler to reason about. I'd still like to understand whether that simplicity justifies the substantial jump from <5% usage to ~95% potential availability. Let's say we had philosophical reasons to suggest a 50% holdback rather than the 5% you're asking for; what would the practical impact be for your experimentation?

Controls: The "Privacy and security > Clear browsing data" control clears included sites' associated Trust Tokens state (code pointer; lengthier description in the design doc's Clearing data and browser settings section).


Great, thank you. I don't think that doc addresses making stored tokens visible on surfaces like chrome://settings/cookies/detail?site=airhorner.com, which would allow users who care to get a little more insight into who's making assertions about them (to whom). Do y'all have plans for such UI?

Developer support: szuend@ (cced) has been busy landing very neat DevTools support; you can find instructions and some screenshots in our web.dev article. I think the DevTools support is probably mature enough to be a sufficient aid for debugging new JS code interacting with an existing token issuance service, or for (say) a site to debug integrating JS library code that uses the API. The NetLog integration still comes in handy when developing a new issuance service, since it provides visibility into the browser's guts to help diagnose specific server-side implementation issues.

Great, thank you for the detail!

-mike

David Van Cleve

unread,
Mar 30, 2021, 3:02:06 PM3/30/21
to blink-dev, Mike West, blink-api-owners-discuss, Charles Harrison, blink-dev, Simon Zünd, David Van Cleve
Thanks, Mike! Responses inline to the two remaining questions.

David

On Wednesday, March 24, 2021 at 3:13:46 AM UTC-7 Mike West wrote:
On Tue, Mar 23, 2021 at 12:17 AM David Van Cleve <dav...@chromium.org> wrote:
Sizing: Rather than working with integrators to produce a specific estimate for "sufficiently large" sizing greater than the current 10% base::Feature rollout (targeting ≤5% usage), we felt it would be simpler to move to a different rollout model. Switching to a rollout where the feature is near-universally enabled (modulo the holdback and environment-specific feature support limitations, like older browser versions and, for now, WebView) hopefully simplifies analysis by substantially mooting questions about adequate sizing.

I agree that the proposed model is simpler to reason about. I'd still like to understand whether that simplicity justifies the substantial jump from <5% usage to ~95% potential availability. Let's say we had philosophical reasons to suggest a 50% holdback rather than the 5% you're asking for; what would the practical impact be for your experimentation?

We haven’t made an effort to evaluate specific usage targets greater than the current 5%-of-PageVisits value, so we don’t have a quantitative answer to that question. Qualitatively, other things equal, less data makes it harder for models to generalize. One factor which can exacerbate this effect is model size: modern classification models often have many parameters, and estimating more parameters requires more data. Another is imbalanced data: in domains like spam detection, “positive” (spam) examples are much less common than “negative” (non-spam) ones.
 

Controls: The "Privacy and security > Clear browsing data" control clears included sites' associated Trust Tokens state (code pointer; lengthier description in the design doc's Clearing data and browser settings section).


Great, thank you. I don't think that doc addresses making stored tokens visible on surfaces like chrome://settings/cookies/detail?site=airhorner.com, which would allow users who care to get a little more insight into who's making assertions about them (to whom). Do y'all have plans for such UI?

We do! We plan to land Trust Tokens-specific settings UI prior to shipping the API. We're still in the relatively early stages of determining the best shape for the UI to take, so it is probably not going to land in the next couple milestones; we'd like to continue the feature's empirical evaluation in parallel with the UI's development.

David Van Cleve

unread,
Apr 11, 2021, 5:32:25 PM4/11/21
to blink-dev, David Van Cleve, Mike West, blink-api-owners-discuss, Charles Harrison, blink-dev, Simon Zünd
Mike and other API owners -

Please let me know if you have any other questions. Thanks! (Sorry for not pinging earlier.)

Thanks for the review,

David

Yoav Weiss

unread,
Apr 12, 2021, 4:02:17 AM4/12/21
to David Van Cleve, blink-dev, Mike West, blink-api-owners-discuss, Charles Harrison, Simon Zünd
On Sun, Apr 11, 2021 at 11:32 PM David Van Cleve <dav...@chromium.org> wrote:
Mike and other API owners -

Please let me know if you have any other questions. Thanks! (Sorry for not pinging earlier.)

Thanks for the review,

David

On Tuesday, March 30, 2021 at 12:02:03 PM UTC-7 David Van Cleve wrote:
Thanks, Mike! Responses inline to the two remaining questions.

David

On Wednesday, March 24, 2021 at 3:13:46 AM UTC-7 Mike West wrote:
On Tue, Mar 23, 2021 at 12:17 AM David Van Cleve <dav...@chromium.org> wrote:
Sizing: Rather than working with integrators to produce a specific estimate for "sufficiently large" sizing greater than the current 10% base::Feature rollout (targeting ≤5% usage), we felt it would be simpler to move to a different rollout model. Switching to a rollout where the feature is near-universally enabled (modulo the holdback and environment-specific feature support limitations, like older browser versions and, for now, WebView) hopefully simplifies analysis by substantially mooting questions about adequate sizing.

I agree that the proposed model is simpler to reason about. I'd still like to understand whether that simplicity justifies the substantial jump from <5% usage to ~95% potential availability. Let's say we had philosophical reasons to suggest a 50% holdback rather than the 5% you're asking for; what would the practical impact be for your experimentation?

We haven’t made an effort to evaluate specific usage targets greater than the current 5%-of-PageVisits value, so we don’t have a quantitative answer to that question. Qualitatively, other things equal, less data makes it harder for models to generalize. One factor which can exacerbate this effect is model size: modern classification models often have many parameters, and estimating more parameters requires more data. Another is imbalanced data: in domains like spam detection, “positive” (spam) examples are much less common than “negative” (non-spam) ones.

That's understandable. At the same time, shipping Trust Tokens to 95% of the population is not-really-distinguishable from actually shipping it.
So, I'd like to clarify - are we talking about enabling the feature for sites that opted-in to the OT?
Will this be a third-party origin trial, where common 3Ps enable it?
What would be the estimated opt-in rate for this?

David Van Cleve

unread,
Apr 13, 2021, 12:52:31 PM4/13/21
to blink-dev, Yoav Weiss, blink-dev, Mike West, blink-api-owners-discuss, Charles Harrison, Simon Zünd, David Van Cleve
Hi Yoav,

Regarding the rollout model's specifics: we're not requesting to graduate from the origin trial; the feature will still be gated by the third party origin trials mechanism. We currently gate the functionality by both the origin trial and a base::Feature: we're just requesting to move to a base::Feature holdback instead of a smaller, explicitly size-calibrated enabled field trial group.

Regarding opt-in rate: Could you please clarify what you mean by opt-in rate? I don't understand the term.

Thanks!

David 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

Yoav Weiss

unread,
Apr 15, 2021, 12:15:52 PM4/15/21
to David Van Cleve, blink-dev, Mike West, blink-api-owners-discuss, Charles Harrison, Simon Zünd
On Tue, Apr 13, 2021 at 6:52 PM David Van Cleve <dav...@chromium.org> wrote:
Hi Yoav,

Regarding the rollout model's specifics: we're not requesting to graduate from the origin trial; the feature will still be gated by the third party origin trials mechanism. We currently gate the functionality by both the origin trial and a base::Feature: we're just requesting to move to a base::Feature holdback instead of a smaller, explicitly size-calibrated enabled field trial group.

That helps clarify things, thanks! :)
 

Regarding opt-in rate: Could you please clarify what you mean by opt-in rate? I don't understand the term.

I wasn't clear, apologies! My question is: If we bump this OT usage cap to 95%, and given that it's a 3P origin trial, which is likely to have many 3Ps opt-in to on behalf of their embedders, do you have any estimate on what %age of overall users will have the feature enabled?
 

Thanks!

David 

David Van Cleve

unread,
Apr 15, 2021, 7:58:03 PM4/15/21
to Yoav Weiss, blink-dev, Mike West, blink-api-owners-discuss, Charles Harrison, Simon Zünd
Hi Yoav,

> My question is: If we bump this OT usage cap to 95%, and given that it's a 3P origin trial, which is likely to have many 3Ps opt-in to on behalf of their embedders, do you have any estimate on what %age of overall users will have the feature enabled?

To be clear, we are not requesting an increase in the feature's OT usage cap (which has a unit of "UseCounter counts" as a fraction of the UseCounter-recorded number of page views). Rather, we are requesting to change the model of the rollout from:

(Current) The feature is gated by a limited field trial (base::Feature) rollout and the Origin Trials framework. To limit the risk of web developer burn-in, the limited field trial rollout ensures the feature's UseCounter-measured usage stays below a specific threshold. 

to:

(Requested) The feature is gated by a nonnegligible field trial (base::Feature) holdback and the Origin Trials framework. A combination of the holdback and period breaking changes limits the risk of web developer burn-in.

I don't think we have public metrics that count the proportion of clients that interact with the feature. Chrome Status shows that the API methods are currently called on around 2% of page loads with the base::Feature rollout somewhere between 10% and 15%. 

Thanks (and sorry for the complexity!),

David

David Van Cleve

unread,
Apr 15, 2021, 7:59:03 PM4/15/21
to Yoav Weiss, blink-dev, Mike West, blink-api-owners-discuss, Charles Harrison, Simon Zünd
("Periodic" breaking changes, that is.)

David Van Cleve

unread,
Apr 20, 2021, 6:55:46 PM4/20/21
to blink-api-owners-discuss, David Van Cleve, blink-dev, Mike West, blink-api-owners-discuss, Charles Harrison, Simon Zünd, Yoav Weiss
Hi Yoav, Mike, and other API owners -

Friendly ping for further review. Please let me know if you have additional questions.

Thanks! 


Thanks!

David 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

Chris Harrelson

unread,
Apr 20, 2021, 8:30:17 PM4/20/21
to David Van Cleve, blink-api-owners-discuss, blink-dev, Mike West, Charles Harrison, Simon Zünd, Yoav Weiss
On Tue, Apr 20, 2021 at 3:55 PM David Van Cleve <dav...@chromium.org> wrote:
Hi Yoav, Mike, and other API owners -

Friendly ping for further review. Please let me know if you have additional questions.

Thanks! 

On Thursday, April 15, 2021 at 4:59:03 PM UTC-7 David Van Cleve wrote:
("Periodic" breaking changes, that is.)

On Thu, Apr 15, 2021 at 4:57 PM David Van Cleve <dav...@chromium.org> wrote:
Hi Yoav,

> My question is: If we bump this OT usage cap to 95%, and given that it's a 3P origin trial, which is likely to have many 3Ps opt-in to on behalf of their embedders, do you have any estimate on what %age of overall users will have the feature enabled?

To be clear, we are not requesting an increase in the feature's OT usage cap (which has a unit of "UseCounter counts" as a fraction of the UseCounter-recorded number of page views). Rather, we are requesting to change the model of the rollout from:

(Current) The feature is gated by a limited field trial (base::Feature) rollout and the Origin Trials framework. To limit the risk of web developer burn-in, the limited field trial rollout ensures the feature's UseCounter-measured usage stays below a specific threshold. 

to:

(Requested) The feature is gated by a nonnegligible field trial (base::Feature) holdback and the Origin Trials framework. A combination of the holdback and period breaking changes limits the risk of web developer burn-in.

I don't think we have public metrics that count the proportion of clients that interact with the feature. Chrome Status shows that the API methods are currently called on around 2% of page loads with the base::Feature rollout somewhere between 10% and 15%.

I still don't completely understand. Are you saying that the trial will be detectable on 95% of pages, but actual usage won't be able to exceed the standard OT usage cap?

To put it another way: what is the maximum percentage of page loads that could have this API being used on them, under your proposal?


Thanks!

David 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/666de3e6-c1db-4ceb-a19d-61798379c992n%40chromium.org.

Chris Harrelson

unread,
May 26, 2021, 12:40:40 PM5/26/21
to Chris Harrelson, David Van Cleve, blink-api-owners-discuss, Mike West, Charles Harrison, Simon Zünd, Yoav Weiss
Hi David,

I sent an email to blink-dev announcing the new agreement to allow increased Origin Trial limits for cases such as the Trust Token API. Could you send an official request to increase limits to blink-dev, so we can approve it tomorrow?

Thanks,
Chris

David Van Cleve

unread,
May 26, 2021, 1:39:48 PM5/26/21
to blink-api-owners-discuss, Chris Harrelson, David Van Cleve, blink-api-owners-discuss, Mike West, Charles Harrison, Simon Zünd, Yoav Weiss, Chris Harrelson
Hi Chris, sorry for the confusing threading: I think my request upthread, which is cced to blink-dev, is still current. (Lesson learned: Start a new thread for each incremental request for approval!)

Thanks again for all your help!

David

Chris Harrelson

unread,
May 27, 2021, 3:32:15 PM5/27/21
to David Van Cleve, blink-api-owners-discuss, David Van Cleve, Mike West, Charles Harrison, Simon Zünd, Yoav Weiss
Hi,

Can you just re-post an email to the main thread saying you'll meet the requirements of my email from two days ago:

* Actual percentage of exposed users will <= 20%
* Monthly reporting to the API owners
* <= 2 milestones (and please specify them in your email)

Then one of us will LGTM it.

David Van Cleve

unread,
May 27, 2021, 4:59:22 PM5/27/21
to blink-api-owners-discuss, David Van Cleve, Mike West, Yoav Weiss, David Van Cleve, blink-api-owners-discuss, Rick Byers, sva...@chromium.org, Charles Harrison, Jason Chase, blink-dev, Alex Russell
Hi API owners,

At chrishtr@'s suggestion, I'm reposting this earlier request for your renewed consideration now that the policy on alternate origin limits for privacy-preserving APIs has landed, and confirming:

* We expect usage to remain below 20%
* We'll check in regularly
* We'll have a backwards-incompatible change within two milestones (in fact, in both M92 and M93)

API owners,

We request your approval to move the Trust Token API origin trial to a rollout limiting burn-in with a 5% holdback, superseding the origin trial's current fixed usage target.

When initially estimating what subsample would suffice to provide generalizable experiment results, our experimentation partners used a particular training procedure to train models on subsampled data and found that a 10% subsample led to reasonable quality. However, this process involved some assumptions in its choice of training procedure used: in the event, these did not correspond well to the approach used to evaluate live data from the Trust Token API origin trial rollout.

In the time since our prior request for a higher, but still strictly limited, alternative origin trial usage target, third-party origin trial procedure design work has proposed an alternate rollout model aiming to avoid burn-in by using a small base::Feature holdback, accompanied by periodic cliffs involving either backwards-incompatible changes or disabling the feature. Rather than requesting a specific further limited increase to the origin trial usage target for this trial, we’d like to switch to the alternate model of limiting burn-in with a persistent 5% holdback and periodic backwards-incompatible changes. We’d additionally like to emphasize again that we believe the Trust Token API has a particularly low risk of burn-in, because trust tokens are intended to be a less useful, but privacy-better, alternative to third party cookies, a currently available alternative.

Thanks again,

David Van Cleve

Chris Harrelson

unread,
May 27, 2021, 5:02:29 PM5/27/21
to David Van Cleve, blink-api-owners-discuss, David Van Cleve, Mike West, Yoav Weiss, Rick Byers, sva...@chromium.org, Charles Harrison, Jason Chase, blink-dev, Alex Russell
Thanks!

LGTM1

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/e3b995a4-cc9d-4038-86af-6be6e7e9ebc8n%40chromium.org.

Yoav Weiss

unread,
May 27, 2021, 5:27:38 PM5/27/21
to Chris Harrelson, David Van Cleve, blink-api-owners-discuss, David Van Cleve, Mike West, Rick Byers, sva...@chromium.org, Charles Harrison, Jason Chase, blink-dev, Alex Russell
LGTM2

Daniel Bratell

unread,
May 27, 2021, 6:07:29 PM5/27/21
to Chris Harrelson, David Van Cleve, blink-api-owners-discuss, David Van Cleve, Mike West, Charles Harrison, Simon Zünd, Yoav Weiss
On 2021-05-27 21:31, 'Chris Harrelson' via blink-api-owners-discuss wrote:
Hi,

Can you just re-post an email to the main thread saying you'll meet the requirements of my email from two days ago:

* Actual percentage of exposed users will <= 20%
* Monthly reporting to the API owners
* <= 2 milestones (and please specify them in your email)

Then one of us will LGTM it.
Or rather three of us (exceptions like this needs 3 LGTMs, not the normal one for origin trials; just writing it here to avoid misunderstandings, it's already been described in Chris' decision motivation post).

Daniel Bratell

unread,
May 27, 2021, 6:09:37 PM5/27/21
to Yoav Weiss, Chris Harrelson, David Van Cleve, blink-api-owners-discuss, David Van Cleve, Mike West, Rick Byers, sva...@chromium.org, Charles Harrison, Jason Chase, blink-dev, Alex Russell

LGMT3 for running beyond the normal limits of an origin trial, adhering to conditions as described. Good luck!

/Daniel

Reply all
Reply to author
Forward
0 new messages