That sounds reasonable to me. LGTM1--On Thu, Sep 3, 2020 at 1:47 AM <dav...@chromium.org> wrote:--We'd like to request an increase from the standard “0.5% of page loads” origin trial usage target, to 5%. We feel this won’t lead to burn-in because the API is intended to be a less useful (but privacy-better) long-term partial substitute for an existing web feature (third-party cookies): the raison d’etre of the API in the current world is prototyping and experimentation.The main goal of this origin trial is to understand whether trust tokens are useful for fraud detection. A key challenge is distinguishing the effect of sparsity---if we enable Trust Tokens for 1% of clients, models trained on observed tokens will see a roughly 1% sample---from the effect of the inherent coarseness of tokens’ encoded information.Our experimentation partners have evaluated existing models analogous to potential trust token-based models on sparse subsamples of their training data; they’ve found that roughly a 10% sample would suffice to retain most of the models’ quality. This translates to estimated UseCounter values of under 5%.^The primary argument for keeping origin trial usage low is the ability to avoid large developers depending on the feature. We don’t expect developers to develop a production reliance on tokens’ embedded information during this origin trial, because trust tokens are harder to use and contain far less information than third-party cookies (the benefit being that they’re much better for privacy). Further, since Trust Tokens is independently gated by a base::Feature, origin trial participants will still see the API unavailable for a substantial majority of their users.^ We currently have the feature enabled for 3% of clients; this number is calibrated conservatively to lead to feature usage below the standard 0.5% UseCounter target. Extrapolating, increasing the rollout to 10% of clients would lead to UseCounter values below a couple percentage points; 5% is an upper bound.
On Thursday, May 21, 2020 at 1:21:18 PM UTC-7, David Van Cleve wrote:Thanks, Rick!On Thu, May 21, 2020 at 4:07 PM Rick Byers <rby...@chromium.org> wrote:I meant to mention that I see the Signed Exchange OT as having a similar precedent here, which we debated at some length. There we required only the origin serving the exchange to opt-in to the OT, not the origin of the signed document itself. Technically this meant one origin could be exposed to the effects of a trial enabled by another origin. 3P OTs are going to give us that property officially, and so I don't think your proposal is too fundamentally different. Of course you need to think through whether sites might change behavior in some way based purely on the existence of the API surface, but I assume the risk in the short-run would be low.RickOn Thu, May 21, 2020 at 3:59 PM Rick Byers <rby...@chromium.org> wrote:Sorry for the delay David. I agree 3p OT is the right solution here, but also that what you propose is a viable fallback that will be nearly equivalent if we believe there's little chance of someone taking a dependency on any of the API without access to the issuance API. Trust Tokens also doesn't seem like the sort of thing we're going to see a huge number of developers interested in trying to use as soon as possible, so that further reduces the risk of burn-in in my mind.I'd say that unless any other API owner objects here in the next 24 hours, you should consider this modified OT plan acceptable as a stop-gap until the 3P OT infrastructure is available.Thanks for reaching out.RickOn Wed, May 20, 2020 at 5:32 PM <dav...@chromium.org> wrote:Friendly ping. I wasn't sure if this was the right destination for this kind of exception request; please let me know if elsewhere (blink-dev?) would be better.--
On Tuesday, May 19, 2020 at 2:12:31 PM UTC-4, David Van Cleve wrote:We're working on origin trialling the Trust Token API and would like your approval for a nonstandard origin trial plan.Background- The API provides the ability to specify cryptographic operations ("issuance," "redemption," and "signing") to execute alongside outgoing requests. The three operations are specified by varying values of the same parameters object; fetch, XHR, and the iframe element's interfaces expand to accept instances of this object.- We expect signing and redemption will be executed by third-party scripts in a wide variety of first-party contexts, and it would be infeasible to have each of these first-party sites enroll in the trial.- The origin trials team is working on launching third party origin trials, which allow a script served from an origin enrolled in an origin trial to provide the origin trial's token, but it's not certain that this will launch by M84.Our planOur preferred plan is to use a third party origin trial, which will allow us to gate all three operations behind the origin trial: the third-party scripts executing redemption and signing will be able to provide origin trial tokens in order to execute the operations.However, assuming third party origin trials don't launch in M84, we'd like to be able to fall back to using a standard origin trial to gate usage of the feature.The API's signing operation uses the output of redemption, which in turn uses the output of issuance, so it's possible to restrict usage of signing and redemption implicitly by gating only issuance behind the origin trial. Since issuance will be executed by first-party script in a relatively small number of contexts, we can require an origin trial token in order to execute issuance. If we see (aggregate) usage exceeding the origin trial limit, we'll use an associated base::Feature as a kill switch, globally removing the API surface.This plan is nonstandard because it would mean that the API surface for signing and redemption is exposed whenever the base::Feature is enabled, both in contexts with origin trial tokens and without, with usage only implicitly limited.Our requestWe'd like your approval since this trial plan changes the stable API surface whenever the base::Feature is enabled (which will be for the duration of the origin trial, but will likely be for a small minority of users).We've cleared this with the origin trials team as a fallback and feel that, while it isn't structurally ideal, monitoring UseCounters and having a base::Feature kill switch will suffice to prevent burn-in.David Van Cleve, Steven Valdez, Charlie Harrison
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/790ba961-5df6-4574-af3c-4f1b41cfbc8b%40chromium.org.
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/2796bc9d-05bc-4a7a-8c5f-6fa7b9ec84b6o%40chromium.org.
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfVD_ATZ-vFBqUy8qTT4Q%2Bm1fcKy3VHRsMSTYA7mbBGQpw%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CANr5HFUK8ig7%2Bo%2BGM%3D4u-CrXb2t5m33vZMxGzhKGY9a17dPZLw%40mail.gmail.com.
API owners,
We request your approval to move the Trust Token API origin trial to a rollout limiting burn-in with a 5% holdback, superseding the origin trial's current fixed usage target.
When initially estimating what subsample would suffice to provide generalizable experiment results, our experimentation partners used a particular training procedure to train models on subsampled data and found that a 10% subsample led to reasonable quality. However, this process involved some assumptions in its choice of training procedure used: in the event, these did not correspond well to the approach used to evaluate live data from the Trust Token API origin trial rollout.
In the time since our prior request for a higher, but still strictly limited, alternative origin trial usage target, third-party origin trial procedure design work has proposed an alternate rollout model aiming to avoid burn-in by using a small base::Feature holdback, accompanied by periodic cliffs involving either backwards-incompatible changes or disabling the feature. Rather than requesting a specific further limited increase to the origin trial usage target for this trial, we’d like to switch to the alternate model of limiting burn-in with a persistent 5% holdback and periodic backwards-incompatible changes. We’d additionally like to emphasize again that we believe the Trust Token API has a particularly low risk of burn-in, because trust tokens are intended to be a less useful, but privacy-better, alternative to third party cookies, a currently available alternative.Sizing: Rather than working with integrators to produce a specific estimate for "sufficiently large" sizing greater than the current 10% base::Feature rollout (targeting ≤5% usage), we felt it would be simpler to move to a different rollout model. Switching to a rollout where the feature is near-universally enabled (modulo the holdback and environment-specific feature support limitations, like older browser versions and, for now, WebView) hopefully simplifies analysis by substantially mooting questions about adequate sizing.
Controls: The "Privacy and security > Clear browsing data" control clears included sites' associated Trust Tokens state (code pointer; lengthier description in the design doc's Clearing data and browser settings section).
Sizing: Rather than working with integrators to produce a specific estimate for "sufficiently large" sizing greater than the current 10% base::Feature rollout (targeting ≤5% usage), we felt it would be simpler to move to a different rollout model. Switching to a rollout where the feature is near-universally enabled (modulo the holdback and environment-specific feature support limitations, like older browser versions and, for now, WebView) hopefully simplifies analysis by substantially mooting questions about adequate sizing.
Controls: The "Privacy and security > Clear browsing data" control clears included sites' associated Trust Tokens state (code pointer; lengthier description in the design doc's Clearing data and browser settings section).
Developer support: szuend@ (cced) has been busy landing very neat DevTools support; you can find instructions and some screenshots in our web.dev article. I think the DevTools support is probably mature enough to be a sufficient aid for debugging new JS code interacting with an existing token issuance service, or for (say) a site to debug integrating JS library code that uses the API. The NetLog integration still comes in handy when developing a new issuance service, since it provides visibility into the browser's guts to help diagnose specific server-side implementation issues.
On Tue, Mar 23, 2021 at 12:17 AM David Van Cleve <dav...@chromium.org> wrote:Sizing: Rather than working with integrators to produce a specific estimate for "sufficiently large" sizing greater than the current 10% base::Feature rollout (targeting ≤5% usage), we felt it would be simpler to move to a different rollout model. Switching to a rollout where the feature is near-universally enabled (modulo the holdback and environment-specific feature support limitations, like older browser versions and, for now, WebView) hopefully simplifies analysis by substantially mooting questions about adequate sizing.I agree that the proposed model is simpler to reason about. I'd still like to understand whether that simplicity justifies the substantial jump from <5% usage to ~95% potential availability. Let's say we had philosophical reasons to suggest a 50% holdback rather than the 5% you're asking for; what would the practical impact be for your experimentation?
Controls: The "Privacy and security > Clear browsing data" control clears included sites' associated Trust Tokens state (code pointer; lengthier description in the design doc's Clearing data and browser settings section).
Great, thank you. I don't think that doc addresses making stored tokens visible on surfaces like chrome://settings/cookies/detail?site=airhorner.com, which would allow users who care to get a little more insight into who's making assertions about them (to whom). Do y'all have plans for such UI?
Mike and other API owners -Please let me know if you have any other questions. Thanks! (Sorry for not pinging earlier.)Thanks for the review,DavidOn Tuesday, March 30, 2021 at 12:02:03 PM UTC-7 David Van Cleve wrote:Thanks, Mike! Responses inline to the two remaining questions.DavidOn Wednesday, March 24, 2021 at 3:13:46 AM UTC-7 Mike West wrote:On Tue, Mar 23, 2021 at 12:17 AM David Van Cleve <dav...@chromium.org> wrote:Sizing: Rather than working with integrators to produce a specific estimate for "sufficiently large" sizing greater than the current 10% base::Feature rollout (targeting ≤5% usage), we felt it would be simpler to move to a different rollout model. Switching to a rollout where the feature is near-universally enabled (modulo the holdback and environment-specific feature support limitations, like older browser versions and, for now, WebView) hopefully simplifies analysis by substantially mooting questions about adequate sizing.I agree that the proposed model is simpler to reason about. I'd still like to understand whether that simplicity justifies the substantial jump from <5% usage to ~95% potential availability. Let's say we had philosophical reasons to suggest a 50% holdback rather than the 5% you're asking for; what would the practical impact be for your experimentation?We haven’t made an effort to evaluate specific usage targets greater than the current 5%-of-PageVisits value, so we don’t have a quantitative answer to that question. Qualitatively, other things equal, less data makes it harder for models to generalize. One factor which can exacerbate this effect is model size: modern classification models often have many parameters, and estimating more parameters requires more data. Another is imbalanced data: in domains like spam detection, “positive” (spam) examples are much less common than “negative” (non-spam) ones.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/ffc2f877-1b98-4bc6-a1ca-d5e6ce88d67fn%40chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.
Hi Yoav,Regarding the rollout model's specifics: we're not requesting to graduate from the origin trial; the feature will still be gated by the third party origin trials mechanism. We currently gate the functionality by both the origin trial and a base::Feature: we're just requesting to move to a base::Feature holdback instead of a smaller, explicitly size-calibrated enabled field trial group.
Regarding opt-in rate: Could you please clarify what you mean by opt-in rate? I don't understand the term.
Thanks!David
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/ffc2f877-1b98-4bc6-a1ca-d5e6ce88d67fn%40chromium.org.
Thanks!David
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.
Hi Yoav, Mike, and other API owners -Friendly ping for further review. Please let me know if you have additional questions.Thanks!On Thursday, April 15, 2021 at 4:59:03 PM UTC-7 David Van Cleve wrote:("Periodic" breaking changes, that is.)On Thu, Apr 15, 2021 at 4:57 PM David Van Cleve <dav...@chromium.org> wrote:Hi Yoav,> My question is: If we bump this OT usage cap to 95%, and given that it's a 3P origin trial, which is likely to have many 3Ps opt-in to on behalf of their embedders, do you have any estimate on what %age of overall users will have the feature enabled?To be clear, we are not requesting an increase in the feature's OT usage cap (which has a unit of "UseCounter counts" as a fraction of the UseCounter-recorded number of page views). Rather, we are requesting to change the model of the rollout from:(Current) The feature is gated by a limited field trial (base::Feature) rollout and the Origin Trials framework. To limit the risk of web developer burn-in, the limited field trial rollout ensures the feature's UseCounter-measured usage stays below a specific threshold.to:(Requested) The feature is gated by a nonnegligible field trial (base::Feature) holdback and the Origin Trials framework. A combination of the holdback and period breaking changes limits the risk of web developer burn-in.I don't think we have public metrics that count the proportion of clients that interact with the feature. Chrome Status shows that the API methods are currently called on around 2% of page loads with the base::Feature rollout somewhere between 10% and 15%.
Thanks!David
--To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/ffc2f877-1b98-4bc6-a1ca-d5e6ce88d67fn%40chromium.org.
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/666de3e6-c1db-4ceb-a19d-61798379c992n%40chromium.org.
API owners,
We request your approval to move the Trust Token API origin trial to a rollout limiting burn-in with a 5% holdback, superseding the origin trial's current fixed usage target.
When initially estimating what subsample would suffice to provide generalizable experiment results, our experimentation partners used a particular training procedure to train models on subsampled data and found that a 10% subsample led to reasonable quality. However, this process involved some assumptions in its choice of training procedure used: in the event, these did not correspond well to the approach used to evaluate live data from the Trust Token API origin trial rollout.
In the time since our prior request for a higher, but still strictly limited, alternative origin trial usage target, third-party origin trial procedure design work has proposed an alternate rollout model aiming to avoid burn-in by using a small base::Feature holdback, accompanied by periodic cliffs involving either backwards-incompatible changes or disabling the feature. Rather than requesting a specific further limited increase to the origin trial usage target for this trial, we’d like to switch to the alternate model of limiting burn-in with a persistent 5% holdback and periodic backwards-incompatible changes. We’d additionally like to emphasize again that we believe the Trust Token API has a particularly low risk of burn-in, because trust tokens are intended to be a less useful, but privacy-better, alternative to third party cookies, a currently available alternative.
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/e3b995a4-cc9d-4038-86af-6be6e7e9ebc8n%40chromium.org.
LGMT3 for running beyond the normal limits of an origin trial,
adhering to conditions as described. Good luck!
/Daniel
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfWmPB2PDwSDqLJvA-7tVQ_6%2BvdNd2c9cgFRXgBOpur_vA%40mail.gmail.com.