Experiment to measure impact on startup latency

[Chris Fredrickson]

Hi API owners,

I'd like to run an experiment to collect some latency metrics for my project. At a high level, my feature is initialized asynchronously at Chrome startup, and (when enabled) must be completely initialized before the Network Service accesses cookies. There's prior art for doing this kind of thing, namely reading the persistent cookie store DB into memory at startup.

Currently, the feature is disabled, so the First-Party Sets backend answers every query synchronously (and the answers are no-ops). When the feature is enabled, some queries may be answered asychronously, if they arrive before the backend is fully initialized. (The answers will still be no-ops for now, since the feature has not launched.) I'd like to run an experiment in which Chrome enables the feature at 50% on Canary/Dev, 50% on Beta, and 1% on Stable, to verify that the latency characteristics of this are acceptable.

Some more details:

• What is affected? Cookie accesses (from both HTTP and scripts) during Chrome startup.
  • Web requests which are not HTTP(S)/WS(S) are not affected.
  • HTTP(S)/WS(S) requests which are uncredentialed are not affected.
  • Cookie accesses (via HTTP or script) that occur after startup are not affected.
• What happens during First-Party Sets initialization?
  • Backend waits to receive a file from component updater, and reads/parses the contents.
    • If the component is not yet installed, or we fail to read the file, we use an invalid base::File instead.
  • Backend reads/parses a command-line flag (or empty string if the flag isn't present).
  • Backend merges the component and flag values.
• What metrics will I collect?
  • PageLoad.InteractiveTiming.FirstInputDelay4
  • PageLoad.PaintTiming.NavigationToFirstContentfulPaint
  • Startup.FirstWebContents.MainNavigationStart
  • Startup.FirstWebContents.MainNavigationFinished
  • Startup.FirstWebContents.NonEmptyPaint3
  • Fine-grained metrics to monitor each part of initialization; see https://crrev.com/c/3501915.
  • Fine-grained metrics to monitor the impacts of query delays. See https://crrev.com/c/3465494.
• Any special concerns?
  • Deadlock. The backend must initialize no matter what, in order to eventually serve queries for cookie accesses.
  • Priority inversion. I've updated all disk reads on the critical path to use USER_BLOCKING priority if this feature is enabled, and BEST_EFFORT otherwise.
• What's the proposed rollout plan?
  • A week or two at 50% on canary/dev; then
  • A week or two at 50% on beta; then
  • 2 weeks at 1% on stable.
  • Platforms: Windows, Linux, Mac, ChromeOS, Lacros, Android browser

Any concerns with this approach, or questions that I haven't addressed? Thanks!

Chris

[Chris Harrelson]

Hi Chris,

Apologies for the slow response.

Could you clarify the web-exposed impact to developers when this experiment is on? Does something change regarding cookies that developers can see?

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/0012295e-8506-42ea-80cd-bb8dd70dcc40n%40chromium.org.

[Yoav Weiss]

On Wed, Mar 9, 2022 at 7:07 PM Chris Harrelson <chri...@chromium.org> wrote:

Hi Chris,

Apologies for the slow response.

Could you clarify the web-exposed impact to developers when this experiment is on? Does something change regarding cookies that developers can see?

On Fri, Mar 4, 2022 at 1:33 PM Chris Fredrickson <cfre...@chromium.org> wrote:

Hi API owners,

I'd like to run an experiment to collect some latency metrics for my project. At a high level, my feature is initialized asynchronously at Chrome startup, and (when enabled) must be completely initialized before the Network Service accesses cookies. There's prior art for doing this kind of thing, namely reading the persistent cookie store DB into memory at startup.

Currently, the feature is disabled, so the First-Party Sets backend answers every query synchronously (and the answers are no-ops). When the feature is enabled, some queries may be answered asychronously, if they arrive before the backend is fully initialized. (The answers will still be no-ops for now, since the feature has not launched.) I'd like to run an experiment in which Chrome enables the feature at 50% on Canary/Dev, 50% on Beta, and 1% on Stable, to verify that the latency characteristics of this are acceptable.

Some more details:

• What is affected? Cookie accesses (from both HTTP and scripts) during Chrome startup

To build on ChrisH's question - will the first requests after startup be sent without credentials, or will they be delayed until the right credentials are available? 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CA%2BN6QZtWN8RzB%3Ds-tjcCBnUxHr9G5jPPfeE9Y0OymYtKF1auVw%40mail.gmail.com.

[Chris Fredrickson]

Sorry for the delayed response, just getting back from OOO. Responses inline.

On Thursday, March 10, 2022 at 1:14:59 AM UTC-5 Yoav Weiss wrote:

On Wed, Mar 9, 2022 at 7:07 PM Chris Harrelson <chri...@chromium.org> wrote:

Hi Chris,

Apologies for the slow response.

Could you clarify the web-exposed impact to developers when this experiment is on? Does something change regarding cookies that developers can see?

The web-exposed impact will be increased latency at Chromium startup time; the semantics of the requests/cookies are unchanged.

 

On Fri, Mar 4, 2022 at 1:33 PM Chris Fredrickson <cfre...@chromium.org> wrote:

Hi API owners,

I'd like to run an experiment to collect some latency metrics for my project. At a high level, my feature is initialized asynchronously at Chrome startup, and (when enabled) must be completely initialized before the Network Service accesses cookies. There's prior art for doing this kind of thing, namely reading the persistent cookie store DB into memory at startup.

Currently, the feature is disabled, so the First-Party Sets backend answers every query synchronously (and the answers are no-ops). When the feature is enabled, some queries may be answered asychronously, if they arrive before the backend is fully initialized. (The answers will still be no-ops for now, since the feature has not launched.) I'd like to run an experiment in which Chrome enables the feature at 50% on Canary/Dev, 50% on Beta, and 1% on Stable, to verify that the latency characteristics of this are acceptable.

Some more details:

• What is affected? Cookie accesses (from both HTTP and scripts) during Chrome startup

To build on ChrisH's question - will the first requests after startup be sent without credentials, or will they be delayed until the right credentials are available? 

They'll be delayed until the right credentials are available.

Right now, the feature's backend answers every query with "no-op", but eventually when the feature launches, the responses will be meaningful. This experiment is solely meant to measure the time needed to initialize the feature in Chromium, rather than measuring the impact of the feature's semantics/usage. (We measured that separately via an origin trial.)

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/0012295e-8506-42ea-80cd-bb8dd70dcc40n%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

[Yoav Weiss]

On Mon, Mar 14, 2022 at 5:30 PM Chris Fredrickson <cfre...@chromium.org> wrote:

Sorry for the delayed response, just getting back from OOO. Responses inline.

On Thursday, March 10, 2022 at 1:14:59 AM UTC-5 Yoav Weiss wrote:

On Wed, Mar 9, 2022 at 7:07 PM Chris Harrelson <chri...@chromium.org> wrote:

Hi Chris,

Apologies for the slow response.

Could you clarify the web-exposed impact to developers when this experiment is on? Does something change regarding cookies that developers can see?

The web-exposed impact will be increased latency at Chromium startup time; the semantics of the requests/cookies are unchanged.

From my perspective, that's user-exposed, but not web-exposed. (web developers cannot detect that this is happening in content and grow to rely on this in weird ways)

As such, I think this is exempt from the Blink process, unless someone disagrees.

Note that you probably still need to go through product-specific processes to approve this.

 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/0012295e-8506-42ea-80cd-bb8dd70dcc40n%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

[Chris Fredrickson]

That sounds reasonable, thanks - just wanted to double-check that this was ok from Blink's perspective. :)

I am going through other approval processes, thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/0012295e-8506-42ea-80cd-bb8dd70dcc40n%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

[Chris Harrelson]

Agreed with Yoav that this sounds like it isn't web-exposed, in the sense of "developers need to know about it because it might break their site". So from the API owners perspective you're good to go.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/0012295e-8506-42ea-80cd-bb8dd70dcc40n%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.