Origin trial timelines with Chrome 4 week releases

[Jason Chase]

Hi API Owners,

With Chrome moving to 4 week releases, that will impact timelines for origin trials.

From an I2E perspective, I think the primary impact is considering the "length" of trial:

• The typical/recommended length for a trial is 3 milestones, currently ~18 weeks.
• A cap of 6 milestones, total, for extended trials (without evidence of changes to avoid burn-in).

I suggest we keep the length/duration guidelines the same, in terms of elapsed time, and adjust number of milestones to match.

• Typical length: 4 milestones, or ~16 weeks (a little shorter than currently).
• Cap on extended trials: 9 milestones, or ~36 weeks.

We see that trials are often extended because developers aren't ready to participate and thus we don't collect enough feedback. Staying with 3 cycles would shorten the duration (in days), which would only make that problem worse.

I don't recall that the recommended 3 cycles was chosen based on deep analysis or hard/fast rules. Most likely it was a reasonable round number, given that releases were usually every 6 weeks (e.g. we don't worry that some releases are longer than others).

Questions:

• Are there other timelines in the I2E process that we need to consider?
• Thoughts on the suggestions above?

Thanks,

Jason

[Chris Harrelson]

+1. This change to adjust to have a consistent total time rather than release count makes sense.

On Tue, Jun 15, 2021 at 12:53 PM Daniel Bratell <brat...@gmail.com> wrote:

I think it makes sense to increase the number of milestones, and possibly shorten the number of weeks.

In the origin trial the team will need to collect feedback, do updates, collect more feedbacks, and so on, and with shorter cycles, more such feature updates will be possible in fewer release cycles. In an ideal world,

So your suggestion seems good to me.

/Daniel

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/bb2d8100-cd8b-4c18-bc33-823580e75553n%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/9b8db03a-d7fe-3c88-4592-85ca23fca05f%40gmail.com.

[Johnny Stenback]

Likewise, +1 from me as well, tying trials to amount of time rather than # of releases (conceptually) makes much more sense to me than the other way around.

[Rick Byers]

+1 to just staying consistent with time (absent some argument for why it was a bad choice). Thanks for driving this Jason!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CACZRgz5JqtV1zphxREfKCy8BG0jeaDETM%3DJ7AsKEW4zzp%3DZg%2BA%40mail.gmail.com.

[Mike West]

Thanks for the suggestion, Jason!

I've documented this change at https://sites.google.com/a/chromium.org/dev/blink/origin-trials/running-an-origin-trial#TOC-How-long-do-Origin-Trials-typically-last-. Is there other documentation we should adjust as well?

-mike

On Wed, Jun 16, 2021 at 8:26 AM 'Yoav Weiss' via blink-api-owners-discuss <blink-api-ow...@chromium.org> wrote:

+1 from me as well!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAFUtAY-6MbuoKL7bTA2mz8R7BP2%3D%3Dsyod61dZGpRNQw9znfKCw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfW6W7j6Q3UWizexZRLt87MX%3DXcjP%3DDMbuha6fs_aonWwg%40mail.gmail.com.