Allow Origin Trials to transition smoothly to launch for "strict additions"?

[Chris Harrelson]

By default, Origin Trials require a breaking period between experiment and launch. This is to force sites to update their code as necessary and/or support the non-OT code paths, so as to prevent experiments from becoming a vehicle for soft launches of features or burn-in of a non-standard API.

Previously we have agreed that if the shipping form of a feature is identical to the form it takes during an Origin Trial (because during the experiment it turned out there was no change needed to the API), then it's ok not to have a breaking period. Our reasoning was that it seems an unnecessary speed bump for no benefit, and in fact causes busywork for partners that spent the time to experiment with the feature.

How about cases where there is a strict addition made to the feature that is 100% backward compatible? e.g. let's say we have an OT for a feature called Foo, that involves one new method on the window object called bar(). During the OT we discover a use case missed in the original Foo API shape, that involves adding a new method baz(), but leaves bar() unchanged, and if baz() is not called the browser behavior is identical.

Would this be an OK situation for "smooth launch"? I think so but would like more opinions, and a joint decision that we can give to feature owners. (Also, this question is not purely theoretical and came up for a recent Origin Trial.)

Chris

[Yoav Weiss]

I think that ideally in such a case, the guidance should be to run the OT one more release with the additive change, ensure it's working well for partners, and by then, we would coalesce into the case of "we want to ship the OT as is".

Is that not feasible in the case where this has come up?

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CA%2BN6QZs%3DGbNPpVPwJx%3DPSbn_KvWSJqZfUzz4MfpNT1wyXRQRuQ%40mail.gmail.com.

[Chris Harrelson]

On Tue, Aug 25, 2020 at 3:10 AM Yoav Weiss <yoav...@google.com> wrote:

I think that ideally in such a case, the guidance should be to run the OT one more release with the additive change, ensure it's working well for partners, and by then, we would coalesce into the case of "we want to ship the OT as is".

Is that not feasible in the case where this has come up?

It might be, but it delays shipping the feature by 6 more weeks. Is there a defined risk that justifies this delay?

[Yoav Weiss]

That's a fair point. I guess that depends on the level of risk related to the additional API surface (and how likely we are to get it wrong).

In cases where that risk is low (e.g. API was tested by partners behind a flag or obvious addition to the API shape), it makes sense IMO to enable a "smooth launch" in order to reduce OT friction.

[Chris Harrelson]

On Tue, Aug 25, 2020 at 8:24 AM Yoav Weiss <yoav...@google.com> wrote:

That's a fair point. I guess that depends on the level of risk related to the additional API surface (and how likely we are to get it wrong).

Origin Trials are not required for shipping features. We leave it up to the judgement of the feature owners and the spec communities they interact with to decide if the current feature is done right.

I think the only considerations we should take into account here are ones about risks to interop/burn-in or mis-use of the purpose of Origin Trials.

 

In cases where that risk is low (e.g. API was tested by partners behind a flag or obvious addition to the API shape), it makes sense IMO to enable a "smooth launch" in order to reduce OT friction.

Ok. Perhaps we should just ask the feature owner to just consider these and state as such for the record in their request for a smooth rollout?

[Yoav Weiss]

On Tue, Aug 25, 2020 at 6:02 PM Chris Harrelson <chri...@chromium.org> wrote:

On Tue, Aug 25, 2020 at 8:24 AM Yoav Weiss <yoav...@google.com> wrote:

That's a fair point. I guess that depends on the level of risk related to the additional API surface (and how likely we are to get it wrong).

Origin Trials are not required for shipping features. We leave it up to the judgement of the feature owners and the spec communities they interact with to decide if the current feature is done right.

I think the only considerations we should take into account here are ones about risks to interop/burn-in or mis-use of the purpose of Origin Trials.

 

In cases where that risk is low (e.g. API was tested by partners behind a flag or obvious addition to the API shape), it makes sense IMO to enable a "smooth launch" in order to reduce OT friction.

Ok. Perhaps we should just ask the feature owner to just consider these and state as such for the record in their request for a smooth rollout?

Yeah, that makes sense to me!

[Mike West]

Generally speaking, I'm ok with this direction. It seems reasonable to suggest that when we discover some missing method in an API through developers' usage and feedback that we should not punish developers for sharing their experience, but allow them to continue using the API along with its extensions. Perhaps a middle ground would be to turn off signups for the given origin trial while allowing existing keys to continue working until the feature hits stable.

That said, a concern that I have with this proposal is that it might bias us towards strictly adding additional methods to APIs rather than changing the APIs in a breaking way, even if we learn that some other spelling of an API would be a better fit for its users. Given that folks signing up for origin trial are explicitly signing up for breakage and change, I wonder whether it might be reasonable to widen the proposal to include shifting straight into breaking changes without pause for existing OT users That entails some risk, insofar as existing OT users would need to update their code to account for the change. I'm not sure how to weigh that risk against the annoyance of a critical feature disappearing for a milestone between OT and shipping.

-mike

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfXK3mxPTAiKrp4BhUFf6aPKbonhkibcu_KctKFHy%3DhR%3DQ%40mail.gmail.com.

[Yoav Weiss]

On Thu, Aug 27, 2020 at 9:43 PM Mike West <mk...@chromium.org> wrote:

Generally speaking, I'm ok with this direction. It seems reasonable to suggest that when we discover some missing method in an API through developers' usage and feedback that we should not punish developers for sharing their experience, but allow them to continue using the API along with its extensions. Perhaps a middle ground would be to turn off signups for the given origin trial while allowing existing keys to continue working until the feature hits stable.

That said, a concern that I have with this proposal is that it might bias us towards strictly adding additional methods to APIs rather than changing the APIs in a breaking way, even if we learn that some other spelling of an API would be a better fit for its users. Given that folks signing up for origin trial are explicitly signing up for breakage and change, I wonder whether it might be reasonable to widen the proposal to include shifting straight into breaking changes without pause for existing OT users That entails some risk, insofar as existing OT users would need to update their code to account for the change. I'm not sure how to weigh that risk against the annoyance of a critical feature disappearing for a milestone between OT and shipping.

FWIW, what we heard from developers that signed up for OTs is that they are more than willing to take on breaking changes (as that puts cost on them), but less willing to account for a gap in support (which puts the cost on their users/customers).

So to me, that seems like the right tradeoff.

[Thomas Steiner]

On Thu, Aug 27, 2020 at 11:06 PM 'Yoav Weiss' via blink-api-owners-discuss <blink-api-ow...@chromium.org> wrote:

On Thu, Aug 27, 2020 at 9:43 PM Mike West <mk...@chromium.org> wrote:

Generally speaking, I'm ok with this direction. It seems reasonable to suggest that when we discover some missing method in an API through developers' usage and feedback that we should not punish developers for sharing their experience, but allow them to continue using the API along with its extensions. Perhaps a middle ground would be to turn off signups for the given origin trial while allowing existing keys to continue working until the feature hits stable.

That said, a concern that I have with this proposal is that it might bias us towards strictly adding additional methods to APIs rather than changing the APIs in a breaking way, even if we learn that some other spelling of an API would be a better fit for its users. Given that folks signing up for origin trial are explicitly signing up for breakage and change, I wonder whether it might be reasonable to widen the proposal to include shifting straight into breaking changes without pause for existing OT users That entails some risk, insofar as existing OT users would need to update their code to account for the change. I'm not sure how to weigh that risk against the annoyance of a critical feature disappearing for a milestone between OT and shipping.

FWIW, what we heard from developers that signed up for OTs is that they are more than willing to take on breaking changes (as that puts cost on them), but less willing to account for a gap in support (which puts the cost on their users/customers).

So to me, that seems like the right tradeoff.

Strong +1, this is very much what I hear from developers as well. Breakage is part of the OT contract, everyone aware of OT as a concept gets that.

[Marijn Kruisselbrink]

Not entirely related to this thread, but are these "requirements" for when a breakage period is or isn't required written down anywhere? The only thing I could find mentioning this breakage period at all is the warning at the end of https://web.dev/origin-trials/, but the stated rationale there seems to be the exact opposite of the start of this thread. I.e. web.dev says that the breakage period is needed to not bias toward avoiding breakages in the API surface, yet the original post in this thread says " if the shipping form of a feature is identical to the form it takes during an Origin Trial " which seems to very much bias towards avoiding breakages in the API surface. Am I misunderstanding either of these?

--

You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CALgRrLkV1pO71j17H%2BAqON6RZr3jCpoX5HUVtLgh_npG_b7V7Q%40mail.gmail.com.

[Marijn Kruisselbrink]

To add to that, per my original understanding of the purpose of the breakage period, cases where the API has changed in a breaking way would potentially be allowed to skip the breakage period, while APIs that haven't changed would require the period. But this thread sounds like it's the exact opposite of that now?

[Chris Harrelson]

On Tue, Sep 15, 2020 at 3:23 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

To add to that, per my original understanding of the purpose of the breakage period, cases where the API has changed in a breaking way would potentially be allowed to skip the breakage period, while APIs that haven't changed would require the period. But this thread sounds like it's the exact opposite of that now?

I hadn't considered the former, but the latter does not require a break anymore as long as the API does not change at all in its shipped form. We already allowed that in at least one intent.

On Tue, Sep 15, 2020 at 3:21 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

Not entirely related to this thread, but are these "requirements" for when a breakage period is or isn't required written down anywhere? The only thing I could find mentioning this breakage period at all is the warning at the end of https://web.dev/origin-trials/, but the stated rationale there seems to be the exact opposite of the start of this thread. I.e. web.dev says that the breakage period is needed to not bias toward avoiding breakages in the API surface, yet the original post in this thread says " if the shipping form of a feature is identical to the form it takes during an Origin Trial " which seems to very much bias towards avoiding breakages in the API surface. Am I misunderstanding either of these?

By bias, are you saying the API owner policy is over-optimized for one scenario, or are you saying there are perverse incentives for developers?

I think we did document the change about allowing no breaking period if absolutely nothing changed in the API, but perhaps I am wrong.

[Marijn Kruisselbrink]

On Tue, Sep 15, 2020 at 4:22 PM Chris Harrelson <chri...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 3:23 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

To add to that, per my original understanding of the purpose of the breakage period, cases where the API has changed in a breaking way would potentially be allowed to skip the breakage period, while APIs that haven't changed would require the period. But this thread sounds like it's the exact opposite of that now?

I hadn't considered the former, but the latter does not require a break anymore as long as the API does not change at all in its shipped form. We already allowed that in at least one intent.

Any thoughts about the former? Specifically for the case of the native file system API where the API changed significantly between the OT and shipping to stable, yet we have partners asking if we can extend the origin trial till after the stable release (currently the OT expires one week before the planned stable release). I'm not sure what to answer since I can't actually find any documentation about the breakage period, and what I thought I understood about the reason for it appears to not be correct.

On Tue, Sep 15, 2020 at 3:21 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

Not entirely related to this thread, but are these "requirements" for when a breakage period is or isn't required written down anywhere? The only thing I could find mentioning this breakage period at all is the warning at the end of https://web.dev/origin-trials/, but the stated rationale there seems to be the exact opposite of the start of this thread. I.e. web.dev says that the breakage period is needed to not bias toward avoiding breakages in the API surface, yet the original post in this thread says " if the shipping form of a feature is identical to the form it takes during an Origin Trial " which seems to very much bias towards avoiding breakages in the API surface. Am I misunderstanding either of these?

By bias, are you saying the API owner policy is over-optimized for one scenario, or are you saying there are perverse incentives for developers? 

That there are perverse incentives for feature developers to ship suboptimal API shapes to avoid the breakage period. 

 

I think we did document the change about allowing no breaking period if absolutely nothing changed in the API, but perhaps I am wrong.

I can't find any documentation about the breaking period either way, so I would be happy if you could point me to whatever documentation exists.

[Chris Harrelson]

On Tue, Sep 15, 2020 at 4:26 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 4:22 PM Chris Harrelson <chri...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 3:23 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

To add to that, per my original understanding of the purpose of the breakage period, cases where the API has changed in a breaking way would potentially be allowed to skip the breakage period, while APIs that haven't changed would require the period. But this thread sounds like it's the exact opposite of that now?

I hadn't considered the former, but the latter does not require a break anymore as long as the API does not change at all in its shipped form. We already allowed that in at least one intent.

Any thoughts about the former? Specifically for the case of the native file system API where the API changed significantly between the OT and shipping to stable, yet we have partners asking if we can extend the origin trial till after the stable release (currently the OT expires one week before the planned stable release). I'm not sure what to answer since I can't actually find any documentation about the breakage period, and what I thought I understood about the reason for it appears to not be correct.

You raise a good point. The purpose of the process is to prevent burn in of an API that is not shipped that prevents us from changing it... other owners, WDYT?
 

On Tue, Sep 15, 2020 at 3:21 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

Not entirely related to this thread, but are these "requirements" for when a breakage period is or isn't required written down anywhere? The only thing I could find mentioning this breakage period at all is the warning at the end of https://web.dev/origin-trials/, but the stated rationale there seems to be the exact opposite of the start of this thread. I.e. web.dev says that the breakage period is needed to not bias toward avoiding breakages in the API surface, yet the original post in this thread says " if the shipping form of a feature is identical to the form it takes during an Origin Trial " which seems to very much bias towards avoiding breakages in the API surface. Am I misunderstanding either of these?

By bias, are you saying the API owner policy is over-optimized for one scenario, or are you saying there are perverse incentives for developers? 

That there are perverse incentives for feature developers to ship suboptimal API shapes to avoid the breakage period. 

 

I think we did document the change about allowing no breaking period if absolutely nothing changed in the API, but perhaps I am wrong.

I can't find any documentation about the breaking period either way, so I would be happy if you could point me to whatever documentation exists.

Right now there is a required breaking period unless the API is completely unchanged. We agreed in response to an earlier state of this thread to allow additional exceptions on a case-by-case basis, but the intent author has to explicitly ask for it and say why. If you want to you can ask for it in your case on blink-dev and supply reasoning. Otherwise I am happy to bring it up at the next API owners meeting this Thursday.

[Marijn Kruisselbrink]

On Tue, Sep 15, 2020 at 4:30 PM Chris Harrelson <chri...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 4:26 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 4:22 PM Chris Harrelson <chri...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 3:23 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

To add to that, per my original understanding of the purpose of the breakage period, cases where the API has changed in a breaking way would potentially be allowed to skip the breakage period, while APIs that haven't changed would require the period. But this thread sounds like it's the exact opposite of that now?

I hadn't considered the former, but the latter does not require a break anymore as long as the API does not change at all in its shipped form. We already allowed that in at least one intent.

Any thoughts about the former? Specifically for the case of the native file system API where the API changed significantly between the OT and shipping to stable, yet we have partners asking if we can extend the origin trial till after the stable release (currently the OT expires one week before the planned stable release). I'm not sure what to answer since I can't actually find any documentation about the breakage period, and what I thought I understood about the reason for it appears to not be correct.

You raise a good point. The purpose of the process is to prevent burn in of an API that is not shipped that prevents us from changing it... other owners, WDYT?

That sounds like the case where a API hasn't changed is exactly the case where skipping the breakage period should not be allowed... Changing an API significantly seems like a much stronger way of preventing burn in rather than having this breakage period (requiring breaking API changes was actually one of the options we originally discussed as an alternative for this breaking period when we designed the origin trial framework/process way back).

 

On Tue, Sep 15, 2020 at 3:21 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

Not entirely related to this thread, but are these "requirements" for when a breakage period is or isn't required written down anywhere? The only thing I could find mentioning this breakage period at all is the warning at the end of https://web.dev/origin-trials/, but the stated rationale there seems to be the exact opposite of the start of this thread. I.e. web.dev says that the breakage period is needed to not bias toward avoiding breakages in the API surface, yet the original post in this thread says " if the shipping form of a feature is identical to the form it takes during an Origin Trial " which seems to very much bias towards avoiding breakages in the API surface. Am I misunderstanding either of these?

By bias, are you saying the API owner policy is over-optimized for one scenario, or are you saying there are perverse incentives for developers? 

That there are perverse incentives for feature developers to ship suboptimal API shapes to avoid the breakage period. 

 

I think we did document the change about allowing no breaking period if absolutely nothing changed in the API, but perhaps I am wrong.

I can't find any documentation about the breaking period either way, so I would be happy if you could point me to whatever documentation exists.

Right now there is a required breaking period unless the API is completely unchanged. We agreed in response to an earlier state of this thread to allow additional exceptions on a case-by-case basis, but the intent author has to explicitly ask for it and say why. If you want to you can ask for it in your case on blink-dev and supply reasoning. Otherwise I am happy to bring it up at the next API owners meeting this Thursday.

So I'd just ask this in the intent-to-ship thread?

[Chris Harrelson]

On Tue, Sep 15, 2020 at 4:56 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 4:30 PM Chris Harrelson <chri...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 4:26 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 4:22 PM Chris Harrelson <chri...@chromium.org> wrote:

On Tue, Sep 15, 2020 at 3:23 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

To add to that, per my original understanding of the purpose of the breakage period, cases where the API has changed in a breaking way would potentially be allowed to skip the breakage period, while APIs that haven't changed would require the period. But this thread sounds like it's the exact opposite of that now?

I hadn't considered the former, but the latter does not require a break anymore as long as the API does not change at all in its shipped form. We already allowed that in at least one intent.

Any thoughts about the former? Specifically for the case of the native file system API where the API changed significantly between the OT and shipping to stable, yet we have partners asking if we can extend the origin trial till after the stable release (currently the OT expires one week before the planned stable release). I'm not sure what to answer since I can't actually find any documentation about the breakage period, and what I thought I understood about the reason for it appears to not be correct.

You raise a good point. The purpose of the process is to prevent burn in of an API that is not shipped that prevents us from changing it... other owners, WDYT?

That sounds like the case where a API hasn't changed is exactly the case where skipping the breakage period should not be allowed... Changing an API significantly seems like a much stronger way of preventing burn in rather than having this breakage period (requiring breaking API changes was actually one of the options we originally discussed as an alternative for this breaking period when we designed the origin trial framework/process way back).

The rationale for the current policy goes as follows: it penalizes partners (in the form of causing work or interruption for them) who have invested in the origin trial heavily, for no clearly articulated reason.

If there are good reasons to skip the breaking period regardless, maybe we should get rid of it, but it is good to have a speed bump clearly differentiating between experiences and shipping features. We don't want intent-to-experiment to be an early launch vehicle.

 

 

On Tue, Sep 15, 2020 at 3:21 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

Not entirely related to this thread, but are these "requirements" for when a breakage period is or isn't required written down anywhere? The only thing I could find mentioning this breakage period at all is the warning at the end of https://web.dev/origin-trials/, but the stated rationale there seems to be the exact opposite of the start of this thread. I.e. web.dev says that the breakage period is needed to not bias toward avoiding breakages in the API surface, yet the original post in this thread says " if the shipping form of a feature is identical to the form it takes during an Origin Trial " which seems to very much bias towards avoiding breakages in the API surface. Am I misunderstanding either of these?

By bias, are you saying the API owner policy is over-optimized for one scenario, or are you saying there are perverse incentives for developers? 

That there are perverse incentives for feature developers to ship suboptimal API shapes to avoid the breakage period. 

 

I think we did document the change about allowing no breaking period if absolutely nothing changed in the API, but perhaps I am wrong.

I can't find any documentation about the breaking period either way, so I would be happy if you could point me to whatever documentation exists.

Right now there is a required breaking period unless the API is completely unchanged. We agreed in response to an earlier state of this thread to allow additional exceptions on a case-by-case basis, but the intent author has to explicitly ask for it and say why. If you want to you can ask for it in your case on blink-dev and supply reasoning. Otherwise I am happy to bring it up at the next API owners meeting this Thursday.

So I'd just ask this in the intent-to-ship thread?

Yep, exactly.

[Alex Russell]

Hey all,

Sorry for the delay in posting this; some hijinks with a lost draft. Anyhow, we debated this topic at some length in yesterday's API OWNERS meeting and agreed a modification of our policy for cases like these. I've tried to capture the agreement and background discussion below. If others agree here it represents the consensus we arrived at, I'll also post to blink-dev with an announcement.

What's Changing

• Feature developers may request "gapless" continuation for existing OT tokens as a part of an I2S
• To be approved for a gapless I2S, the API OWNERS will look for strong evidence from Origin Trial feedback (as referenced from the I2E thread or provided directly in the I2S) that developers are not requesting potentially breaking changes and are extraordinarily supportive of the feature launching as-is. Feedback from OT surveys, public blog posts, or testimonials from external developers in the thread about their experience with the API will all be considered.
• The API OWNERS reserve the right to LGTM an intent to ship without approving a gapless extension. Teams with a history that indicates a failure to address developer feedback should expect a more difficult gapless I2S process and may be asked to present their case to the OWNERS in one of our weekly meetings.

We hope to review rates of gapless vs. gapfull I2S submissions as a fraction of total OTs on a regular basis and expect that adjustments to this policy may be forthcoming as a result.

Background and Considerations

Origin Trials were designed to enable Chrome to avoid most of the worst "premature compatibility" hazards of previous systems of API experimentation on the web. The philosophy behind the design has been cautious, having seen other systems fail to contain "early" features and with long experience of "go fever" in teams that want to ship and may be willing to (unintentionally) fail to take critical feedback seriously when already convinced of the utility and soundness of their design.

The OT mechanism adopted a basket of conservative policies to constrain the potential reach of experimental features, including:

• Limits on overall use
• Constraints on consecutive trial renewals
• A "cooling off" period between trial and shipping to keep all parties in the mind of experimentation

Over the years we have tuned various aspects of this, tightening policy in some areas (e.g., renewal limits) and loosening in others (usage limits). We have reason to believe that the policy is working well today for early design iteration. Many features use OTs and some (including the trial in question) have run multiple trials with breaking changes in APIs between them to validate various aspects of a design.

It has always been our intent to prevent "soft launch" expectations from growing around OTs. If Chromium contributors and web developers come to depend on, or have a default expectation of, non-breakage between OT iterations -- or between OT and the final shipped APIs -- our goal to create a space that encourages Chromium feature developers to honestly evaluate feedback and iterate aggressively will be in jeopardy. This concern is what has given rise to the cooling-off period gap between successful OTs and stable launches.

In creating an (exceptional and more difficult) process for gapless shipping, we explicitly acknowledge that such a process raises the risk of OTs straying from their intended purposes and becoming a "soft launch" mechanism. Consequently, this policy change is explicitly conditional on reputation (Chromium is a community with social norms and we value our long-term reputations) and may be revised in future. We trust web developers to best represent their own experiences of in-development features, and so we also explicitly call out their own-voiced feedback as important in the requests for gapless shipping.

Lastly, the API OWNERS will continue to practice a recusal policy for LGTMing features they contribute to directly or manage, and we continue to ask the entire community -- web developers included -- for their feedback and perspectives on how these processes are working.

Thanks, and stay safe.

[Jason Chase]

Now that we have features following this revised process, I realized a potential issue that I missed in my earlier read.

The gapless process explicitly calls out "existing tokens" for continuation. I interpret that to mean that we should block new registrations for the trial, once the continuation has been approved?

If so, that's not currently supported by the registration/renewal functionality in the OT console. Origin trial tokens are write-once - set to expire no later than the trial end date when issued, and cannot be updated. Any extension to a trial will require developers to renew, and be issued new tokens. Currently, the OT console has the ability to disable a trial, blocking tokens from being issued. However, it does not distinguish between new registrations and renewal of existing registrations.

It would be straightforward to add functionality to the OT console limit token issuance to renewals/existing registrations. But, I'm wondering about the rationale behind the limitation?  Is there a specific risk to new developers joining the trial, given the feature is approved to ship shortly? Is it to prevent "soft launch" expectations from growing? I can imagine edge cases around such a limitation that might be counter to the intent. For example, new developers signup for trial when the I2S is sent, before the trial is changed in the OT console, skirting the limitation. Or conversely, an already registered partner needs to register a new origin after I2S, due some change in serving infrastructure.

Thanks,

Jason

[Chris Harrelson]

On Fri, Sep 25, 2020 at 8:15 AM Jason Chase <cha...@chromium.org> wrote:

Now that we have features following this revised process, I realized a potential issue that I missed in my earlier read.

The gapless process explicitly calls out "existing tokens" for continuation. I interpret that to mean that we should block new registrations for the trial, once the continuation has been approved?

If so, that's not currently supported by the registration/renewal functionality in the OT console. Origin trial tokens are write-once - set to expire no later than the trial end date when issued, and cannot be updated. Any extension to a trial will require developers to renew, and be issued new tokens. Currently, the OT console has the ability to disable a trial, blocking tokens from being issued. However, it does not distinguish between new registrations and renewal of existing registrations.

I thought we were planning to decouple the token itself from its expiry, so that sites don't need to re-deploy to get a refreshed token and instead just need to "ask to extend" on our dashboard. If we do that work would it solve this issue also?

 

It would be straightforward to add functionality to the OT console limit token issuance to renewals/existing registrations. But, I'm wondering about the rationale behind the limitation?  Is there a specific risk to new developers joining the trial, given the feature is approved to ship shortly? Is it to prevent "soft launch" expectations from growing? I can imagine edge cases around such a limitation that might be counter to the intent. For example, new developers signup for trial when the I2S is sent, before the trial is changed in the OT console, skirting the limitation. Or conversely, an already registered partner needs to register a new origin after I2S, due some change in serving infrastructure.

I think the rationale is to limit the scope of a "soft launch" yes. If it takes a little while to upgrade OT tech to match I don't think that is the end of the world. 

 

On Friday, September 18, 2020 at 9:36:48 PM UTC-4 Alex Russell wrote

• To be approved for a gapless I2S, the API OWNERS will look for strong evidence from Origin Trial feedback (as referenced from the I2E thread or provided directly in the I2S) that developers are not requesting potentially breaking changes and are extraordinarily supportive of the feature launching as-is. Feedback from OT surveys, public blog posts, or testimonials from external developers in the thread about their experience with the API will all be considered

I don't think this is quite what we agreed. An OT that has a breaking change can still request gapless; the key is only to show evidence of OT partner engagement and feedback. Other API owners, WDYT?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/cef83caf-5348-421b-aea9-1767183440bdn%40chromium.org.

[Jason Chase]

On Friday, September 25, 2020 at 12:31:37 PM UTC-4 Chris Harrelson wrote:

On Fri, Sep 25, 2020 at 8:15 AM Jason Chase <cha...@chromium.org> wrote:

Now that we have features following this revised process, I realized a potential issue that I missed in my earlier read.

The gapless process explicitly calls out "existing tokens" for continuation. I interpret that to mean that we should block new registrations for the trial, once the continuation has been approved?

If so, that's not currently supported by the registration/renewal functionality in the OT console. Origin trial tokens are write-once - set to expire no later than the trial end date when issued, and cannot be updated. Any extension to a trial will require developers to renew, and be issued new tokens. Currently, the OT console has the ability to disable a trial, blocking tokens from being issued. However, it does not distinguish between new registrations and renewal of existing registrations.

I thought we were planning to decouple the token itself from its expiry, so that sites don't need to re-deploy to get a refreshed token and instead just need to "ask to extend" on our dashboard. If we do that work would it solve this issue also?

Not exactly, the plan is to decouple token expiry from the renewal feedback requirement, so new tokens aren't required every 6 weeks. The plan is that tokens can last the length of the trial, but still expire on the end date. That preserves the limited duration property of trials, without needing to change the client-side infrastructure in Chromium. (We have an internal design doc with details, I'll follow up on getting that shared publicly). As planned, it would not address the issue. 

We could explore alternatives that would remove the need to issue new tokens for a gapless extension. However, if the goal is to limit the "soft launch" aspect, it seems reasonable to keep the extra friction of requiring developers to get/deploy new tokens to cover the period until the feature ships. That friction is once per trial, and I believe we expect/hope that gapless extensions are the exception, not the rule.

 

It would be straightforward to add functionality to the OT console limit token issuance to renewals/existing registrations. But, I'm wondering about the rationale behind the limitation?  Is there a specific risk to new developers joining the trial, given the feature is approved to ship shortly? Is it to prevent "soft launch" expectations from growing? I can imagine edge cases around such a limitation that might be counter to the intent. For example, new developers signup for trial when the I2S is sent, before the trial is changed in the OT console, skirting the limitation. Or conversely, an already registered partner needs to register a new origin after I2S, due some change in serving infrastructure.

I think the rationale is to limit the scope of a "soft launch" yes. If it takes a little while to upgrade OT tech to match I don't think that is the end of the world. 

Ok, sounds good. I'll come up with a plan for the OT console to handle gapless extensions, and limit to existing registrations.

[Chris Harrelson]

On Fri, Sep 25, 2020 at 12:55 PM Jason Chase <cha...@chromium.org> wrote:

On Friday, September 25, 2020 at 12:31:37 PM UTC-4 Chris Harrelson wrote:

On Fri, Sep 25, 2020 at 8:15 AM Jason Chase <cha...@chromium.org> wrote:

Now that we have features following this revised process, I realized a potential issue that I missed in my earlier read.

The gapless process explicitly calls out "existing tokens" for continuation. I interpret that to mean that we should block new registrations for the trial, once the continuation has been approved?

If so, that's not currently supported by the registration/renewal functionality in the OT console. Origin trial tokens are write-once - set to expire no later than the trial end date when issued, and cannot be updated. Any extension to a trial will require developers to renew, and be issued new tokens. Currently, the OT console has the ability to disable a trial, blocking tokens from being issued. However, it does not distinguish between new registrations and renewal of existing registrations.

I thought we were planning to decouple the token itself from its expiry, so that sites don't need to re-deploy to get a refreshed token and instead just need to "ask to extend" on our dashboard. If we do that work would it solve this issue also?

Not exactly, the plan is to decouple token expiry from the renewal feedback requirement, so new tokens aren't required every 6 weeks. The plan is that tokens can last the length of the trial, but still expire on the end date. That preserves the limited duration property of trials, without needing to change the client-side infrastructure in Chromium. (We have an internal design doc with details, I'll follow up on getting that shared publicly). As planned, it would not address the issue. 

We could explore alternatives that would remove the need to issue new tokens for a gapless extension. However, if the goal is to limit the "soft launch" aspect, it seems reasonable to keep the extra friction of requiring developers to get/deploy new tokens to cover the period until the feature ships. That friction is once per trial, and I believe we expect/hope that gapless extensions are the exception, not the rule.

 

It would be straightforward to add functionality to the OT console limit token issuance to renewals/existing registrations. But, I'm wondering about the rationale behind the limitation?  Is there a specific risk to new developers joining the trial, given the feature is approved to ship shortly? Is it to prevent "soft launch" expectations from growing? I can imagine edge cases around such a limitation that might be counter to the intent. For example, new developers signup for trial when the I2S is sent, before the trial is changed in the OT console, skirting the limitation. Or conversely, an already registered partner needs to register a new origin after I2S, due some change in serving infrastructure.

I think the rationale is to limit the scope of a "soft launch" yes. If it takes a little while to upgrade OT tech to match I don't think that is the end of the world. 

Ok, sounds good. I'll come up with a plan for the OT console to handle gapless extensions, and limit to existing registrations.

Excellent thanks!

[Yoav Weiss]

On Fri, Sep 25, 2020 at 6:31 PM Chris Harrelson <chri...@chromium.org> wrote:

On Fri, Sep 25, 2020 at 8:15 AM Jason Chase <cha...@chromium.org> wrote:

Now that we have features following this revised process, I realized a potential issue that I missed in my earlier read.

The gapless process explicitly calls out "existing tokens" for continuation. I interpret that to mean that we should block new registrations for the trial, once the continuation has been approved?

If so, that's not currently supported by the registration/renewal functionality in the OT console. Origin trial tokens are write-once - set to expire no later than the trial end date when issued, and cannot be updated. Any extension to a trial will require developers to renew, and be issued new tokens. Currently, the OT console has the ability to disable a trial, blocking tokens from being issued. However, it does not distinguish between new registrations and renewal of existing registrations.

I thought we were planning to decouple the token itself from its expiry, so that sites don't need to re-deploy to get a refreshed token and instead just need to "ask to extend" on our dashboard. If we do that work would it solve this issue also?

 

It would be straightforward to add functionality to the OT console limit token issuance to renewals/existing registrations. But, I'm wondering about the rationale behind the limitation?  Is there a specific risk to new developers joining the trial, given the feature is approved to ship shortly? Is it to prevent "soft launch" expectations from growing? I can imagine edge cases around such a limitation that might be counter to the intent. For example, new developers signup for trial when the I2S is sent, before the trial is changed in the OT console, skirting the limitation. Or conversely, an already registered partner needs to register a new origin after I2S, due some change in serving infrastructure.

I think the rationale is to limit the scope of a "soft launch" yes. If it takes a little while to upgrade OT tech to match I don't think that is the end of the world. 

 

On Friday, September 18, 2020 at 9:36:48 PM UTC-4 Alex Russell wrote

• To be approved for a gapless I2S, the API OWNERS will look for strong evidence from Origin Trial feedback (as referenced from the I2E thread or provided directly in the I2S) that developers are not requesting potentially breaking changes and are extraordinarily supportive of the feature launching as-is. Feedback from OT surveys, public blog posts, or testimonials from external developers in the thread about their experience with the API will all be considered

I don't think this is quite what we agreed. An OT that has a breaking change can still request gapless; the key is only to show evidence of OT partner engagement and feedback. Other API owners, WDYT?

That's also how I understood it. We want to enable gapless OT also in the case of breaking changes, to avoid incentivizing shipping without addressing feedback.

[Chris Harrelson]

I added text here: https://sites.google.com/a/chromium.org/dev/blink/launching-features

"Origin Trial and launch, you may request one. The process to do so is to include this request in your Intent to Ship email. In the request, you must show clear evidence that developers engaged with the Origin Trial and that their concerns were taken into account in the final API design and implementation. LGTMs for the Intent to Ship imply approval of the request."

I'll email blink-dev on Monday unless there is a concern with any of this text.