Process for different types of trials?

[Jason Chase]

Hi API Owners,

We have a formal intent process for origin trials, where "origin trial" = "experiment with a new feature".

We've started to take advantage of the origin trial infrastructure for different types of trials. This includes:

- "deprecation trial": specifically, Web Components V0 where the feature will be removed, but the trial allows continued use post-removal in Chrome.

- "opt-in/opt-out trial": specifically, tab freezing based on heuristics (discussion). Developers can opt-out if the freezing breaks their site, or opt-in to test in advance.

I'd like to clarify what process should be used to review/approve/extend these different types of trials. For example, for Web Components V0, I'd say the trial was implicitly approved as part of the I2D.

I don't think we should use the existing origin trial process as-is, because it's meant to safeguard against potential issues that seem unique to adding experimental features. In particular, to ensure we aren't bypassing the launch process to early/defacto ship a feature. 

On the other hand, I think we need oversight from API owners for these different types of trials, with similarities in the process. I'd expect these different types to happen less frequently than origin trials. Given that, it seems pointless to spend much effort coming up with standalone process for each type.

My strawman proposal:

- Document what makes these trials special (where principles/decisions are different from origin trials) in some 

- Use the existing I2E templates to start or extend all types of trials, not just origin trials

  - Requestors can "N/A" any sections that only apply to origin trials

  - Requestor manually call out the different type, e.g. in the summary "Note: this is a deprecation trial ..."

  - NOTE: I haven't checked what this means for the plan to move all intents to chromestatus.com

- API owners review as usual (i.e. 1 LGTM is sufficient)

- If volumes/confusion increase significantly, then consider making changes to the intent templates.

Thoughts?

Thanks,

Jason

[Chris Harrelson]

The plan sounds fine to me too.

On Thu, Jul 18, 2019 at 7:52 AM 'Joe Medley' via blink-api-owners-discuss <blink-api-ow...@chromium.org> wrote:

I'll take on what needs to be done in Chrome Status. Separate labeling is critical to external communication. As it is, we need to start with "It's an origin trial, but it's not" and throw a bunch of words to explain what it really means. That's a recipe for misunderstanding and confusion. These same labels and definitions need to be in our beta release posts and other communications. 

I have problems with the terms we have so far. 'Deprecation trial' sounds like something we might change our minds about. That is certainly not true of Web Components v0. 'Opt-in/opt-out trial' at least works. The phrase is clunky. I think we can do better.

Unfortunately, I don't have any ideas for either.

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

On Thu, Jul 18, 2019 at 12:48 AM 'Yoav Weiss' via blink-api-owners-discuss <blink-api-ow...@chromium.org> wrote:

Given that these intents count as simple I2Es today, your plan seems like an incremental improvement on that. Considering chromestatus.com involvement (e.g. updating the templates there) also makes sense.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAMi8Y5XdcUw4dVyA5bDykyNwh%3DQjmgco7q7v_M5HyA5%2Bwqc0GA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfXVYAVyTYQFJsyunPxDPtQa6iF7WtagL6CYM38%2BXDqE3A%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAJUhtG-6sSnJor3Ych0mFLz3YgqgfCuUnPLaX_eao%2BV60YBemQ%40mail.gmail.com.

[Mason Freed]

I know this is using the origin trial infrastructure, but no need to keep "trial" in the name. How about "Deprecation Extension"? That sounds pretty straightforward and probably doesn't need much/any further explanation.

[Mason Freed]

Hmm, I guess nothing is truly clear. The way I'm using the term (and the system) is that the deprecation is being extended. We have deprecated (warned against future use) the feature, and we will continue to (just) deprecate it, and not actually completely remove it. I suppose something like "Removal Reprieve" would be a more correct way to say that?

On Fri, Jul 19, 2019 at 10:44 AM Joe Medley <jme...@google.com> wrote:

With respect, I don't think it's as clear as you suppose. I think you mean that the feature is going to be removed but it's being deprecated for a certain extension of its lifetime. I could, if I'm an external engineer who does follow what we do closely, take it to mean that the deprecation started in some previous period and is now being extended.

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

[Mason Freed]

Forgive me if I'm missing something, but isn't that precisely what it's doing?

On Mon, Jul 22, 2019 at 7:15 AM Joe Medley <jme...@google.com> wrote:

So you really don't think it's sounds like it's extending a deprecation that started previously? I guarantee we'll spend the next two years explaining it and reexplaining it.

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

On Sun, Jul 21, 2019 at 10:17 AM Yoav Weiss <yoav...@google.com> wrote:

"deprecation extension" sounds pretty clear to me...

On Fri, Jul 19, 2019, 15:26 Joe Medley <jme...@google.com> wrote:

Well, I'm trying to think beyond your use case to any deprecation flag that uses the OT mechanism.

[Ian Clelland]

Deprecation is the period where we say we're going to remove it, but haven't removed it yet. With the origin trial token, you can extend that period, for your site, for a limited time. So, it *is* extending the deprecation period, which started at the point we announced the upcoming removal.

But I'm looking at it with the eyes of a browser developer, not a web developer, and we should word this for the intended audience.

On Mon, Jul 22, 2019 at 10:15 AM 'Joe Medley' via blink-api-owners-discuss <blink-api-ow...@chromium.org> wrote:

So you really don't think it's sounds like it's extending a deprecation that started previously? I guarantee we'll spend the next two years explaining it and reexplaining it.

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

On Sun, Jul 21, 2019 at 10:17 AM Yoav Weiss <yoav...@google.com> wrote:

"deprecation extension" sounds pretty clear to me...

On Fri, Jul 19, 2019, 15:26 Joe Medley <jme...@google.com> wrote:

Well, I'm trying to think beyond your use case to any deprecation flag that uses the OT mechanism.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAJUhtG9oUzeQPJc5c38_3Aiy3TJnYeYHuRQCiFFaO4jk%3Dd%2BU_A%40mail.gmail.com.

[Yoav Weiss]

Do we have multiple competing proposals for this? If so, maybe we can run a survey with the intended audience (e.g. by tweeting it out by developer-followed accounts) and see which one they found least confusing.

You received this message because you are subscribed to the Google Groups "experimentation-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to experimentation...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/experimentation-dev/CAK_TSX%2B_-Vhvcm%3Dz7q%2B8QicmeA58YyNAcD--XgRT4gZCu8wVZg%40mail.gmail.com.

[cha...@chromium.org]

There seems to be a few different discussions going on here, beyond the original question. I'll try to summarize.

1) What should the process be for different kinds of trials?

My proposal was roughly to use the existing process for origin trials, with tweaks to intent templates. There seems to be consensus on this plan, based on comments from Yoav and Chris, and lack of explicit objection. I'll take an action item to produce/update guidance for other types of trials (i.e. opt-in/opt-out, deprecations, etc.)

2) Integration of trial(s) process with chromestatus.com

Again, seems to be consensus that this is the right thing to do. Joe has volunteered to own this. I'll followup with Joe separately to figure out how to proceed.

3) Naming of different types of trials

Naming is hard, and I don't think we're near consensus yet. I agree with Joe's comment that this is critical for external communication. However, I don't think resolving the naming should block the short-term plan of using the existing intent process. I suggest that naming could be addressed as part of the integration with chromestatus.com

Please chime in with any concerns or inaccuracy with this summary or next steps. Otherwise, I would consider this thread resolved, and we can start new threads as needed.

Thanks,

Jason

On Monday, July 22, 2019 at 2:04:30 PM UTC-4, Yoav Weiss wrote:

Do we have multiple competing proposals for this? If so, maybe we can run a survey with the intended audience (e.g. by tweeting it out by developer-followed accounts) and see which one they found least confusing.

On Mon, Jul 22, 2019 at 11:43 AM Ian Clelland <icle...@chromium.org> wrote:

Deprecation is the period where we say we're going to remove it, but haven't removed it yet. With the origin trial token, you can extend that period, for your site, for a limited time. So, it *is* extending the deprecation period, which started at the point we announced the upcoming removal.

But I'm looking at it with the eyes of a browser developer, not a web developer, and we should word this for the intended audience.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAMi8Y5XdcUw4dVyA5bDykyNwh%3DQjmgco7q7v_M5HyA5%2Bwqc0GA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfXVYAVyTYQFJsyunPxDPtQa6iF7WtagL6CYM38%2BXDqE3A%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAJUhtG-6sSnJor3Ych0mFLz3YgqgfCuUnPLaX_eao%2BV60YBemQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAJUhtG9oUzeQPJc5c38_3Aiy3TJnYeYHuRQCiFFaO4jk%3Dd%2BU_A%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "experimentation-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to experimentation-dev+unsub...@chromium.org.