Hi,I have several changes that I'd like us to make to the policy for Origin Trials, to account for the existence and mitigate the risks of long-running OTs.1. Remove the requirement to ask for a gapless Origin TrialWhy: experience so far has shown that there isn't really a risk to letting an OT proceed to launch in a gapless way. This reduces the bureaucratic steps to get to shipping, without any significant new risk.
2. Any Origin Trial may request up to N milestones with the current policy (i.e. ~ no spec required, no TAG review required; only need an explainer and description of how the API works).I propose N = 6, which is a bit less than 6 months.Why: I don't think we have a hard and fast rule for this at present, this just formalizes our current best practice.
3. Extensions of the Origin Trial beyond N can only request up to M at a time. Approval of the first extension is blocked on evidence of the following:
* A draft spec (early draft is ok, but must be spec-like and associated with the appropriate standardization venue, or WICG)
* A TAG review in progress
* bit.ly/blink-signals requests
* Substantial evidence of outreach for feedback from the community
* Progress on WPT testsWhy: these additional artifacts demonstrate progress towards meeting the bar for an I2S, which substantially decreases the risk of the OT running too long.I propose M=3.4. Approval to go to N+2M or beyond requires "substantial additional progress" on the item 3 bullet points. (API owners should try to proactively provide guidance on what that should be on a case-by-case basis during approval of the previous extension, to make this less vague.)Why: progress towards I2S should be sustained throughout the OT.Feedback?Chris
--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CA%2BN6QZvOAiFKfCDhVRJyuVozXwFphandzQadF6syodG6iapNGg%40mail.gmail.com.
On Thu, Mar 24, 2022 at 8:53 PM Chris Harrelson <chri...@chromium.org> wrote:Hi,I have several changes that I'd like us to make to the policy for Origin Trials, to account for the existence and mitigate the risks of long-running OTs.1. Remove the requirement to ask for a gapless Origin TrialWhy: experience so far has shown that there isn't really a risk to letting an OT proceed to launch in a gapless way. This reduces the bureaucratic steps to get to shipping, without any significant new risk.I am hesitant about this change. I think the bar for a gapless origin trial should perhaps be higher, but more automatic. In particular, it should require demonstrating that there are no worthwhile backward-incompatible changes on the table, that might be stifled by the desire to be gapless. And if you can demonstrate that, then gapless shipping should be automatically granted by the API owners.Specifically, I think the incentive at the conclusion of an origin trial should be to make changes in response to feedback, and not to dismiss or resist feedback because it would require backward-incompatible changes that would break the gaplessness.One way to implement this would be to require that gapless requests come with an audit of all open spec issues, categorized into "addressing this will require backward-incompatible changes, but we have decided not to address it and so are now closing the issue" vs. "we can address this in the future without compat concerns". If such an audit is provided, then the request should be approved automatically by the API owners.Although in theory such an audit should be done for all Intent to Ships, I think it makes special sense for gapless OT requests. It basically uses the carrot of gaplessness to incentivize proper respect for the compat value of the Blink project. Given how gapless origin trials are naturally in tension with our compat value, via how they can be used as an early-shipping mechanism, this seems like a good tradeoff to make.
Hi Domenic,On Fri, Mar 25, 2022 at 8:52 AM Domenic Denicola <dom...@chromium.org> wrote:On Thu, Mar 24, 2022 at 8:53 PM Chris Harrelson <chri...@chromium.org> wrote:Hi,I have several changes that I'd like us to make to the policy for Origin Trials, to account for the existence and mitigate the risks of long-running OTs.1. Remove the requirement to ask for a gapless Origin TrialWhy: experience so far has shown that there isn't really a risk to letting an OT proceed to launch in a gapless way. This reduces the bureaucratic steps to get to shipping, without any significant new risk.I am hesitant about this change. I think the bar for a gapless origin trial should perhaps be higher, but more automatic. In particular, it should require demonstrating that there are no worthwhile backward-incompatible changes on the table, that might be stifled by the desire to be gapless. And if you can demonstrate that, then gapless shipping should be automatically granted by the API owners.Specifically, I think the incentive at the conclusion of an origin trial should be to make changes in response to feedback, and not to dismiss or resist feedback because it would require backward-incompatible changes that would break the gaplessness.One way to implement this would be to require that gapless requests come with an audit of all open spec issues, categorized into "addressing this will require backward-incompatible changes, but we have decided not to address it and so are now closing the issue" vs. "we can address this in the future without compat concerns". If such an audit is provided, then the request should be approved automatically by the API owners.Although in theory such an audit should be done for all Intent to Ships, I think it makes special sense for gapless OT requests. It basically uses the carrot of gaplessness to incentivize proper respect for the compat value of the Blink project. Given how gapless origin trials are naturally in tension with our compat value, via how they can be used as an early-shipping mechanism, this seems like a good tradeoff to make.What you're suggesting sounds a lot like the current policy. Can you tell me more about how it differs? (And if it doesn't, that's fine; just asking to make sure I didn't miss an important aspect.) Also please share any examples to share from which we can learn.
Another question I have is what bad outcome you're worried about. Is it that the teams will be incented to avoid any breaking changes to APIs, so that they can achieve gapless?
I'm not convinced that the incentives will be substantially more than with the current policy (including the audits mentioned below), but I'm certainly open to learning more. When the original policy forbidding gapless was put in place, I thought it made sense, but experience has as far as I can tell shown that it wasn't a significant problem in practice, and instead mostly a cause of worry and extra box-checking work for teams. It could be that once we remove the existing controls, bad things will happen, but if so we can learn from it and adjust accordingly.Last point: the audit you mention is to some extent done already today, albeit in a less comprehensive/more distributed manner, in order to surface concerns in general (not just for the gapless question). In addition to a degree of due diligence by API owners, we rely on the existing feedback mechanisms, such as comments on the issue repo, TAG review, signals requests, and the blink-dev thread itself, to surface concerns. I think that in practice, each of these input channels does regularly surface concerns when they are present, and then we know about them and can fix them before shipping.
On Fri, Mar 25, 2022 at 5:51 PM Chris Harrelson <chri...@chromium.org> wrote:Hi Domenic,On Fri, Mar 25, 2022 at 8:52 AM Domenic Denicola <dom...@chromium.org> wrote:On Thu, Mar 24, 2022 at 8:53 PM Chris Harrelson <chri...@chromium.org> wrote:Hi,I have several changes that I'd like us to make to the policy for Origin Trials, to account for the existence and mitigate the risks of long-running OTs.1. Remove the requirement to ask for a gapless Origin TrialWhy: experience so far has shown that there isn't really a risk to letting an OT proceed to launch in a gapless way. This reduces the bureaucratic steps to get to shipping, without any significant new risk.I am hesitant about this change. I think the bar for a gapless origin trial should perhaps be higher, but more automatic. In particular, it should require demonstrating that there are no worthwhile backward-incompatible changes on the table, that might be stifled by the desire to be gapless. And if you can demonstrate that, then gapless shipping should be automatically granted by the API owners.Specifically, I think the incentive at the conclusion of an origin trial should be to make changes in response to feedback, and not to dismiss or resist feedback because it would require backward-incompatible changes that would break the gaplessness.One way to implement this would be to require that gapless requests come with an audit of all open spec issues, categorized into "addressing this will require backward-incompatible changes, but we have decided not to address it and so are now closing the issue" vs. "we can address this in the future without compat concerns". If such an audit is provided, then the request should be approved automatically by the API owners.Although in theory such an audit should be done for all Intent to Ships, I think it makes special sense for gapless OT requests. It basically uses the carrot of gaplessness to incentivize proper respect for the compat value of the Blink project. Given how gapless origin trials are naturally in tension with our compat value, via how they can be used as an early-shipping mechanism, this seems like a good tradeoff to make.What you're suggesting sounds a lot like the current policy. Can you tell me more about how it differs? (And if it doesn't, that's fine; just asking to make sure I didn't miss an important aspect.) Also please share any examples to share from which we can learn.I think I am stressing more the triaging of open issues, and less the developer feedback.As an example, https://groups.google.com/a/chromium.org/g/blink-dev/c/TAsiU_Da-9g/m/WUD33xjCAgAJ asked for a gapless extension. They have 9 open issues, including several from other implementers. Their I2S called out disagreement on #11, which is good. But it was unclear whether the others might cause compat problems if changed.https://groups.google.com/a/chromium.org/g/blink-dev/c/YoefXLTQsw0/m/jOu-wZKcAQAJ is another one, with a prominent naming issue remaining open. This saw some discussion on the thread via Philip's first reply, but it was notably not called out in the original I2S.https://groups.google.com/a/chromium.org/g/blink-dev/c/zOsGZGMGiM4/m/r9JpKgA4AgAJ is a good example as well. Note that most of its open issues were after the 2020-09-21 gapless request At the time, I chimed in as spec mentor to say that I thought the issues would not block interoperable implementation. I think it would have been better if I (as spec mentor) or the team had specifically considered the open issues through the compat lens; for example the proposals in #182 and #162 would be rather compat-impacting if they were to be adopted, so it would have been better if they had been closed, or at least discussed on the Intent to Ship.
On Mon, Mar 28, 2022 at 12:25 PM Domenic Denicola <dom...@chromium.org> wrote:On Fri, Mar 25, 2022 at 5:51 PM Chris Harrelson <chri...@chromium.org> wrote:Hi Domenic,On Fri, Mar 25, 2022 at 8:52 AM Domenic Denicola <dom...@chromium.org> wrote:On Thu, Mar 24, 2022 at 8:53 PM Chris Harrelson <chri...@chromium.org> wrote:Hi,I have several changes that I'd like us to make to the policy for Origin Trials, to account for the existence and mitigate the risks of long-running OTs.1. Remove the requirement to ask for a gapless Origin TrialWhy: experience so far has shown that there isn't really a risk to letting an OT proceed to launch in a gapless way. This reduces the bureaucratic steps to get to shipping, without any significant new risk.I am hesitant about this change. I think the bar for a gapless origin trial should perhaps be higher, but more automatic. In particular, it should require demonstrating that there are no worthwhile backward-incompatible changes on the table, that might be stifled by the desire to be gapless. And if you can demonstrate that, then gapless shipping should be automatically granted by the API owners.Specifically, I think the incentive at the conclusion of an origin trial should be to make changes in response to feedback, and not to dismiss or resist feedback because it would require backward-incompatible changes that would break the gaplessness.One way to implement this would be to require that gapless requests come with an audit of all open spec issues, categorized into "addressing this will require backward-incompatible changes, but we have decided not to address it and so are now closing the issue" vs. "we can address this in the future without compat concerns". If such an audit is provided, then the request should be approved automatically by the API owners.Although in theory such an audit should be done for all Intent to Ships, I think it makes special sense for gapless OT requests. It basically uses the carrot of gaplessness to incentivize proper respect for the compat value of the Blink project. Given how gapless origin trials are naturally in tension with our compat value, via how they can be used as an early-shipping mechanism, this seems like a good tradeoff to make.What you're suggesting sounds a lot like the current policy. Can you tell me more about how it differs? (And if it doesn't, that's fine; just asking to make sure I didn't miss an important aspect.) Also please share any examples to share from which we can learn.I think I am stressing more the triaging of open issues, and less the developer feedback.As an example, https://groups.google.com/a/chromium.org/g/blink-dev/c/TAsiU_Da-9g/m/WUD33xjCAgAJ asked for a gapless extension. They have 9 open issues, including several from other implementers. Their I2S called out disagreement on #11, which is good. But it was unclear whether the others might cause compat problems if changed.https://groups.google.com/a/chromium.org/g/blink-dev/c/YoefXLTQsw0/m/jOu-wZKcAQAJ is another one, with a prominent naming issue remaining open. This saw some discussion on the thread via Philip's first reply, but it was notably not called out in the original I2S.https://groups.google.com/a/chromium.org/g/blink-dev/c/zOsGZGMGiM4/m/r9JpKgA4AgAJ is a good example as well. Note that most of its open issues were after the 2020-09-21 gapless request At the time, I chimed in as spec mentor to say that I thought the issues would not block interoperable implementation. I think it would have been better if I (as spec mentor) or the team had specifically considered the open issues through the compat lens; for example the proposals in #182 and #162 would be rather compat-impacting if they were to be adopted, so it would have been better if they had been closed, or at least discussed on the Intent to Ship.The current process has the following question:Describe the degree of interoperability risk. For a new feature, the main risk is that it fails to become an interoperable part of the web platform if other browsers do not implement it. For a removal, please review our principles of web compatibility. Please include citation links below where possible. Examples include resolutions from relevant standards bodies (e.g. W3C working group), tracking bugs, or links to online conversations. Example.How about we modify it to also say:Please list known spec issues which may cause web compat risk if resolved (e.g. resulting in change to naming or structure of the API).
Another question I have is what bad outcome you're worried about. Is it that the teams will be incented to avoid any breaking changes to APIs, so that they can achieve gapless?Yes. I think this is easiest to see with name choices. We have a few recent threads with teams requesting gapless OTs and defending their current name choices. I'm sure those teams are arguing in good faith, and believe their current name choices to be the best! But I do wonder if the option of gaplessness were not on the table, if they would be more receptive to some of the naming feedback, because partners would have to update their code anyway to account for the gap.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CA%2BN6QZvjr4eJyfuG665owSqp%2BsabHRBMW%2BCSSTQEmXcXs9TSxg%40mail.gmail.com.
Another question I have is what bad outcome you're worried about. Is it that the teams will be incented to avoid any breaking changes to APIs, so that they can achieve gapless?Yes. I think this is easiest to see with name choices. We have a few recent threads with teams requesting gapless OTs and defending their current name choices. I'm sure those teams are arguing in good faith, and believe their current name choices to be the best! But I do wonder if the option of gaplessness were not on the table, if they would be more receptive to some of the naming feedback, because partners would have to update their code anyway to account for the gap.Domenic - I think that you're making an assumption here that e.g. a naming change is something that OT participants can't deal with in a gapless scenario, but I'm not sure this is true in most cases.It seems like participants need to feature detect the new API anyway (as it's not supported by other browsers, can go away, etc), and as part of that, could create a feature detection path for the new name (assuming they have enough head start), enabling feature teams to rename without breaking OT participants, even in a gapless scenario.
This LGTM, but a few clarifying questions:
For question 3, are we looking for evidence of progress on all 5 requirements? Merely most? Some?
For question 4, I thought I understood Domenic's concern, but the
way it's written here makes me unsure. I think we're trying to
understand if other vendors have given reasonable feedback in the
form of spec issues that should be addressed before shipping,
under the assumption that it would improve interop in the future,
but might affect compat w/ existing OT usage. Is that correct?
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfXvbu95-cRsB4ApEE422HVTBQdoZ%2BxwKtMn%3DYkzOEOr%3Dg%40mail.gmail.com.
This LGTM, but a few clarifying questions:
For question 3, are we looking for evidence of progress on all 5 requirements? Merely most? Some?
For question 4, I thought I understood Domenic's concern, but the way it's written here makes me unsure. I think we're trying to understand if other vendors have given reasonable feedback in the form of spec issues that should be addressed before shipping, under the assumption that it would improve interop in the future, but might affect compat w/ existing OT usage. Is that correct?
LGTM
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CA%2BN6QZuES1b35nv_3%3DBY8OOY2VT_KRRoGL%3D4aPgBPhKQNDvTPA%40mail.gmail.com.