Contact emails
cha...@chromium.org, owe...@chromium.org
The experimentation-dev group exists for general discussion.
Spec (in progress)
Explainer: https://github.com/jpchase/ExperimentalFramework/blob/master/explainer.md
A formal spec will be developed in the future, as the design stabilizes.
Summary
We’re ready to start prototyping the Experimental Framework, according to the design doc shared with blink-dev. This design is a work in progress, but in short our goal is to explore possible approaches for exposing experimental APIs in the browser, but only to specific/registered origins, for limited durations and subject to global usage caps.
Developer registration will be required to limit experiment access, but implementing the registration system is independent of the Chromium intent process. We haven’t started work on the registration system, other than some initial design. Currently, we expect to require developers sign up similarly to the way the Chrome Web Store works for extensions and app developers today. Additionally, developers will need to agree to provide feedback as the experiment progresses.
In addition, dk...@chromium.org is working on a proposal for updating the Blink process to handle experiments. The new process should not allow auto-renewing of experiments without explicit discussion. We anticipate that proposal will be shared with blink-dev in the near future.
Motivation
As explained in Doing Science On The Web, a better approach is needed for experimentation with new APIs. The ultimate goal is to enable experimentation and iteration on pre-standardization APIs, in a way that gathers meaningful feedback, but without burn-in of the APIs.
Compatibility Risk
From a web-facing API perspective, there is low risk, as very little of the framework is exposed to web developers. Specifically, the only impact is the addition of a <meta> tag variant to provide a key to the browser which will enable features for specific origins. We intend to iterate on this as we gather feedback from other browser vendors and web developers.
A specific goal of the Framework is to ensure long-term compatibility risk for new features is reduced. This will be accomplished through UMA and RAPPOR monitoring of feature use and rate limiting of feature use to ensure that experimental features do not reach usage levels which would prevent them from being withdrawn from the platform.
Ongoing technical constraints
None.
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Windows, Mac, Linux, ChromeOS, Android: Yes
Android WebView: Unsure. First, it’s not yet clear that experimentation is needed within a web view. Second, some of the anticipated infrastructure may not be compatible.
OWP launch tracking bug
https://code.google.com/p/chromium/issues/detail?id=538593
Link to entry on the feature dashboard
https://www.chromestatus.com/features/6331324939894784
Requesting approval to ship?
No
How would you feature detect in a css stylesheet?
Glad to see this moving forward!Can this framework also be used for features that interact with the network, i.e. rely on network headers? How would you feature detect in a css stylesheet?
How does this relate to navigator.connect()?
How will this framework interact with finch?
best-jochen
Thanks for the responses!
If you don't use finch, will you implement a separate pipeline for the gathering of stats and for the evaluation of the experiments?
Thanks
Jochen
We discussed this very question in the design doc, starting with my question "Making it part of the markup does not sound tricky," but it wasn't really resolved, so I'm still curious. Making it all work with RuntimeEnabledFeatures would be sweet, but that's simply not compatible with doing it in markup.
Imagine that we have [ExperimentEnabled] on window.xxx.The current design is:- when the experiment is enabled, window.xxx exists and works as a valid web API.- when the experiment is disabled, window.xxx does not exist.The latter requires to hide the DOM attribute/method from the window object, which adds a lot of complexity to the binding layer (I'm reviewing this CL).Alternately, wouldn't the following behavior be acceptable?- when the experiment is enabled, window.xxx exists and works as a valid web API.- when the experiment is disabled, window.xxx exists but returns undefined with a kind error message.
Is the plan that the experimental APIs will always have unpredictable names? If so, the risk to exposing the properties when the experiment is disabled seems pretty low to me. I agree that it would be unfortunate to have to add this much complexity to the bindings.
Feature detection would be a little awkward I assume (instead of "'feature_x1591' in window" you'd probably want to test against undefined explicitly: "window.feature_x1591 !== undefined", right?). Perhaps the properties for experimental APIs should always be unenumerable to further reduce the risk of impacting sites that haven't explicitly enabled them?I assume interface objects could be handled in a similar way without too much trouble, right? I.e. 'window.MyClass_b87a4' would have the value undefined when the experiment is disabled, yes 'new MyClass_b87a4()' would work when it's enabled.That said, there are other reasons to want the ability to add/remove bindings at runtime, so if there is a good way to do it properly, perhaps it's worth the effort?
On Tue, Dec 15, 2015 at 9:53 PM, Rick Byers <rby...@chromium.org> wrote:Is the plan that the experimental APIs will always have unpredictable names? If so, the risk to exposing the properties when the experiment is disabled seems pretty low to me. I agree that it would be unfortunate to have to add this much complexity to the bindings.No -- we threw that idea around for a while -- it definitely does get the message across that you're going to have to update your site's code every few weeks, but in the end I think we felt like it was more important that the API we expose "feel" as much like the proposed final API as possible.Using the actual proposed property names also reduces developer friction. It makes it more likely that developers who want to try out code that they copy, from a blog post, for instance, have a positive experience with the API, and don't get stuck and frustrated because they missed one place where they needed to customize the property name.Feature detection would be a little awkward I assume (instead of "'feature_x1591' in window" you'd probably want to test against undefined explicitly: "window.feature_x1591 !== undefined", right?). Perhaps the properties for experimental APIs should always be unenumerable to further reduce the risk of impacting sites that haven't explicitly enabled them?I assume interface objects could be handled in a similar way without too much trouble, right? I.e. 'window.MyClass_b87a4' would have the value undefined when the experiment is disabled, yes 'new MyClass_b87a4()' would work when it's enabled.That said, there are other reasons to want the ability to add/remove bindings at runtime, so if there is a good way to do it properly, perhaps it's worth the effort?Definitely (a couple of us were just talking about about that bug last week in this context, actually). And I'm sure that once we have the capability, we'll find all kinds of new uses for it. It may not be necessary for an MVP of this framework, but I'd really like to see us aim for that kind of implementation in the long run.
Isn't the confusion we get from add/remove bindings exactly the same confusion as libraries / users can cause to themselves by installing JS shims on the DOM objects?If so, the users' feet may already be so full of holes that adding another footgun doesn't increase the pain by any noticeable margin.
On Wed, Dec 16, 2015 at 5:11 AM, Kentaro Hara <har...@chromium.org> wrote:On Wed, Dec 16, 2015 at 12:59 PM, Ian Clelland <icle...@chromium.org> wrote:On Tue, Dec 15, 2015 at 9:53 PM, Rick Byers <rby...@chromium.org> wrote:Is the plan that the experimental APIs will always have unpredictable names? If so, the risk to exposing the properties when the experiment is disabled seems pretty low to me. I agree that it would be unfortunate to have to add this much complexity to the bindings.No -- we threw that idea around for a while -- it definitely does get the message across that you're going to have to update your site's code every few weeks, but in the end I think we felt like it was more important that the API we expose "feel" as much like the proposed final API as possible.Using the actual proposed property names also reduces developer friction. It makes it more likely that developers who want to try out code that they copy, from a blog post, for instance, have a positive experience with the API, and don't get stuck and frustrated because they missed one place where they needed to customize the property name.
Feature detection would be a little awkward I assume (instead of "'feature_x1591' in window" you'd probably want to test against undefined explicitly: "window.feature_x1591 !== undefined", right?). Perhaps the properties for experimental APIs should always be unenumerable to further reduce the risk of impacting sites that haven't explicitly enabled them?I assume interface objects could be handled in a similar way without too much trouble, right? I.e. 'window.MyClass_b87a4' would have the value undefined when the experiment is disabled, yes 'new MyClass_b87a4()' would work when it's enabled.That said, there are other reasons to want the ability to add/remove bindings at runtime, so if there is a good way to do it properly, perhaps it's worth the effort?Definitely (a couple of us were just talking about about that bug last week in this context, actually). And I'm sure that once we have the capability, we'll find all kinds of new uses for it. It may not be necessary for an MVP of this framework, but I'd really like to see us aim for that kind of implementation in the long run.Hmm. I guess it will confuse developers if we have the ability to add/remove DOM bindings at runtime.Imagine the following scenario:(1) Element.xxx is added.(2) e = document.createElement("div")(3) e.xxx exists and works as a valid web API.(4) Element.xxx is removed.(5) e2 = document.createElement("div")(6) e2.xxx does not exist. e.xxx exists but does not work. This would be confusing.To prevent the confusion, whether the DOM binding is enabled or disabled should be determined when the JS context is initialized and it should not change until the context terminates. What do you think?
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
On Wed, Dec 16, 2015 at 3:24 AM, Harald Alvestrand <h...@google.com> wrote:Isn't the confusion we get from add/remove bindings exactly the same confusion as libraries / users can cause to themselves by installing JS shims on the DOM objects?If so, the users' feet may already be so full of holes that adding another footgun doesn't increase the pain by any noticeable margin.On Wed, Dec 16, 2015 at 5:11 AM, Kentaro Hara <har...@chromium.org> wrote:On Wed, Dec 16, 2015 at 12:59 PM, Ian Clelland <icle...@chromium.org> wrote:On Tue, Dec 15, 2015 at 9:53 PM, Rick Byers <rby...@chromium.org> wrote:Is the plan that the experimental APIs will always have unpredictable names? If so, the risk to exposing the properties when the experiment is disabled seems pretty low to me. I agree that it would be unfortunate to have to add this much complexity to the bindings.No -- we threw that idea around for a while -- it definitely does get the message across that you're going to have to update your site's code every few weeks, but in the end I think we felt like it was more important that the API we expose "feel" as much like the proposed final API as possible.Using the actual proposed property names also reduces developer friction. It makes it more likely that developers who want to try out code that they copy, from a blog post, for instance, have a positive experience with the API, and don't get stuck and frustrated because they missed one place where they needed to customize the property name.Ok, I see the upsides there. I assume people are OK with the risk that, when it comes time to actually ship an API that was previously experimental, we might (in the worst case) need to rename the API in order to be compatible with code left over on the web designed for the experimental version, right?Feature detection would be a little awkward I assume (instead of "'feature_x1591' in window" you'd probably want to test against undefined explicitly: "window.feature_x1591 !== undefined", right?). Perhaps the properties for experimental APIs should always be unenumerable to further reduce the risk of impacting sites that haven't explicitly enabled them?I assume interface objects could be handled in a similar way without too much trouble, right? I.e. 'window.MyClass_b87a4' would have the value undefined when the experiment is disabled, yes 'new MyClass_b87a4()' would work when it's enabled.That said, there are other reasons to want the ability to add/remove bindings at runtime, so if there is a good way to do it properly, perhaps it's worth the effort?Definitely (a couple of us were just talking about about that bug last week in this context, actually). And I'm sure that once we have the capability, we'll find all kinds of new uses for it. It may not be necessary for an MVP of this framework, but I'd really like to see us aim for that kind of implementation in the long run.Hmm. I guess it will confuse developers if we have the ability to add/remove DOM bindings at runtime.Imagine the following scenario:(1) Element.xxx is added.(2) e = document.createElement("div")(3) e.xxx exists and works as a valid web API.(4) Element.xxx is removed.(5) e2 = document.createElement("div")(6) e2.xxx does not exist. e.xxx exists but does not work. This would be confusing.
To prevent the confusion, whether the DOM binding is enabled or disabled should be determined when the JS context is initialized and it should not change until the context terminates. What do you think?Do we delay initializing the JS context until after we have received/parsed at least the HTML head? Even if we do, is it important to support the scenario where the <meta> tag is added by JS at runtime?
Perhaps we can simplify this / remove footguns by saying that we'll support dynamic addition of APIs only, not dynamic removal?
Is it entirely hopeless to build the experimental framework on top of RuntimeEnabledFeatures by pre-scanning for the <meta> tag somewhere in the networking code before the renderer process is even started, or at least while there's time to kill and start another renderer instead?Are any <meta http-equiv="x"> currently handled outside of Blink?
Wouldn't it be OK to simply require that the <meta> be in the first resource that you fetch when entering a URL and pressing enter? (The question is moot if it isn't actually possible, of course.)
Right, HTTP headers only doesn't seem sufficient, but if HTTP headers will part of it, then using <meta http-equiv> comes quite naturally. So, are any <meta http-equiv> headers recognized before the renderer is started?
That makes sense, as far as creating an execution environment that developers can reason about. We could disable adding any new APIs via the experimental framework as soon as a script or link tag is encountered, for instance.
That may mean that it's no longer possible to introduce a JavaScript API to add keys, say, from a service worker, but we still have HTTP headers for that if we need them.
How does process limit relate to this? This should just be toggling the visibility of APIs in the v8::Context. I do think we need to make this like charset where it must be at the start of the <head> before any script runs in that context.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/experimentation-dev/CAK_TSXJyH6ZOfXac81AqJCihh_bMZsG69e%3DhNFHrfvXjaNS4zg%40mail.gmail.com.You received this message because you are subscribed to the Google Groups "experimentation-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to experimentation...@chromium.org.
To post to this group, send email to experimen...@chromium.org.