Intent to Implement and Ship: beforescriptexecute and afterscriptexecute events

[Rob Wu]

Contact emails
r...@robwu.nl

Spec
http://www.w3.org/html/wg/drafts/html/master/scripting-1.html#execute-the-script-block

Summary
The beforescriptexecute and afterscriptexecute events allows web pages and extension developers to monitor the insertion of <script> tags in a web page and prevent the script from executing.

Motivation
Chrome currently lacks a fine-grained way to control the execution of scripts. With these standardized events, an extension like NoScript can become more realistic for Chrome.

Compatibility Risk
Firefox: Shipped
Internet Explorer: No public signals
Safari: No public signals, despite feature requests from web developers (https://bugs.webkit.org/show_bug.cgi?id=91463)
Web developers: Positive

Opera: Supported until it switched to Blink/V8.

Ongoing technical constraints
None

Will this feature be supported on all five Blink platforms (Windows, Mac, Linux, Chrome OS and Android)?
Yes.

OWP launch tracking bug?
http://crbug.com/168175

Link to entry on the feature dashboard
http://www.chromestatus.com/admin/features/edit/5711871906676736

Requesting approval to ship?
Yes

[Rafael Weinstein]

This strikes me like something which is possibly appropriate for extensions, but would concern me to expose to all web pages.

Are there non-extension use-cases for this capability?

[Rob Wu]

Non-extension use cases of these events include monkey-patching code (e.g. for script loaders) and gathering statistics (e.g. for debugging).

PS. The chromestatus link in my first mail should be http://www.chromestatus.com/features/5711871906676736.

[Adam Barth]

I'm not sure we should implement these events.

On Mon Mar 24 2014 at 2:29:10 PM, Rob Wu <r...@robwu.nl> wrote:

Non-extension use cases of these events include monkey-patching code (e.g. for script loaders) and gathering statistics (e.g. for debugging).

The extension use cases are probably better served with extension-specific APIs that don't require extensions to inject content script into every web page.  For example, maybe we need to elaborate the declarative web request API in some way to handle these use cases.

If you want to write a monkey-patching script loader, you probably should use a service worker to be able to manipulate responses at the network layer instead of in the DOM.  As for statistics, we can address those use cases via something like the WebPerformance API.

Adam

[Rob Wu]

2014-03-24 22:36 GMT+01:00 Adam Barth <aba...@google.com>:

I'm not sure we should implement these events.

On Mon Mar 24 2014 at 2:29:10 PM, Rob Wu <r...@robwu.nl> wrote:

Non-extension use cases of these events include monkey-patching code (e.g. for script loaders) and gathering statistics (e.g. for debugging).

The extension use cases are probably better served with extension-specific APIs that don't require extensions to inject content script into every web page.  For example, maybe we need to elaborate the declarative web request API in some way to handle these use cases.

The webRequest APIs cannot be used to detect and block inline scripts.

And it cannot be used to modify response bodies either (read-only access is in development though).

 

If you want to write a monkey-patching script loader, you probably should use a service worker to be able to manipulate responses at the network layer instead of in the DOM.

That sounds like a really big hammer, especially in third-party libraries.

 

 

 As for statistics, we can address those use cases via something like the WebPerformance API.

How would you profile the time spent on running scripts without before-script-execute and after-script-execute events, without editing all involved code and insert console.time(++globalCounter); and console.timeEnd(globalCounter);?

Mutation observers won't fit the bill, because these can only detect tag insertion, not execution.

[Elliott Sprehn]

On Mon, Mar 24, 2014 at 3:21 PM, Rob Wu <r...@robwu.nl> wrote:

2014-03-24 22:36 GMT+01:00 Adam Barth <aba...@google.com>:

I'm not sure we should implement these events.

I also don't think we should implement these. It feels like more beforeload events.

 

On Mon Mar 24 2014 at 2:29:10 PM, Rob Wu <r...@robwu.nl> wrote:

Non-extension use cases of these events include monkey-patching code (e.g. for script loaders) and gathering statistics (e.g. for debugging).

The extension use cases are probably better served with extension-specific APIs that don't require extensions to inject content script into every web page.  For example, maybe we need to elaborate the declarative web request API in some way to handle these use cases.

The webRequest APIs cannot be used to detect and block inline scripts.

And it cannot be used to modify response bodies either (read-only access is in development though).

 

If you want to block scripts that should be an extension API. It doesn't seem like normal web applications need this.

 

[...]

 

 

 

 As for statistics, we can address those use cases via something like the WebPerformance API.

How would you profile the time spent on running scripts without before-script-execute and after-script-execute events, without editing all involved code and insert console.time(++globalCounter); and console.timeEnd(globalCounter);?

Mutation observers won't fit the bill, because these can only detect tag insertion, not execution.

 

We can expose numbers on the <script> element itself for that use case. Note that you can also get pretty good behavior, except for blocking the parser (which is rarely what you want) by using a custom element instead.

ex. http://jsfiddle.net/K5fMa/

This is a custom element that fires the events you want, and has the performance metrics.

- E

[Tab Atkins Jr.]

On Mon, Mar 24, 2014 at 4:43 PM, Elliott Sprehn <esp...@chromium.org> wrote:
> Note that you can also get pretty good behavior, except for blocking the
> parser (which is rarely what you want) by using a custom element instead.

Make that *never* what you want. The sole reason that scripts block
the parser is because of document.write(), and no one should ever be
using that, ever.

~TJ

[Hajime Morrita]

Probably it is worth mentioning why there are pushbacks from Blink developers:

Historically, mutation events are one of the big source of security vulnerabilities. One popular attack pattern is to use beforeXxx events to do some ridiculous thing that browser internals don't expect. MutationObserver was then invented to remove these footgun-ish characteristics from mutation events while providing almost same utility to web developers. 

The @beforescriptexecute and @afterscriptexecute events proposed here have certain similarities between the notorious mutation events. That's why there are some skepticism around this. I personally have same concern. As fixed several mutation event related crashes, I now strongly feel these features are better be killed if possible, and wish we don't make similar mistakes anymore.

Declarative approach like WebRequest is safer in general, and limiting scope to extensions is lowering security risk as the feature cannot be used on wild Web where many attackers live. 

Hope this illustrates background color of some reactions.

On Mon, Mar 24, 2014 at 4:43 PM, Elliott Sprehn <esp...@chromium.org> wrote:

[Ojan Vafai]

http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-February/034629.html lays out some more compelling use-cases (to me anyways). That said, I agree that the use-cases listed don't justify the implementation burden.

[Domenic Denicola]

On Monday, March 24, 2014 5:36:43 PM UTC-4, Adam Barth wrote: 

If you want to write a monkey-patching script loader, you probably should use a service worker to be able to manipulate responses at the network layer instead of in the DOM.  As for statistics, we can address those use cases via something like the WebPerformance API.

This is very interesting. It seems feasible to me, based on my limited service-worker knowledge, that you could create a 100% faithful polyfill for these events using service worker. Can someone with more knowledge confirm? If so, that is great news in terms of service worker being a primitive that addresses many use cases.

That said, the idea of third-party scripts needing coordination with the page's service worker to use these events is a bit unfortunate, as it means libraries could not depend on them without saying "copy and paste this extra code into your service worker."

Two other points:

- I believe these events would be exactly what would be needed to implement RequireJS-like module loading without wrapping all your modules in define(function (require, exports, module) { ... }). In beforescriptexecute, you set window.require, window.exports, and window.module to their appropriate values for the just-loaded script; in afterscriptexecute, you pick up changes caused by usage and manipulation of those three variables, store them in the module lookup table, and then remove them. (As is, most module loaders who want to avoid the wrapper do so by XHR-ing the script source, adding the wrapper with string concatenation, and then eval'ing it.) This strikes me as a very compelling use case.

- If Blink's position is that these events should not exist due to inherently-problematic characteristics, would it be appropriate to request they be removed from the standard, and coordinate with Firefox on such a removal? Alternately, do the Firefox developers have any feedback on if perhaps there is an implementation strategy that avoids the problems of mutation events? Or would there be a revision to the standard that would be more palatable (e.g. a "ScriptObserver")?

[Jake Archibald]

On Tue, Mar 25, 2014 at 12:46 PM, Domenic Denicola <dom...@domenicdenicola.com> wrote:

On Monday, March 24, 2014 5:36:43 PM UTC-4, Adam Barth wrote: 

If you want to write a monkey-patching script loader, you probably should use a service worker to be able to manipulate responses at the network layer instead of in the DOM.  As for statistics, we can address those use cases via something like the WebPerformance API.

This is very interesting. It seems feasible to me, based on my limited service-worker knowledge, that you could create a 100% faithful polyfill for these events using service worker. Can someone with more knowledge confirm? If so, that is great news in terms of service worker being a primitive that addresses many use cases.

You can certainly use it to run code before & after the execution of a script, but it might fall short of a 100% faithful polyfill.

Yeah, if request.purpose is "script", fetch the script & modify it before shipping it to the page. This code could fire beforescriptexecute & afterscriptexecute accordingly. Making beforescriptexecute cancellable is where it gets tricky. If the default was prevented, you'd need to prevent further execution. This could be done by wrapping the body of the code up and evalling it, but if you don't do that (the default-prevented case) you're still going to get a load event, which you shouldn't.

[Jens Lindström]

On Mon, Mar 24, 2014 at 9:45 PM, Rob Wu <r...@robwu.nl> wrote:
> Contact emails
> r...@robwu.nl
>
> Spec
> http://www.w3.org/html/wg/drafts/html/master/scripting-1.html#execute-the-script-block
>
> Summary
> The beforescriptexecute and afterscriptexecute events allows web pages and
> extension developers to monitor the insertion of <script> tags in a web page
> and prevent the script from executing.

> Compatibility Risk

> Opera: Supported until it switched to Blink/V8.

Opera/Presto never supported these events AFAIK. We had roughly
equivalent events that were part of the "User Javascript" mechanism
(i.e. site-patching scripts installed by the user or by an extension,)
but not available to regular page scripts.

--
Jens

[smaug]

On 03/25/2014 02:46 PM, Domenic Denicola wrote:
> On Monday, March 24, 2014 5:36:43 PM UTC-4, Adam Barth wrote:
>
> If you want to write a monkey-patching script loader, you probably should use a service worker to be able to manipulate responses at the network
> layer instead of in the DOM. As for statistics, we can address those use cases via something like the WebPerformance API.
>
>
> This is very interesting. It seems feasible to me, based on my limited service-worker knowledge, that you could create a 100% faithful polyfill for
> these events using service worker. Can someone with more knowledge confirm? If so, that is great news in terms of service worker being a primitive
> that addresses many use cases.
>
> That said, the idea of third-party scripts needing coordination with the page's service worker to use these events is a bit unfortunate, as it means
> libraries could not depend on them without saying "copy and paste this extra code into your service worker."
>
> Two other points:
>

> - I believe these events would be exactly what would be needed to implement RequireJS-like module loading /without/ wrapping all your modules in

> define(function (require, exports, module) { ... }). In beforescriptexecute, you set window.require, window.exports, and window.module to their
> appropriate values for the just-loaded script; in afterscriptexecute, you pick up changes caused by usage and manipulation of those three variables,
> store them in the module lookup table, and then remove them. (As is, most module loaders who want to avoid the wrapper do so by XHR-ing the script
> source, adding the wrapper with string concatenation, and then eval'ing it.) This strikes me as a very compelling use case.
>
> - If Blink's position is that these events should not exist due to inherently-problematic characteristics, would it be appropriate to request they be
> removed from the standard, and coordinate with Firefox on such a removal? Alternately, do the Firefox developers have any feedback on if perhaps there
> is an implementation strategy that avoids the problems of mutation events?

I don't see how these events have the same issues what DOMNodeRemoved (that is the problematic mutation event) has.
When beforescriptexecute is dispatched, the element is already in DOM, so no mutations involved here.
beforescriptexecute essentially just adds a way to execute some other script before the script for the <script>
is executed.


sicking implemented this stuff, so he may have comments too.


-Olli

> Or would there be a revision to the standard that would be more palatable
> (e.g. a "ScriptObserver")?
>

> To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org
> <mailto:blink-dev+...@chromium.org>.

[Boris Zbarsky]

On 3/25/14 8:46 AM, Domenic Denicola wrote:
> , do the Firefox developers have any
> feedback on if perhaps there is an implementation strategy that avoids
> the problems of mutation events?

The fundamental problem with mutation events was that the points when
they needed to fire were problematic. In particular, they fired _after_
all the sanity-checking for the mutation had been performed... but could
then change the DOM around so that the mutation was once again not OK.

before/afterscriptexecute do not have this problem to nearly the same
degree. They fire immediately before/after the browser was going to run
arbitrary script anyway. The only interesting check we have is for
cases when beforescriptexecute breaks the link between the Document and
Window somehow (which might even be impossible nowadays; we've been
getting of manual reference-dropping like that), in which case the
script and afterscriptexecute are not run. Since the spec has no cases
in which a Document loses the reference to its Window, that check is not
in the spec, of course.

I would like to understand better what people's specific concerns are
here, because to me this looks nothing like mutation events. The key
difference, again, is that mutation events had script running in the
middle of an algorithm that could mess with the data structure the
algorithm was working on, while here we have script running when we were
going to run script anyway.

-Boris

[Adam Barth]

On Tue Mar 25 2014 at 8:09:42 AM, Boris Zbarsky <bzba...@mit.edu> wrote:

I would like to understand better what people's specific concerns are
here, because to me this looks nothing like mutation events.  The key
difference, again, is that mutation events had script running in the
middle of an algorithm that could mess with the data structure the
algorithm was working on, while here we have script running when we were
going to run script anyway.

The reasons I don't think we should implement this API are unrelated to mutation events.

The primary use cases for the feature are for extensions, but this feature would encourage extension authors to design their extension with content scripts that run in every web page, which is a pattern we'd rather discourage than encourage.  Instead, these use cases are better addressed with an extension-specific API, perhaps similar to (or an elaboration of) the declarative WebRequest API.

There are some non-extension use cases, but those are less compelling than the extension use cases and can be addressed in more generic ways by better explaining the underlying platform (e.g., custom elements, service workers).

I wouldn't close the door on implementing these events, but I don't think we should implement them today.

Adam

[Jonas Sicking]

On Tue, Mar 25, 2014 at 7:06 AM, smaug <sm...@welho.com> wrote:
> sicking implemented this stuff, so he may have comments too.

Like Boris points out, I believe these events are different from
mutation events in many ways. They fire at a time when the browser has
to be ready for arbitrary script to execute anyway.

But when used for inline <script> elements they do share the problem
that they fire synchronously. So as a page author you do run the risk
that other code will run in the middle of your code execution.
Synchronous events is generally a bad thing for script authors in that
they could cause 3rd party code to run while you're in an inconsistent
state. (Though it's also often a requested feature from page authors
since it allows them to run make their own code hook in and take
appropriate action immediately).

I don't see any implementation difficulties though.

Regarding polyfills, one thing that is important to keep in mind is
that these events fire synchronously before and after the <script>
runs. This makes it possible for these events to modify the
environment that the script, and *only* the script sees. For example
by adding a window.require function in beforescriptexecute and remove
it in afterscriptexecute.
This can be emulated for some scripts. For an inline
<polyfilled-script>s by doing
fire("beforescriptexecute");eval(node.textContent);fire("afterscriptexecute"),
and for same-origin <polyfilled-script> by doing an XHR and then
fire("beforescriptexecute");eval(xhr.resultText);fire("afterscriptexecute").

However normal <script> elements allow loading and evaluating
cross-origin scripts without requiring CORS. I don't see a way of
doing that while then also inserting synchronous events before and
after the script evaluation, even with service workers. Though it
would probably be possible to add some APIs to make it possible if
we're ok with allowing a CORSResponse to be eval()ed even though we
don't allow it to be inspected. (I think there would be some
complications involved).

/ Jonas

[Anne van Kesteren]

On Tue, Mar 25, 2014 at 4:01 PM, Jonas Sicking <jo...@sicking.cc> wrote:
> However normal <script> elements allow loading and evaluating
> cross-origin scripts without requiring CORS. I don't see a way of
> doing that while then also inserting synchronous events before and
> after the script evaluation, even with service workers. Though it
> would probably be possible to add some APIs to make it possible if
> we're ok with allowing a CORSResponse to be eval()ed even though we
> don't allow it to be inspected. (I think there would be some
> complications involved).

You mean OpaqueResponse I think. A CORSResponse can be read as that is
what CORS is for.


--
http://annevankesteren.nl/

[Adam Barth]

Sounds like we haven't explained enough of how HTMLScriptElement works to let developers create a custom script element.  Perhaps we need some sort of script loading/executing API that can load and execute OpaqueResponses.

Adam

[Domenic Denicola]

From: Adam Barth <aba...@google.com>

> Sounds like we haven't explained enough of how HTMLScriptElement works to let developers create a custom script element.  Perhaps we need some sort of script loading/executing API that can load and execute OpaqueResponses.

I believe this may tie in to the work Dave Herman is doing on the ES6 Realms API [1] [2]. CC'ing him. Note especially slides near the end starting with "future-proofing with polymorphism," which could allow sending OpaqueResponses into the Realms API for execution.

[1]: https://speakerdeck.com/dherman/status-report-es6-modules
[2]: https://gist.github.com/dherman/7568885

[Adam Barth]

On Tue Mar 25 2014 at 9:22:33 AM, Domenic Denicola <dom...@domenicdenicola.com> wrote:

From: Adam Barth <aba...@google.com>

> Sounds like we haven't explained enough of how HTMLScriptElement works to let developers create a custom script element.  Perhaps we need some sort of script loading/executing API that can load and execute OpaqueResponses.

I believe this may tie in to the work Dave Herman is doing on the ES6 Realms API [1] [2]. CC'ing him. Note especially slides near the end starting with "future-proofing with polymorphism," which could allow sending OpaqueResponses into the Realms API for execution.

Neat.  I'll have to study that proposal in more detail.

Thanks,

Adam

[srou...@gmail.com]

- If Blink's position is that these events should not exist due to inherently-problematic characteristics, would it be appropriate to request they be removed from the standard, and coordinate with Firefox on such a removal? Alternately, do the Firefox developers have any feedback on if perhaps there is an implementation strategy that avoids the problems of mutation events? Or would there be a revision to the standard that would be more palatable (e.g. a "ScriptObserver")?

Personally, I just need a reliable way to run code between script files. So if a library js file loads, I need to create hooks in there before the app js code that uses the library. 

afterscriptexecute is a nice clean way to do that which I use on Mozilla. I was using the beforeload event on Chrome until recently when they removed that. :( The WebRequestAPI reports things in network order not script execution order, so it is useless. Though today I am looking into using it to rewrite all JS files to fire the events themselves. It might frighten a few developers to see their code changed in flight though, but there seems to be no other solution.

-s

[herbe...@gmail.com]

This is very important functionality. The developers of AdBlock Plus (one of the most installed addons) need this to implement a new feature: https://issues.adblockplus.org/ticket/3207
But they can't because Chromium developers refuse to implement a feature that is officially part of the W3C HTML5 standard.

Firefox already implemented this, you didn't.

Therefore Chromium browsers fail certain HTML5 tests, like this one: http://www.w3c-test.org/html/semantics/scripting-1/the-script-element/script-before-after-events.html
Firefox passes all these tests.

I thought Chromium is all about supporting standards like HTML5?

How hard can it be?

[Dimitri Glazkov]

Thread necromancy!

Looking at back the thread, I don't believe the implementation complexity was ever brought up as a sticking point. The discussion basically faded out after https://groups.google.com/a/chromium.org/d/msg/blink-dev/bChX6leKqtg/kMmrCaVapJYJ, which I still believe to be a pretty solid position and the reason we aren't implementing these events in Blink. 

If you need this sort of API for extensions, perhaps starting a conversation in chromium-extensions mailing list would be a good next step?

:DG<

[PhistucK]

Is this really in development, Rob?

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Rob Wu]

I haven't developed this feature further since the discussion got stuck. Adam was strongly opposed to this feature, and no-one else brought stronger arguments in favor of it. In this thread, parallels with the awful mutation events were drawn, but others have already countered this concern.

There is demand for this feature (as seen by the occasional comment on this thread and the number of stars on the issue), and someone willing to implement this (me), but without an reviewer who approves of the changes, this feature is not going to happen.

If this proposal gets approved, I will write some tests which cover all edge cases that I can think of, and then implement beforescriptexecute/afterscriptexecute.

Kind regards,

 Rob

 https://robwu.nl

[Elliott Sprehn]

Is this feature still important if we have ES modules? It feels very strange for script authors to want to listen for global before/after execute hooks on every script element in the entire page, you really only want them around a specific script that uses your special module system?

If this is important I'd rather we explored allowing <script type="my/thing"> and registering the my/thing with a handler that gets called back. A global "script will run!" hook is against the modularity principles of web components. We should have fewer global hooks and more local hooks.

[Rob Wu]

ES6 modules have removed the need for script loaders; This event is mainly useful for acting upon (unknown) (third-party) scripts, either because the CMS is terrible, or in user scripts / extensions. <script type="my/thing"> assumes that the page author has full control over the script.

I was thinking about use cases, and the only concrete one is ad blockers, for blocking inline scripts. That can't be effective, since <script> InjectMyAd() </script> can just be replaced with <img onerror=" InjectMyAd(); ">. And a declarative approach (like CSP, but then with patterns) would be more effective.

I agree that we shouldn't add more global hooks if there are no huge benefits, so I retract my support for this feature.

If everyone agrees, we can close the associated bug as WontFix, remove the dashboard item and propose removal of the event from the spec.

Kind regards,

 Rob

 https://robwu.nl

[Steven Roussey]

If you remove it, please add it back to the extension api (which is my use case). -steve

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org. Steven Roussey

[Rob Wu]

beforescriptexecute / afterscriptexecute has never been a part of Chrome or its extension API.

What exactly is your use case?

Kind regards,

 Rob

 https://robwu.nl

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org. Steven Roussey

[Steven Roussey]

I gather data about the state in between scripts running. See  https://code.google.com/p/chromium/issues/detail?id=168175 Though you already know about that one. -steve

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org. Steven Roussey Steven Roussey

[rh...@raymondhill.net]

uBlock Origin does use `beforescriptexecute` on Firefox[1] -- and it's definitely a positive for users. Sure, like other aspects of a blocker, a site can still try to work around it -- but this is a reality which also must be dealt with for all sort of other mechanisms used by blockers -- yet this doesn't render the use of all those mechanisms which can be worked around as obsolete and irrelevant.

I could actually argue that for the case of `onerror=InjectMyAd()`, this should trigger a `beforescriptexecute` event for an implicit script tag containing the `InjectMyAd()` code.

[1] https://github.com/gorhill/uBlock/wiki/Inline-script-tag-filtering#concrete-examples-of-usefulness