Non-Trivial SpecialPowers Usage Considered Harmful

[Bobby Holley]

I've mentioned this before [1], but recent checkins in mozilla-central
suggest that this message could use repeating.

Please don't write new tests that do tricky things with SpecialPowers.wrap
and friends (like SpecialPowers.Cc and SpecialPowers.Cu, which do an
implicit .wrap()).

These wrappers are intended to be a quick-and-dirty tool to frob things
from mochitest-plain, but have lots of quirks that cause complicated use
cases to fail in mysterious ways. Moreover, the transitive nature of the
wrappers usually cause them to spread much farther than the author
intended. This can waste a lot of time, and significantly reduces the value
of your tests, because they're testing a complicated/magical/best-effort
proxy layer rather than what we actually ship.

Here are some Do's and Don'ts:

Do:
* Poke various privileged controls (triggered GCs, setting prefs, clearing
caches) to improve the usefulness of a test.
* Examine cross-origin objects to avoid complicated postMessage-based tests.

Don't:
* Import a framework like Task.jsm or Promise.jsm.
* Implement an XPCOM component.
* Interact directly with the JSM backend of a DOM API.

If you find yourself itching to do something complicated, write a
mochitest-chrome test. The default template [2] now generates html files
(rather then XUL files), so the ergonomics there should be easier than
before.

If you don't want to write a mochitest-chrome test for some reason, you can
also use SpecialPowers.loadChromeScript(uri), which lets mochitest-plain
asynchronously load a privileged JS file in its own privileged scope.

Please enforce this during review, and don't hesitate to get in touch with
me if you have questions.

Cheers,
bholley

[1]
https://groups.google.com/d/msg/mozilla.dev.platform/AkX7jmLRQwg/KLqMxWLeQaYJ
[2] perl testing/mochitest/gen_template.pl -type chrome -b 555555

[Kyle Huey]

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

Can you look at what Marionette uses SpecialPowers for to see if it is
a problem? It might just be passing it through to the script the
Marionette client wants to execute, I'm not sure.

- Kyle

[Ehsan Akhgari]

Note that anything tested in mochitest-chrome is not tested on Firefox
OS at all, which is a *great* reason to avoid writing mochitest-chrome
at all costs.

Cheers,
Ehsan

[Bobby Holley]

On Wed, Aug 13, 2014 at 3:13 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
> Note that anything tested in mochitest-chrome is not tested on Firefox OS at
> all, which is a *great* reason to avoid writing mochitest-chrome at all
> costs.

Well, until bug 797164 is fixed. Mochitest-chrome is absolutely the
right framework for tests that need to do extensive privileged things,
so I think we should fix our automation story rather than telling
people to write flaky tests in the wrong framework.

bholley

[Ehsan Akhgari]

Sure, but it's important to know the trade-offs.

[Jed Davis]

Bobby Holley <bobby...@gmail.com> writes:
[...]

> If you find yourself itching to do something complicated, write a
> mochitest-chrome test. The default template [2] now generates html files
> (rather then XUL files), so the ergonomics there should be easier than
> before.
>
> If you don't want to write a mochitest-chrome test for some reason, you can
> also use SpecialPowers.loadChromeScript(uri), which lets mochitest-plain
> asynchronously load a privileged JS file in its own privileged scope.

On e10s-enabled platforms, does loadChromeScript run the script in the
parent process? There are currently a few mochitests (plain) that are
SpecialPowers'ing nsLocalFile (or other classes that do direct
filesystem access) in the content process, and I'd like to change them
to remote that part of the test to the parent process -- preferably
without reducing test coverage. (See also: https://bugzil.la/1043470#c6 )

--Jed

[Bobby Holley]

On Mon, Aug 18, 2014 at 1:14 PM, Jed Davis <j...@mozilla.com> wrote:

> Bobby Holley <bobby...@gmail.com> writes:
> [...]
> > If you find yourself itching to do something complicated, write a
> > mochitest-chrome test. The default template [2] now generates html files
> > (rather then XUL files), so the ergonomics there should be easier than
> > before.
> >
> > If you don't want to write a mochitest-chrome test for some reason, you
> can
> > also use SpecialPowers.loadChromeScript(uri), which lets mochitest-plain
> > asynchronously load a privileged JS file in its own privileged scope.
>
> On e10s-enabled platforms, does loadChromeScript run the script in the
> parent process?

It operates over the message manager, so I believe that it does:
http://mxr.mozilla.org/mozilla-central/source/testing/specialpowers/content/SpecialPowersObserverAPI.js#346