Intent to enable e10s by default when running tests locally

[Andrew Halberstadt]

Bug 1243083 tracks enabling e10s by default when running tests locally.
I intend to do this for as many harnesses as possible unless someone
says otherwise.

The implication is that the --e10s flag will be renamed to
--disable-e10s. So you'll need to pass in --disable-e10s if you wish to
run without e10s enabled. This also means that tests that are skipped
with 'e10s' will no longer run by default locally. The hope is that this
will make it easier for devs to find and fix tests that are broken with
e10s enabled, as well as ensure that new tests work with e10s right out
of the gate. CI jobs will not be affected, only running locally.

One potential sticking point is mochitest-chrome. It currently does not
get run in e10s in CI, so local configuration should mirror this.
However, since --e10s is being removed, this means it would be
impossible to run mochitest-chrome in e10s without modifying source
files. Maybe this just needs some argparse hackery.

If you have any concerns or know of other suites that should explicitly
*not* be run with e10s enabled by default, please let me know.

Andrew

[Boris Zbarsky]

On 3/24/16 12:15 PM, Andrew Halberstadt wrote:
> If you have any concerns or know of other suites that should explicitly
> *not* be run with e10s enabled by default, please let me know.

Do we have any plans for making --debugger not completely useless when
running tests in e10s mode? There are various ways of hacking around
it, but it would be awesome if we just made it work by default somehow...

-Boris

[Andrew Halberstadt]

I'm not aware of work around this. If --debugger is completely busted
with e10s, I could potentially make --debugger imply --disable-e10s
until it gets fixed. Is there a bug on file?

I also forgot to mention that command defaults are likely coming soon,
so once bug 1255450 lands you'll be able to make a .machrc with:

[defaults]
mochitest = --disable-e10s

[Boris Zbarsky]

On 3/24/16 12:43 PM, Andrew Halberstadt wrote:
> I'm not aware of work around this. If --debugger is completely busted
> with e10s, I could potentially make --debugger imply --disable-e10s
> until it gets fixed. Is there a bug on file?

I don't know of one.

It's not that it's busted per se, it's that it attaches the debugger to
the parent process. Which is not very helpful, in most cases. What
would be ideal (at least on unixy systems) is if we managed to detect
the child process starting and popped up an instance of $TERM or so,
with a debugger running in it and attached to the child.

> I also forgot to mention that command defaults are likely coming soon,
> so once bug 1255450 lands you'll be able to make a .machrc with:
>
> [defaults]
> mochitest = --disable-e10s

Ah, nice. Still, I agree it would be good to be testing e10s if we can
make the debugging experience there better.

-Boris

[Bobby Holley]

+1. The MOZ_DEBUG_CHILD_PROCESS hack is pretty arcane.

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Aaron Klotz]

I know that most people aren't debugging e10s on Windows, but if you
are, here's a protip (provided that you are using WinDbg):

If you include the "-o" option in the debugger args, WinDbg will
automatically attach itself to all child processes that are started by
the chrome process. No special environment variables or process startup
sleeps required.

-Aaron

[Paul Adenot]

Do we know whether `set follow-fork-mode child` in gdb would work ? If
not, can we fix it ? It would be a pretty good experience for most
developers that only care about the child.

Paul.

On Thu, Mar 24, 2016, at 06:05 PM, Aaron Klotz wrote:
> I know that most people aren't debugging e10s on Windows, but if you
> are, here's a protip (provided that you are using WinDbg):
>
> If you include the "-o" option in the debugger args, WinDbg will
> automatically attach itself to all child processes that are started by
> the chrome process. No special environment variables or process startup
> sleeps required.
>
> -Aaron
>
> On 3/24/2016 10:51 AM, Boris Zbarsky wrote:

[Jeff Muizelaar]

We fork a process to test gfx early on so 'set follow-for-mode child'
might end up following that.
'set detach-on-fork off' will keep you attached to everything though.

-Jeff

On Thu, Mar 24, 2016 at 1:21 PM, Paul Adenot <pa...@paul.cx> wrote:
> Do we know whether `set follow-fork-mode child` in gdb would work ? If
> not, can we fix it ? It would be a pretty good experience for most
> developers that only care about the child.
>
> Paul.
>
> On Thu, Mar 24, 2016, at 06:05 PM, Aaron Klotz wrote:
>> I know that most people aren't debugging e10s on Windows, but if you
>> are, here's a protip (provided that you are using WinDbg):
>>
>> If you include the "-o" option in the debugger args, WinDbg will
>> automatically attach itself to all child processes that are started by
>> the chrome process. No special environment variables or process startup
>> sleeps required.
>>
>> -Aaron
>>
>> On 3/24/2016 10:51 AM, Boris Zbarsky wrote:

[Andrew McCreight]

On Thu, Mar 24, 2016 at 9:15 AM, Andrew Halberstadt <
ahalbe...@mozilla.com> wrote:

> Bug 1243083 tracks enabling e10s by default when running tests locally.
> I intend to do this for as many harnesses as possible unless someone
> says otherwise.

Great!

[some text deleted]

One potential sticking point is mochitest-chrome. It currently does not
> get run in e10s in CI, so local configuration should mirror this.
> However, since --e10s is being removed, this means it would be
> impossible to run mochitest-chrome in e10s without modifying source
> files. Maybe this just needs some argparse hackery.
>

My impression was that mochitest-chrome doesn't actually run with e10s,
even when you specify the flag. Is that not correct?

[Andrew Halberstadt]

Afaict, ./mach mochitest -f chrome --e10s enables e10s. At least the
python side isn't overriding that default. Maybe there's some JS code
somewhere that overrides it.

But if the intent is for mochitest-chrome to never run with e10s enabled
anyway, then I guess not having that option isn't a big deal.

[Ehsan Akhgari]

On 2016-03-24 1:25 PM, Andrew McCreight wrote:
> One potential sticking point is mochitest-chrome. It currently does not
>> get run in e10s in CI, so local configuration should mirror this.
>> However, since --e10s is being removed, this means it would be
>> impossible to run mochitest-chrome in e10s without modifying source
>> files. Maybe this just needs some argparse hackery.
>>
>

> My impression was that mochitest-chrome doesn't actually run with e10s,
> even when you specify the flag. Is that not correct?

No. You currently can force it to run in e10s with --e10s like other
mochitest variants.

[Felipe G]

Yeah, --e10s enables e10s in the browser for mochitest-chrome. However,
the test harness is a .xul file opened in a tab, and that runs that tab as
non-remote, so for most tests it ends up testing the same thing as not
using --e10s. Other tabs and/or windows opened manually by the test would
be remote.


On Thu, Mar 24, 2016 at 3:05 PM, Ehsan Akhgari <ehsan....@gmail.com>
wrote:

> On 2016-03-24 1:25 PM, Andrew McCreight wrote:

> > One potential sticking point is mochitest-chrome. It currently does not
> >> get run in e10s in CI, so local configuration should mirror this.
> >> However, since --e10s is being removed, this means it would be
> >> impossible to run mochitest-chrome in e10s without modifying source
> >> files. Maybe this just needs some argparse hackery.
> >>
> >

> > My impression was that mochitest-chrome doesn't actually run with e10s,
> > even when you specify the flag. Is that not correct?
>
> No. You currently can force it to run in e10s with --e10s like other
> mochitest variants.
>

> _______________________________________________
> firefox-dev mailing list
> firef...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>

[Ehsan Akhgari]

On Thu, Mar 24, 2016 at 2:10 PM, Felipe G <fel...@gmail.com> wrote:

> Yeah, --e10s enables e10s in the browser for mochitest-chrome. However,
> the test harness is a .xul file opened in a tab, and that runs that tab as
> non-remote, so for most tests it ends up testing the same thing as not
> using --e10s. Other tabs and/or windows opened manually by the test would
> be remote.
>

D'oh, I have been working on this stuff and I didn't realize that's the
case (I was operating as if a passing mochitest-chrome when run in e10s
actually works with e10s). That's extremely confusing. :( Should we
refuse to accept --e10s when running mochitest-chrome to help avoid this
confusion?

>
> On Thu, Mar 24, 2016 at 3:05 PM, Ehsan Akhgari <ehsan....@gmail.com>
> wrote:
>
>> On 2016-03-24 1:25 PM, Andrew McCreight wrote:

>> > One potential sticking point is mochitest-chrome. It currently does not
>> >> get run in e10s in CI, so local configuration should mirror this.
>> >> However, since --e10s is being removed, this means it would be
>> >> impossible to run mochitest-chrome in e10s without modifying source
>> >> files. Maybe this just needs some argparse hackery.
>> >>
>> >

>> > My impression was that mochitest-chrome doesn't actually run with e10s,
>> > even when you specify the flag. Is that not correct?
>>
>> No. You currently can force it to run in e10s with --e10s like other
>> mochitest variants.
>>
>> _______________________________________________
>> firefox-dev mailing list
>> firef...@mozilla.org
>> https://mail.mozilla.org/listinfo/firefox-dev
>>
>
>

--
Ehsan

[Felipe G]

Yeah, that sounds like another good outcome of replacing --e10s with
--disable-e10s.

On Thu, Mar 24, 2016 at 3:59 PM, Ehsan Akhgari <ehsan....@gmail.com>
wrote:

> On Thu, Mar 24, 2016 at 2:10 PM, Felipe G <fel...@gmail.com> wrote:
>
>> Yeah, --e10s enables e10s in the browser for mochitest-chrome. However,
>> the test harness is a .xul file opened in a tab, and that runs that tab as
>> non-remote, so for most tests it ends up testing the same thing as not
>> using --e10s. Other tabs and/or windows opened manually by the test would
>> be remote.
>>
>
> D'oh, I have been working on this stuff and I didn't realize that's the
> case (I was operating as if a passing mochitest-chrome when run in e10s
> actually works with e10s). That's extremely confusing. :( Should we
> refuse to accept --e10s when running mochitest-chrome to help avoid this
> confusion?
>
>
>>
>> On Thu, Mar 24, 2016 at 3:05 PM, Ehsan Akhgari <ehsan....@gmail.com>
>> wrote:
>>
>>> On 2016-03-24 1:25 PM, Andrew McCreight wrote:

>>> > One potential sticking point is mochitest-chrome. It currently does not
>>> >> get run in e10s in CI, so local configuration should mirror this.
>>> >> However, since --e10s is being removed, this means it would be
>>> >> impossible to run mochitest-chrome in e10s without modifying source
>>> >> files. Maybe this just needs some argparse hackery.
>>> >>
>>> >

[Andrew McCreight]

I have already filed bug 1255095 about this, as you are far from the first
person to be confused by this!

Andrew

On Thu, Mar 24, 2016 at 11:59 AM, Ehsan Akhgari <ehsan....@gmail.com>

wrote:

> On Thu, Mar 24, 2016 at 2:10 PM, Felipe G <fel...@gmail.com> wrote:
>
>> Yeah, --e10s enables e10s in the browser for mochitest-chrome. However,
>> the test harness is a .xul file opened in a tab, and that runs that tab as
>> non-remote, so for most tests it ends up testing the same thing as not
>> using --e10s. Other tabs and/or windows opened manually by the test would
>> be remote.
>>
>
> D'oh, I have been working on this stuff and I didn't realize that's the
> case (I was operating as if a passing mochitest-chrome when run in e10s
> actually works with e10s). That's extremely confusing. :( Should we
> refuse to accept --e10s when running mochitest-chrome to help avoid this
> confusion?
>
>
>>
>> On Thu, Mar 24, 2016 at 3:05 PM, Ehsan Akhgari <ehsan....@gmail.com>
>> wrote:
>>
>>> On 2016-03-24 1:25 PM, Andrew McCreight wrote:

>>> > One potential sticking point is mochitest-chrome. It currently does not
>>> >> get run in e10s in CI, so local configuration should mirror this.
>>> >> However, since --e10s is being removed, this means it would be
>>> >> impossible to run mochitest-chrome in e10s without modifying source
>>> >> files. Maybe this just needs some argparse hackery.
>>> >>
>>> >

[Mike de Boer]

Slightly related: since bug 1253233 landed (well, hum, the most important parts of it), it’s possible to add fully functioning <browser remote=“true”/> elements to mochitest-chrome test windows. Since many chrome test use the pattern of adding a browser element and do stuff with it upon load, I thought this might be a nice pattern to reuse. (Run the current test code twice, once for each browser element.)

You can also use Task.jsm and ContentTask.jsm tasks just like you’re used to from mochitest-browser tests. You can also use Assert.* assertions _inside_ frame scripts spawned by ContentTask.
Example that converts a mochitest-chrome test to use ContentTask: https://hg.mozilla.org/mozilla-central/rev/2aad094c7cf8#l1.34 <https://hg.mozilla.org/mozilla-central/rev/2aad094c7cf8#l1.34>

Just FYI, have fun!

Mike.

> On 24 Mar 2016, at 20:27, Andrew McCreight <amccr...@mozilla.com> wrote:
>
> I have already filed bug 1255095 about this, as you are far from the first
> person to be confused by this!
>
> Andrew
>

> On Thu, Mar 24, 2016 at 11:59 AM, Ehsan Akhgari <ehsan....@gmail.com <mailto:ehsan....@gmail.com>>

> wrote:
>
>> On Thu, Mar 24, 2016 at 2:10 PM, Felipe G <fel...@gmail.com> wrote:
>>
>>> Yeah, --e10s enables e10s in the browser for mochitest-chrome. However,
>>> the test harness is a .xul file opened in a tab, and that runs that tab as
>>> non-remote, so for most tests it ends up testing the same thing as not
>>> using --e10s. Other tabs and/or windows opened manually by the test would
>>> be remote.
>>>
>>
>> D'oh, I have been working on this stuff and I didn't realize that's the
>> case (I was operating as if a passing mochitest-chrome when run in e10s
>> actually works with e10s). That's extremely confusing. :( Should we
>> refuse to accept --e10s when running mochitest-chrome to help avoid this
>> confusion?
>>
>>
>>>
>>> On Thu, Mar 24, 2016 at 3:05 PM, Ehsan Akhgari <ehsan....@gmail.com>
>>> wrote:
>>>
>>>> On 2016-03-24 1:25 PM, Andrew McCreight wrote:

>>>>> One potential sticking point is mochitest-chrome. It currently does not
>>>>>> get run in e10s in CI, so local configuration should mirror this.
>>>>>> However, since --e10s is being removed, this means it would be
>>>>>> impossible to run mochitest-chrome in e10s without modifying source
>>>>>> files. Maybe this just needs some argparse hackery.
>>>>>>
>>>>>

>>>>> My impression was that mochitest-chrome doesn't actually run with e10s,
>>>>> even when you specify the flag. Is that not correct?
>>>>
>>>> No. You currently can force it to run in e10s with --e10s like other
>>>> mochitest variants.
>>>>
>>>> _______________________________________________
>>>> firefox-dev mailing list
>>>> firef...@mozilla.org
>>>> https://mail.mozilla.org/listinfo/firefox-dev
>>>>
>>>
>>>
>>
>>
>> --
>> Ehsan
>>

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org <mailto:dev-pl...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/dev-platform <https://lists.mozilla.org/listinfo/dev-platform>

[Karl Tomlinson]

Felipe G. writes:

> Yeah, --e10s enables e10s in the browser for mochitest-chrome. However,
> the test harness is a .xul file opened in a tab, and that runs that tab as
> non-remote, so for most tests it ends up testing the same thing as not
> using --e10s. Other tabs and/or windows opened manually by the test would
> be remote.

What makes that first tab non-remote?

I had guessed that documents from "chrome:" would run in the browser
process (by default, at least), but is it all determined by
attributes on a browser element?

[Blake Kaplan]

Karl Tomlinson <moz...@karlt.net> wrote:
> I had guessed that documents from "chrome:" would run in the browser
> process (by default, at least), but is it all determined by
> attributes on a browser element?

It's more complicated than that. See <http://mxr.mozilla.org/mozilla-central/source/browser/modules/E10SUtils.jsm#28> for the gory details.
--
Blake Kaplan

[Tom Tromey]

>>>>> "Paul" == Paul Adenot <pa...@paul.cx> writes:

Paul> Do we know whether `set follow-fork-mode child` in gdb would work ? If
Paul> not, can we fix it ? It would be a pretty good experience for most
Paul> developers that only care about the child.

I try it from time to time and file bugs with upstream gdb for the
problems I run into. It works ok-ish but I wouldn't really recommend it
yet. There's ongoing work in gdb to make this work more nicely, but I
don't think there's any easy place to track it.

Tom

[L. David Baron]

On Thursday 2016-03-24 12:51 -0400, Boris Zbarsky wrote:
> On 3/24/16 12:43 PM, Andrew Halberstadt wrote:
> >I'm not aware of work around this. If --debugger is completely busted
> >with e10s, I could potentially make --debugger imply --disable-e10s
> >until it gets fixed. Is there a bug on file?
>
> I don't know of one.
>
> It's not that it's busted per se, it's that it attaches the debugger to the
> parent process. Which is not very helpful, in most cases.

If your debugger is "rr" (which it should be!), then what happens
today is perfectly fine. :-)

(And I find debugging with rr to be a better experience with e10s
than without, since I don't have to worry about hitting breakpoints
for chrome stuff.)

-David

--
𝄞 L. David Baron http://dbaron.org/ 𝄂
𝄢 Mozilla https://www.mozilla.org/ 𝄂
Before I built a wall I'd ask to know
What I was walling in or walling out,
And to whom I was like to give offense.
- Robert Frost, Mending Wall (1914)

[Blair McBride]

Default options imply the default is somehow favoured over the
non-default. Which, for this, makes me wonder: Is getting tests passing
on non-e10s less important now?

I've found the ergonomics around testing both to be quite a footgun -
it's a manual process to run tests under both modes, and it's too easy
to forget to run tests under the other mode. Would be handy to have a
switch to enable running under both modes (or even better, defaulting to
both modes).

- Blair

[Boris Zbarsky]

On 4/2/16 5:37 PM, L. David Baron wrote:
> If your debugger is "rr" (which it should be!)

It often is, but I'm not always debugging on Linux...

-Boris

[Andrew Halberstadt]

My understanding is shipping e10s is the top priority, so I believe that
implies running tests there is (slightly) favoured.

But I like your idea for a dual mode. I'm on the fence whether it would
be a good default or not as it will double the time to run tests
locally, and many people likely don't need to test both (as try will do
it for them). This might be a good use case for command aliases (landing
soon in bug 1255450 hopefully).

Andrew

[Ehsan Akhgari]

On Sun, Apr 3, 2016 at 11:51 PM, Blair McBride <bmcb...@mozilla.com> wrote:

> Default options imply the default is somehow favoured over the
> non-default. Which, for this, makes me wonder: Is getting tests passing on
> non-e10s less important now?
>

Fennec doesn't use e10s, so at least for tests that cover both desktop and
Android, they're both equally important. Plus, as I understand the plans,
we're going to be shipping e10s and non-e10s on desktop at the same time to
different users at least for a while.

> I've found the ergonomics around testing both to be quite a footgun - it's
> a manual process to run tests under both modes, and it's too easy to forget
> to run tests under the other mode. Would be handy to have a switch to
> enable running under both modes (or even better, defaulting to both modes).
>
> - Blair
>
>
>
>
> Andrew Halberstadt wrote:
>

>> _______________________________________________
>> firefox-dev mailing list
>> firef...@mozilla.org
>> https://mail.mozilla.org/listinfo/firefox-dev
>>
>
> _______________________________________________
> firefox-dev mailing list
> firef...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>

--
Ehsan

[Andrew Halberstadt]

On 04/04/16 09:37 AM, Ehsan Akhgari wrote:
> On Sun, Apr 3, 2016 at 11:51 PM, Blair McBride <bmcb...@mozilla.com> wrote:
>
>> Default options imply the default is somehow favoured over the
>> non-default. Which, for this, makes me wonder: Is getting tests passing on
>> non-e10s less important now?
>>
>
> Fennec doesn't use e10s, so at least for tests that cover both desktop and
> Android, they're both equally important. Plus, as I understand the plans,
> we're going to be shipping e10s and non-e10s on desktop at the same time to
> different users at least for a while.

The default on Fennec will remain non-e10s. And yes, running non-e10s is
still important on desktop too. But if you *had* to pick one or the other..

I filed a bug [1] to at least implement a dual mode. We can debate
whether it should be default or not later.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1261823

[Andrew Halberstadt]

This has now landed on central. The following suites are e10s by default
and require the --disable-e10s flag to run non-e10s:

* marionette
* mochitest (excluding chrome, a11y and android)
* reftest (excluding android)
* talos
* web-platform-tests

I also added some logging to marionette, mochitest and reftest both at
the start and end of the test run to make it easier to see whether e10s
was enabled or not (ctrl-f for e10s).

Please file bugs blocking bug 1243083 if you believe something is amiss.
Finally, bug 1261823 was filed to investigate a dual-mode that runs both
e10s and non-e10s. There is no eta at this time.

-Andrew