Identifying DevTools Platform Work

[Jordan Santell]

Today in our meeting, Shu brought up ways to identify what platform work is
needed (for current tools/fixes) so we know what areas need help, and a way
to keep track. bgrins brought up that our uservoice's[0] top 10 issues are
all because we took shortcuts around not doing platform work due to the
current team's JS-focused skillset, ultimately leading us developing in
corners in some scenarios.

First step is to identify which bugs are open due to needing platform help,
which coincides with Axel's working with platform to establish a bridge
between us and platform, ensuring new features are released with proper
tooling support from the start.

Let's tag these bugs with the whiteboard tag *[devtools-platform]*.

Shout out here if there are components you'd like to tackle and tag -- I'll
jump on perf/canvas/shader/audio.

[0] https://ffdevtools.uservoice.com

--
Jordan Santell
Sr. Software Engineer | DevTools Performance & Memory

[Axel Kratel]

So just to make things clear, I am trying to bring visibility to fix three
issues:

1) figure out all existing issues on devtools side, get them prioritized
and put them on the "to be worked on" stack for platform.

2) figure out all the things that have already been worked on platform side
for which there is no tooling support (and also probably no APIs for
devtools to leverage).

3) Get ahead of the curve and identify all planned platform work where
devtools engineering needs to get involved ahead of time to predict what
sort of APIs we will need in the future so that the APIs get implemented
while the feature is implemented.

Right now, the list for 1) and 2) are huge, our hope is that eventually it
will all be down to a few items each quarter. But I am worried we will
never catch up due to lack of resources on the platform side. Hopefully, if
we document how bad it is, the resource problem will be fixed.




On Thu, Aug 20, 2015 at 9:57 AM, Jordan Santell <jsan...@mozilla.com>
wrote:

> _______________________________________________
> dev-developer-tools mailing list
> dev-devel...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-developer-tools
>

[Patrick Brosset]

Thanks Shu for bringing up the discussion.

For the inspector, I'll try to tag the bugs I know about.

Yesterday in the meeting, I said I didn't know of any big existing
inspector bugs (as in things that are broken) that were due to platform
bugs.
But, for sure, there are plenty of things we could change on platform to
help. I see 3 categories:

1. things we've worked around in JS, but could have done better if we'd had
the right skills then,
2. features we just don't have yet because we lack some platform APIs,
3. new web features that are being or will be implemented that would
require tooling.

For 1:

- The box-model highlighter is a good example of something we implemented
using JS-XUL only at the beginning, and had to rewrite completely, using a
new platform API (I added, mentored by platform people) so it would work on
firefox os, fennec, etc...
Axel mentioned a lack of resources, but I believe most of us have some C++
experience and, when mentored, can fix things like this.

For 2:

- The "styles as-authored" project that Tom mentioned also needs platform
work. We've lived so far without it because we can still show CSS style to
the user, only it's the CSS style that was interpreted by the browser, so
all invalid properties are removed. Other devtools show the style exactly
as it was authored, and we've wanted to do that to, to level-up.
As Tom said, he spends a lot of his time adding new chrome-only platform
APIs or modifying existing ones to make this work.
- The DOM walker that we use to show the DOM tree in the inspector used to
not show anonymous content or pseudo-elements. Again, we lived with it for
a long while because it's not like the inspector didn't work at all. But at
some stage, showing :before and :after in the DOM tree became important and
we worked with platform to make that happen.

For 3, I have 2 examples of things we're doing:

- The WebAnimations API (new user-facing API for animations) is being
implemented on the platform and right when that started, platform reached
out to us (Brian Birtles) in order to get some tooling right from the
start. It's a good example of how things should go for new features because
we've been working together with them for a while, syncing our roadmaps.

- Another example of a new feature is CSS Grid. Layout platform people are
working on implementing this new w3c spec, and when it's implemented, it'd
be great to have devtools support for it. So I've reached out to Mats
Palmgren who's doing most of the implementation and we're going to start
working together on this soon (Q4 I believe).

Patrick

[Jordan Santell]

Dump of perf bugs that need platform work:

1163901, 1196636, 1171090, 1131661, 1132531, 1186580, 1169115,1132525,
1090110, 1074935,1176087,1157924,1170261,
1170865,1158616,929469,1152988,1152839,1100289,1104202,1164252,1152397,1164333,1153706,1163430,
1162688,1161906, 1132529, 1152874, 1150987, 1159308, 1152400, 1152564,
1152416, 1152886, 1153701, 1152887, 1152884, 1152097, 1051005, 968510,
1067891

They do not include the current set of work in the scope of
allocations/memory that Nick and Jim are working on, or the JIT Coach
platform work via Shu and Kannan, or the OTMT/graphics markers work
Victor's doing.

A majority of them are marker bugs for adding metadata to layout markers,
with another group based around the SPS profiler.

When going through the bugs with this in mind, there are several bugs that
are designed around the fact that we'd do the "long way", and implement
something in an actor rather than a more reliable platform way -- those are
not listed above, as it's more grey, and not necessarily a 1:1 relationship
with the platform-equivalent, and we'd probably do different architectural
work anyway, was just interesting to see leaky implementations show up in
bug titles.

On Fri, Aug 21, 2015 at 12:27 AM, Patrick Brosset <pbro...@mozilla.com>
wrote:

[Jet Villegas]

Can we schedule some time to triage these? We want to support the dev tools
work and there's a lot here, so some prioritizing is in order.

Thanks,

--Jet

On Fri, Aug 21, 2015 at 7:59 AM, Jordan Santell <jsan...@mozilla.com>

[Axel Kratel]

They're all tagged now. I am adding Jim's bugs and several others I was
also able to find. So there's quite a bit more than the list above, as
Jordan already suggested.

On Fri, Aug 21, 2015 at 10:26 AM, Jet Villegas <jvil...@mozilla.com>
wrote:

[Eddy Bruel]

We have a lot of papercuts that involve platform work when it comes to
pausing the debugger.

Because the debugger runs in the same thread as the debuggee, it cannot
actually pause the latter. Instead, we create the illusion that the
debuggee is paused by spinning up a nested event loop and suppressing event
handling for debuggee events.

The problem is that our event suppression for debuggee events is not
watertight. This can cause event handling code to run even when the
debuggee is supposed to be paused. The most glaring examples of this are
XHR and postMessage.

I consider this a pretty big deal, because I imagine XHR to be something
that's used quite often, and the only thing worse than having no debugger
is having a debugger you can't trust.

Unfortunately, the *proper* solution to this problem would involve a
complete refactor of how event handling works in Firefox, which is not
feasible without significant investment of platform resources. Without such
an investment, the best we can do is write special case code to fix some of
these issues.

The bugs in question are:
801304 <https://bugzilla.mozilla.org/show_bug.cgi?id=801304> 903183
<https://bugzilla.mozilla.org/show_bug.cgi?id=903183> 1039795
<https://bugzilla.mozilla.org/show_bug.cgi?id=1039795> 1044074
<https://bugzilla.mozilla.org/show_bug.cgi?id=1044074> 1124122
<https://bugzilla.mozilla.org/show_bug.cgi?id=1124122> 1145201
<https://bugzilla.mozilla.org/show_bug.cgi?id=1145201> 1157861
<https://bugzilla.mozilla.org/show_bug.cgi?id=1157861>1178721
<https://bugzilla.mozilla.org/show_bug.cgi?id=1178721>

These bugs are all grouped under the following meta-bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=dbg-pause

I've taken the liberty to mark all these bugs as [devtools-platform] in the
whiteboard.

Hope that helps,


Eddy

[Hallvord Reiar Michaelsen Steen]

On Mon, Sep 7, 2015 at 3:46 PM, Eddy Bruel <ejpb...@mozilla.com> wrote:

> We have a lot of papercuts that involve platform work when it comes to
> pausing the debugger.
>
> Because the debugger runs in the same thread as the debuggee, it cannot
> actually pause the latter. Instead, we create the illusion that the
> debuggee is paused by spinning up a nested event loop and suppressing event
> handling for debuggee events.
>

Won't E10S make it a lot easier to fix this the right way, so that the
debugger can run in a separate process from the debugee and actually pause
the activity in the corresponding tab?
-Hallvord

[Dylan Barrell]

I assume this problem also exists when debugging extensions? That
would explain some strange behavior I have been seeing...

On Mon, Sep 7, 2015 at 9:46 AM, Eddy Bruel <ejpb...@mozilla.com> wrote:
> We have a lot of papercuts that involve platform work when it comes to
> pausing the debugger.
>
> Because the debugger runs in the same thread as the debuggee, it cannot
> actually pause the latter. Instead, we create the illusion that the
> debuggee is paused by spinning up a nested event loop and suppressing event
> handling for debuggee events.
>

[Eddy Bruel]

Unfortunately, the debugger was designed to run in the same thread as the
debuggee, so it's not quite that easy.

There are probably other reasons why this can't be done easily. Jim should
be able to tell you the details.

On Mon, Sep 7, 2015 at 4:11 PM, Hallvord Reiar Michaelsen Steen <
hst...@mozilla.com> wrote:

> On Mon, Sep 7, 2015 at 3:46 PM, Eddy Bruel <ejpb...@mozilla.com> wrote:
>
>> We have a lot of papercuts that involve platform work when it comes to
>> pausing the debugger.
>>
>> Because the debugger runs in the same thread as the debuggee, it cannot
>> actually pause the latter. Instead, we create the illusion that the
>> debuggee is paused by spinning up a nested event loop and suppressing
>> event
>> handling for debuggee events.
>>
>

[Eddy Bruel]

That's definitely possible, in particular when you are using APIs such as
XHR or postMessage. Promises also interact with XHR in adverse ways when it
comes to pausing the debuggee. See the bug list above for details on these
issues.

On Mon, Sep 7, 2015 at 4:28 PM, Dylan Barrell <dy...@barrell.com> wrote:

> I assume this problem also exists when debugging extensions? That
> would explain some strange behavior I have been seeing...
>

> On Mon, Sep 7, 2015 at 9:46 AM, Eddy Bruel <ejpb...@mozilla.com> wrote:
> > We have a lot of papercuts that involve platform work when it comes to
> > pausing the debugger.
> >
> > Because the debugger runs in the same thread as the debuggee, it cannot
> > actually pause the latter. Instead, we create the illusion that the
> > debuggee is paused by spinning up a nested event loop and suppressing
> event
> > handling for debuggee events.
> >

[Nicolas B. Pierron]

From what I know, when we debug from a separate process, we just use the
remote debugger protocol to tell the remote process to spawn a debugger to
analyze the debuggee, so this should not change any of the comments made by
Eddy.

I guess we could instrument the only entry-point of execution to call into
the Debugger if we ever attempt to call a function when the compartment is
paused. In which case we could resume immediately and let the Debugger call
the callback it-self when we resume the execution.

I guess this proposal should work fine, as long as callbacks are used for
reporting status and that their return values is not used for anything by
their callers.

--
Nicolas B. Pierron

[Eddy Bruel]

On Mon, Sep 7, 2015 at 4:32 PM, Nicolas B. Pierron <
nicolas....@mozilla.com> wrote:
>
>
> I guess we could instrument the only entry-point of execution to call into
> the Debugger if we ever attempt to call a function when the compartment is
> paused. In which case we could resume immediately and let the Debugger
> call the callback it-self when we resume the execution.
>

Hi Nicolas,

Let me see if I understand what you said correctly. You are suggesting the
following:

1. Automatically pause the debugger whenever a call is made to a function
that lives in a compartment that is paused.
2. Record this call somewhere in the debugger server, so we can call it
again late.
3. Resume with a resumption value that causes the function to return
immediately, causing no debuggee code to run.
4. When the debugger unpauses, call each function that we recorded during
step 2 before exiting the nested event loop and allowing the debuggee to
continue.

This is a really interesting idea. If it works, it should be much easier to
implement than the approach we had in mind to suppress XHR events. I'd like
to talk to you about it more in depth. What timezone are you currently
based in? Perhaps we could schedule an irc chat tomorrow?

> I guess this proposal should work fine, as long as callbacks are used for
> reporting status and that their return values is not used for anything by
> their callers.
>
>

This is true for all DOM event handlers that I'm aware of at least, but it
might not be true for others. Who would be the best person to ask about
this? Boris or Olli perhaps?


> --
> Nicolas B. Pierron

[Nicolas B. Pierron]

On 09/07/2015 04:58 PM, Eddy Bruel wrote:
> On Mon, Sep 7, 2015 at 4:32 PM, Nicolas B. Pierron <
> nicolas....@mozilla.com> wrote:
>>
>>
>> I guess we could instrument the only entry-point of execution to call into
>> the Debugger if we ever attempt to call a function when the compartment is
>> paused. In which case we could resume immediately and let the Debugger
>> call the callback it-self when we resume the execution.
>>
>
> Hi Nicolas,
>
> Let me see if I understand what you said correctly. You are suggesting the
> following:
>
> 1. Automatically pause the debugger whenever a call is made to a function
> that lives in a compartment that is paused.
> 2. Record this call somewhere in the debugger server, so we can call it
> again late.
> 3. Resume with a resumption value that causes the function to return
> immediately, causing no debuggee code to run.
> 4. When the debugger unpauses, call each function that we recorded during
> step 2 before exiting the nested event loop and allowing the debuggee to
> continue.
>
> This is a really interesting idea. If it works, it should be much easier to
> implement than the approach we had in mind to suppress XHR events. I'd like
> to talk to you about it more in depth. What timezone are you currently
> based in? Perhaps we could schedule an irc chat tomorrow?

Yes, this is what I was thinking about.
I am in Paris.
Ok for IRC tomorrow.

>> I guess this proposal should work fine, as long as callbacks are used for
>> reporting status and that their return values is not used for anything by
>> their callers.
>>
>>
> This is true for all DOM event handlers that I'm aware of at least, but it
> might not be true for others. Who would be the best person to ask about
> this? Boris or Olli perhaps?

I don't know who is the best person, Boris might know.

Also, I recall something about event handlers, where you can change the way
each event is propagated[1]. So, this might not work for such cases.

[1] https://developer.mozilla.org/en-US/docs/Web/API/Event/stopPropagation

--
Nicolas B. Pierron

[Shu-yu Guo]

Which entry point is that, Invoke?

On Mon, Sep 7, 2015 at 7:32 AM, Nicolas B. Pierron <
nicolas....@mozilla.com> wrote:

> On 09/07/2015 04:11 PM, Hallvord Reiar Michaelsen Steen wrote:
>

> From what I know, when we debug from a separate process, we just use the
> remote debugger protocol to tell the remote process to spawn a debugger to
> analyze the debuggee, so this should not change any of the comments made by
> Eddy.
>

> I guess we could instrument the only entry-point of execution to call into
> the Debugger if we ever attempt to call a function when the compartment is
> paused. In which case we could resume immediately and let the Debugger
> call the callback it-self when we resume the execution.
>

> I guess this proposal should work fine, as long as callbacks are used for
> reporting status and that their return values is not used for anything by
> their callers.
>

> --
> Nicolas B. Pierron
>
> _______________________________________________
> dev-developer-tools mailing list
> dev-devel...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-developer-tools
>

--
shu