Developer tools and separated iframes

[Bill McCloskey]

Hi everyone,

Recently on the platform team we've started discussing the practicality of
running iframes in isolation from the main page. This might either take the
form of running them in separate processes or separate threads. Either way,
JS for the iframe could be running at the same time as JS for the main
page, although only if they're not same-origin.

There's some platform work needed to do this, but also a good chunk of work
to convert frame scripts. Right now, frame scripts run with chrome
privileges and have access to all frames in a tab simultaneously. We would
need to modify our scripts and APIs so that they run on a per-frame basis
and then aggregate the data from all the frames together in the chrome
process. For most scripts this would be relatively straightforward,
although it would take some time to get right.

Developer tools pose a special challenge here. IIRC, they use a small frame
script that then loads a bunch of devtools code to implement the tab actor.
So we would need to change them to use frame actors instead, and then
modify all the tools to work with frame actors. This seems like a lot of
work.

I took a look at Chrome's devtools since they already implement an
experimental out-of-process iframe feature. It looks like the JS tools
(debugger and console) require you to choose a particular frame in which to
evaluate code or set a breakpoint. On the other hand, tools like the
inspector try to present a unified view of the page by aggregating data
from different frames.

I'm writing to ask how much effort it would be to modify our developer
tools protocols, backend, and UI in the same way. Would it take a month,
six months, or a year? How many people would be required? Who knows the
most about the underlying issues here? Which tools would need the most
work? Are there any tools where this just isn't possible or wouldn't make
sense?

Thanks,
Bill

[Nick Fitzgerald]

TabActor wishes it was more like RelatedBrowsingContextActor. AFAICT, any
JS within a related browsing context has to run sequentially -- that's not
changing is it? A cross origin iframe would be a different context
(right?), and I think we would be fine not having it observable at the same
time as the top level window. With those things in mind, this seems like a
step in the correct direction.

(Warning: bit of an off the cuff braindump incoming)

We already have a iframe/context chooser[0] and I imagine we could extend
it for this case.

[0] http://i.imgur.com/7DvON6B.png

​Another question is protocol compatibility with old servers and start up
time if we add an indirection/protocol round trip after getting tabs to get
contexts within the tab.

An alternative option would be to treat cross origin iframes like how we
treat workers, where we have a list of them on the sidebar and can open
whole new toolboxes when we want to attach to them.

On Fri, Aug 26, 2016 at 2:54 PM, Bill McCloskey <wmccl...@mozilla.com>
wrote:

> _______________________________________________
> dev-developer-tools mailing list
> dev-devel...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-developer-tools
>

[J. Ryan Stinnett]

On Fri, Aug 26, 2016 at 4:54 PM, Bill McCloskey <wmccl...@mozilla.com>
wrote:

> Developer tools pose a special challenge here. IIRC, they use a small frame
> script that then loads a bunch of devtools code to implement the tab actor.
> So we would need to change them to use frame actors instead, and then
> modify all the tools to work with frame actors. This seems like a lot of
> work.
>
> I took a look at Chrome's devtools since they already implement an
> experimental out-of-process iframe feature. It looks like the JS tools
> (debugger and console) require you to choose a particular frame in which to
> evaluate code or set a breakpoint. On the other hand, tools like the
> inspector try to present a unified view of the page by aggregating data
> from different frames.
>

I would imagine we would also want our inspector to show a unified view of
the page in an OOP frame world. I think a natural approach would be to try
to make it as "transparent" to the UI as we can. So, there would still be a
single tab actor that the UI connects to. In the server that is running as
part of the frame script for the tab, we would handle the IPC between
different frames and then ship that back to the UI as requested over the
protocol.

For other tools, we might decide to have UI to switch the focused frame (we
already have a frame chooser) and only worry about that frame. We might
still present a single "entry point" tab actor for tools that take this
path, and handle the focused frame internally to the server as well.

This "transparent" approach is similar to how we approach e10s today. When
the UI connects to the server, it starts talking to a component in the main
process. It delegates various communication over IPC to another server in
the tab's frame script, but that forwarding process is hidden from the UI.

>
> I'm writing to ask how much effort it would be to modify our developer
> tools protocols, backend, and UI in the same way. Would it take a month,
> six months, or a year? How many people would be required?
>

It feels like a large project that affects most tools to _some_ degree, but
with varying amounts of effort required per tool depending on how they
choose to present their UI in the OOP frame world.

Is there a schedule for this work on the platform side at this stage?

> Who knows the most about the underlying issues here?
>

I would say myself and Alex Poirot know the most about the general tab
actor structure. Eddy Bruel added worker support, and we'd want to consider
what we learned from that.

I know less about the debugger myself, so people like James Long, Eddy
Bruel, Nick Fitzgerald, Jim Blandy, and Jason Laster would have a better
idea about that area.

> Which tools would need the most
> work? Are there any tools where this just isn't possible or wouldn't make
> sense?
>

Unsure which need the _most_ work yet... I think there's a path forward for
every tool, assuming we want to do the work to convert each one to the OOP
frame world (and we might decide to drop tools with very low usage if their
conversion cost is too great).

- Ryan

[J. Ryan Stinnett]

On Fri, Aug 26, 2016 at 5:17 PM, Nick Fitzgerald <nfitz...@mozilla.com>
wrote:

> We already have a iframe/context chooser[0] and I imagine we could extend
> it for this case.
>

Right, we could use that concept, and it might be okay for some tools. I
imagine we'll have other tools that still want to display a unified view
though, so it might be a bit of a hybrid world.

> An alternative option would be to treat cross origin iframes like how we
> treat workers, where we have a list of them on the sidebar and can open
> whole new toolboxes when we want to attach to them.
>

Do we have any info (even anecdotes!) on how users perceive multiple
toolboxes for different contexts? I am bit wary of a new toolbox for a
frame (sounds heavy weight to me), but I also have not done much worker
debugging lately (where we use this concept already).

- Ryan

[Bill McCloskey]

On Fri, Aug 26, 2016 at 3:17 PM, Nick Fitzgerald <nfitz...@mozilla.com>
wrote:

> TabActor wishes it was more like RelatedBrowsingContextActor. AFAICT, any
> JS within a related browsing context has to run sequentially -- that's not
> changing is it?
>

No, that's not changing.

> A cross origin iframe would be a different context (right?), and I think
> we would be fine not having it observable at the same time as the top level
> window. With those things in mind, this seems like a step in the correct
> direction.
>
> (Warning: bit of an off the cuff braindump incoming)
>

> We already have a iframe/context chooser[0] and I imagine we could extend
> it for this case.
>

> [0] http://i.imgur.com/7DvON6B.png
>

OK, that's good. I'm assuming the user can switch back and forth during
debugging. It seems like the hard case to support is when they pause one
iframe at a breakpoint or something, switch to another frame that's still
running, and maybe independently stop the second iframe at another
breakpoint. How would the existing code work in that sort of scenario?

Although maybe we wouldn't want to present that kind of complexity to the
user. When one frame stops, we could pause all the others as well. I wonder
if there are any weird edge cases if we did that though. If the user steps
across a postMessage call and has a breakpoint in the message event handler
in another frame, we would probably want to make sure that we break in the
message event handler and not just in the instruction after postMessage.
I'm not sure how to guarantee that.

> ​Another question is protocol compatibility with old servers and start up
> time if we add an indirection/protocol round trip after getting tabs to get
> contexts within the tab.
>

> An alternative option would be to treat cross origin iframes like how we
> treat workers, where we have a list of them on the sidebar and can open
> whole new toolboxes when we want to attach to them.
>

> On Fri, Aug 26, 2016 at 2:54 PM, Bill McCloskey <wmccl...@mozilla.com>
> wrote:
>
>> Hi everyone,
>>
>> Recently on the platform team we've started discussing the practicality of
>> running iframes in isolation from the main page. This might either take
>> the
>> form of running them in separate processes or separate threads. Either
>> way,
>> JS for the iframe could be running at the same time as JS for the main
>> page, although only if they're not same-origin.
>>
>> There's some platform work needed to do this, but also a good chunk of
>> work
>> to convert frame scripts. Right now, frame scripts run with chrome
>> privileges and have access to all frames in a tab simultaneously. We would
>> need to modify our scripts and APIs so that they run on a per-frame basis
>> and then aggregate the data from all the frames together in the chrome
>> process. For most scripts this would be relatively straightforward,
>> although it would take some time to get right.
>>

>> Developer tools pose a special challenge here. IIRC, they use a small
>> frame
>> script that then loads a bunch of devtools code to implement the tab
>> actor.
>> So we would need to change them to use frame actors instead, and then
>> modify all the tools to work with frame actors. This seems like a lot of
>> work.
>>
>> I took a look at Chrome's devtools since they already implement an
>> experimental out-of-process iframe feature. It looks like the JS tools
>> (debugger and console) require you to choose a particular frame in which
>> to
>> evaluate code or set a breakpoint. On the other hand, tools like the
>> inspector try to present a unified view of the page by aggregating data
>> from different frames.
>>

>> I'm writing to ask how much effort it would be to modify our developer
>> tools protocols, backend, and UI in the same way. Would it take a month,

>> six months, or a year? How many people would be required? Who knows the
>> most about the underlying issues here? Which tools would need the most

>> work? Are there any tools where this just isn't possible or wouldn't make
>> sense?
>>

[Nick Fitzgerald]

On Fri, Aug 26, 2016 at 3:46 PM, Bill McCloskey <wmccl...@mozilla.com>
wrote:
​

OK, that's good. I'm assuming the user can switch back and forth during
> debugging. It seems like the hard case to support is when they pause one
> iframe at a breakpoint or something, switch to another frame that's still
> running, and maybe independently stop the second iframe at another
> breakpoint. How would the existing code work in that sort of scenario?
>
> Although maybe we wouldn't want to present that kind of complexity to the
> user. When one frame stops, we could pause all the others as well. I wonder
> if there are any weird edge cases if we did that though. If the user steps
> across a postMessage call and has a breakpoint in the message event handler
> in another frame, we would probably want to make sure that we break in the
> message event handler and not just in the instruction after postMessage.
> I'm not sure how to guarantee that.
>

​IIRC, when a worker or the main thread pauses, we race to pause the other
threads before they hit their own breakpoint. Eddy can confirm or deny this
and get into more details.​

[James Long]

No replying to any specific point, but the new debugger is equipped to
handle this case potentially well because we've been expecting to support
debugging workers in the same toolbox. We don't have that yet, and need to
figure out a lot of the UX issues, but those are the same UX we would deal
with here. Technically, the code can handle multiple connections very well
though.

On Friday, August 26, 2016, Nick Fitzgerald <nfitz...@mozilla.com> wrote:

> On Fri, Aug 26, 2016 at 3:46 PM, Bill McCloskey <wmccl...@mozilla.com

> <javascript:;>>

> _______________________________________________
> dev-developer-tools mailing list
> dev-devel...@lists.mozilla.org <javascript:;>
> https://lists.mozilla.org/listinfo/dev-developer-tools
>

[David Bruant]

Hi Bill,

Le 26/08/2016 à 23:54, Bill McCloskey a écrit :
> Recently on the platform team we've started discussing the practicality of
> running iframes in isolation from the main page. This might either take the
> form of running them in separate processes or separate threads. Either way,
> JS for the iframe could be running at the same time as JS for the main
> page, although only if they're not same-origin.

I'm happy to read this as I'm quite interested in seeing this happen as
a web developer :-) [1]
Where can I read more about these discussions you've had on the platform
team?

Thanks,

David

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=961689

[Jim Blandy]

On Fri, Aug 26, 2016 at 11:00 PM, James Long <jl...@mozilla.com> wrote:

> No replying to any specific point, but the new debugger is equipped to
> handle this case potentially well because we've been expecting to support
> debugging workers in the same toolbox. We don't have that yet, and need to
> figure out a lot of the UX issues, but those are the same UX we would deal
> with here. Technically, the code can handle multiple connections very well
> though.

This relates to a question that came up last week: what sorts of issues do
you foresee in supporting simultaneous Firefox and Chrome protocol
connections from our UI to our server? I've been assuming that they'd just
be two entirely separate connections, and UI components would interact with
one or the other as necessary.

Difficulties will arise when one protocol assumes that it is the only thing
messing with the debuggee/document/thing being observed, and thus has no
way to cope with effects brought about through the other protocol. But I
don't know if there are any such situations.

[Ehsan Akhgari]

On 2016-08-26 6:46 PM, Bill McCloskey wrote:
> On Fri, Aug 26, 2016 at 3:17 PM, Nick Fitzgerald
> <nfitz...@mozilla.com <mailto:nfitz...@mozilla.com>> wrote:
>
> TabActor wishes it was more like RelatedBrowsingContextActor.
> AFAICT, any JS within a related browsing context has to run
> sequentially -- that's not changing is it?
>
>
> No, that's not changing.

I'm not familiar with the terminology here, but I read above as "JS
within an iframe and within a page will always run sequentially."

Assuming I understand that correctly, I think we should design things in
a way that we can change that in the future. Here is the world I would
like us to end up in:

We will load *all* cross-origin iframes in a separate process. We can
start by sharing a single process for all of them, which is what Chrome
is working on shipping at first. Note that with that model, we will
essentially have two main threads, which run concurrently, so JS code in
the iframe can start to run in the middle of JS code that is running on
the main page, and vice versa.

There are a few cases where there is a cross-origin method needs to
occur synchronously. Obviously we can't call into the other process
when it's actually running JS, but we'd need to protect things such as
nested event loops and whatnot so that you won't receive a cross-process
call asking to run some JS in your context while an alert() is open,
etc. So from the viewpoint of each side, calls made from the other
process will be serialized with the JS running within its own context,
but otherwise the two main threads will run concurrently which should
not be observable.

One devtools related question there is, would the devtools code make us
have to support more cross-origin cross-process synchronous calls than
is exposed on the Web. If we need to support other cross-origin
features for devtools, it would be nice to know now!

Thanks,
Ehsan

[Bill McCloskey]

On Wed, Aug 31, 2016 at 10:26 AM, Ehsan Akhgari <ehsan....@gmail.com>
wrote:

Sorry, you're right. I just re-read the spec, and it looks like I made a
mistake. The unit we eventually want an actor to be 1:1 with is a "unit of
related similar-origin browsing contexts" [1]. As Ehsan has noted, the
"similar-origin" part is really important because it allows us to run
different iframes on the same page in different threads. That's really the
crucial thing we're asking about: how much work would it be for devtools to
support this.

It seems like we've mostly heard about the JS side. I'm curious about other
tools: inspector, performance tools, etc.

[1]
https://www.w3.org/TR/html51/browsers.html#units-of-related-similar-origin-browsing-contexts

[James Long]

On Mon, Aug 29, 2016 at 5:13 PM, Jim Blandy <jbl...@mozilla.com> wrote:
>
>
> This relates to a question that came up last week: what sorts of issues do
> you foresee in supporting simultaneous Firefox and Chrome protocol
> connections from our UI to our server? I've been assuming that they'd just
> be two entirely separate connections, and UI components would interact with
> one or the other as necessary.
>
> Difficulties will arise when one protocol assumes that it is the only
> thing messing with the debuggee/document/thing being observed, and thus has
> no way to cope with effects brought about through the other protocol. But I
> don't know if there are any such situations.
>
>

I agree that there would be difficulties, although currently we do support
multiple connections to a single tab. Not sure how well-defined that is,
but it does seem to work pretty well.

However, I don't think that's what I was describing. I was thinking that
multiple connections would be to distinct contexts, such as workers, which
have their own debugger server and everything (no page context). I was
thinking if frames were isolated and we had a connection per frame, none of
the connections would overlap. But maybe I misunderstood something.

[Jim Blandy]

On Wed, Aug 31, 2016 at 12:40 PM, James Long <jl...@mozilla.com> wrote:

> On Mon, Aug 29, 2016 at 5:13 PM, Jim Blandy <jbl...@mozilla.com> wrote:
>
>>
>> This relates to a question that came up last week: what sorts of issues
>> do you foresee in supporting simultaneous Firefox and Chrome protocol
>> connections from our UI to our server? I've been assuming that they'd just
>> be two entirely separate connections, and UI components would interact with
>> one or the other as necessary.
>>
>> Difficulties will arise when one protocol assumes that it is the only
>> thing messing with the debuggee/document/thing being observed, and thus has
>> no way to cope with effects brought about through the other protocol. But I
>> don't know if there are any such situations.
>>
>>
> I agree that there would be difficulties, although currently we do support
> multiple connections to a single tab. Not sure how well-defined that is,
> but it does seem to work pretty well.
>
> However, I don't think that's what I was describing.
>

Yes - I was hijacking the thread to answer a distinct question that I was
wondering about: how well does the new devtools.html code support having
tools within a single toolbox mix Chrome and Firefox protocols to do their
work?

[James Long]

Relatively well; it's what you initially described. They are entirely
separate connections, and and actions will "pick" one of them to do the
desired work with, the local state will reflect the result of all protocol
calls, and the UI just display the local state. Basically: the interaction
with the protocol is pushed off to the edge of the program, so it's very
easy to dynamically swap them out even for every single action (get source
text, etc).

[Jim Blandy]

That's what I was looking for (and hoping to hear). Thanks very much!

[Patrick Brosset]

About the iframe switcher tool that currently exists in devtools:

It's handy in some cases to users who want to filter out some things and
only focus on one iframe (although I doubt that it's used very often).
Maybe they want to see a smaller DOM tree, maybe less scripts to debug, but
it's seen as just a way for them to make their lives easier, not as
something they *must* use it to see parts of the DOM tree or scripts they
wouldn't see otherwise.
So we shouldn't require them to use that tool and always strive to present
a unified view if we can.

For instance, we did introduce a way to spawn a new toolbox to debug
workers but IIRC, that was just a temporary decision driven by the fact
that we just couldn't do it differently. In terms of UX, it'd be so much
better to see all scripts and be able to set breakpoints to them in the one
same debugger.

In the inspector, we definitely want to present a unified view of the page.
That is what we do today, and forcing people to have to switch to another
context would be very frustrating and perceived badly.
So I agree with what Ryan said, in that we'll need to aggregate the data
from different frames on our backend, and still provide just one TabActor
to the inspector front-end code.

Right now, the inspector backend uses an instance of inIDeepTreeWalker to
walk the full DOM of the page, including going into iframes and retrieving
pseudo-elements.
It also uses a MutationObserver on each document node to be alerted of
changes in all subtrees.

So that part would have to change. I expect it to remain the same for the
DOM in the parent document and in all same-origin iframes, but for
cross-origin iframes, we're going to have to start listening for messages
from actors in other processes.
This doesn't sound terribly hard to do, but will take time to get right,
and I can see one specific impact: right now we wait until the full DOM is
ready before telling the inspector that it can initialize. With this
change, we'd have to possibly load the DOM of cross-origin iframes
independently.

> _______________________________________________
> dev-developer-tools mailing list
> dev-devel...@lists.mozilla.org

> https://lists.mozilla.org/listinfo/dev-developer-tools
>