Sync frame tree node ids?

[Dominic Mazzoni]

Question that I asked during the site isolation summit:

Would it be possible to sync frame tree node ids to each RenderFrame?

For accessibility, we need to replicate a tree of accessible elements for each frame, and reassemble these to create one virtual accessibility tree. Right now when a frame has an iframe element in it, it passes the routing id of the child frame as part of that accessible element. Then in the browser process we need to convert that routing id to a frame tree node id.

It could simplify things if we could just stick with a frame tree node id.

[Charlie Reis]

Are you referring to code like this?

void RenderFrameHostImpl::UpdateCrossProcessIframeAccessibility(
    const std::map<int32, int>& node_to_frame_routing_id_map) {
  for (const auto& iter : node_to_frame_routing_id_map) {
    // This is the id of the accessibility node that has a child frame.
    int32 node_id = iter.first;
    // The routing id from either a RenderFrame or a RenderFrameProxy.
    int frame_routing_id = iter.second;

    FrameTree* frame_tree = frame_tree_node()->frame_tree();
    FrameTreeNode* child_frame_tree_node = frame_tree->FindByRoutingID(
        GetProcess()->GetID(), frame_routing_id);
    if (child_frame_tree_node) {
      FrameAccessibility::GetInstance()->AddChildFrame(
          this, node_id, child_frame_tree_node->frame_tree_node_id());
    }
  }
}

I wonder if adding a RenderFrameHostImpl::GetFrameTreeNodeID(int frame_routing_id) would hide the boilerplate code there and make it easier to read?  The above chunk of code would become:

    int64 ftn_id = GetFrameTreeNodeID(frame_routing_id);

    if (ftn_id != MSG_ROUTING_NONE)

      FrameAccessibility::GetInstance()->AddChildFrame(this, node_id, ftn_id);

The new helper function would limit its search to the RFH's process and frame tree, as we were doing before.

To be clear, I'm not completely opposed to your suggestion of replicating the FTN ID, but there are a few reasons to avoid that if we can:

1) The renderer can't know the FTN ID at all times.  When the frame is first constructed, we use a sync IPC to get a routing ID from the browser's IO thread, but the FTN ID isn't allocated until later on the browser's UI thread.  This would introduce some complexity with having to check whether the FTN ID is known in the renderer or not.

2) There are some security concerns with letting a renderer name a FTN ID, which could refer to any frame in the entire browser process.  The ease of passing them around might be outweighed by having to validate them.

3) It would be nice to keep FTN IDs in a higher layer, only known in the browser process.  The renderer doesn't have much use for them itself, beyond echoing them back to the browser.

In that sense, making it as easy as possible to map between routing ID and FTN ID in the browser seems desirable to me.

Thoughts?

Charlie

[Dominic Mazzoni]

Answers below:

On Tue, Jan 27, 2015 at 3:03 PM, Charlie Reis <cr...@chromium.org> wrote:

Are you referring to code like this?

void RenderFrameHostImpl::UpdateCrossProcessIframeAccessibility(
    const std::map<int32, int>& node_to_frame_routing_id_map) {
  for (const auto& iter : node_to_frame_routing_id_map) {
    // This is the id of the accessibility node that has a child frame.
    int32 node_id = iter.first;
    // The routing id from either a RenderFrame or a RenderFrameProxy.
    int frame_routing_id = iter.second;

    FrameTree* frame_tree = frame_tree_node()->frame_tree();
    FrameTreeNode* child_frame_tree_node = frame_tree->FindByRoutingID(
        GetProcess()->GetID(), frame_routing_id);
    if (child_frame_tree_node) {
      FrameAccessibility::GetInstance()->AddChildFrame(
          this, node_id, child_frame_tree_node->frame_tree_node_id());
    }
  }
}

I wonder if adding a RenderFrameHostImpl::GetFrameTreeNodeID(int frame_routing_id) would hide the boilerplate code there and make it easier to read?  The above chunk of code would become:

    int64 ftn_id = GetFrameTreeNodeID(frame_routing_id);

    if (ftn_id != MSG_ROUTING_NONE)

      FrameAccessibility::GetInstance()->AddChildFrame(this, node_id, ftn_id);

The new helper function would limit its search to the RFH's process and frame tree, as we were doing before.

Yes, that's the code I was talking about.

I'm not worried about it being hard to read - helper functions to make it more straightforward with less boilerplate sounds great but wouldn't really change the design.

Basically, if the frame tree node id was available from the renderer, then we wouldn't need to do the translation at all - the accessibility message would be self-contained and would contain all of the information needed to form a complete accessibility tree spanning frames.

This would be particularly helpful for Chrome OS, where the accessibility messages get routed directly to ChromeVox running in an extension process, via an extension API. Currently we can pass the accessibility messages through unmodified, and someday with Mojo we may even be able to send them directly from the renderer to the extension process without even involving the browser process. In order to support OOPIF on Chrome OS, though, we'd need to unpickle the message in the browser process, turn render frame ids into frame tree node ids, and then pickle the message again.

More below:

 

To be clear, I'm not completely opposed to your suggestion of replicating the FTN ID, but there are a few reasons to avoid that if we can:

1) The renderer can't know the FTN ID at all times.  When the frame is first constructed, we use a sync IPC to get a routing ID from the browser's IO thread, but the FTN ID isn't allocated until later on the browser's UI thread.  This would introduce some complexity with having to check whether the FTN ID is known in the renderer or not.

I think it'd be okay if it's unknown for some short time as long as we can register a callback when the FTN id is available in the renderer.

 

2) There are some security concerns with letting a renderer name a FTN ID, which could refer to any frame in the entire browser process.  The ease of passing them around might be outweighed by having to validate them.

How is this different from a routing ID, though?

 

3) It would be nice to keep FTN IDs in a higher layer, only known in the browser process.  The renderer doesn't have much use for them itself, beyond echoing them back to the browser.

Every renderer has to know about the whole frame tree, though - it has to know the whole tree, and the name of each window, and stuff like that. I understand that there's a lot of stuff about FrameTreeNode that should be known only to the browser, but I don't see the harm in having a way to identify nodes in the frame tree that's the same across processes.

Note that it doesn't have to be a FTN ID - if we synced the process id and routing id of the *real* RenderFrame, that'd work just as well for us.

 

In that sense, making it as easy as possible to map between routing ID and FTN ID in the browser seems desirable to me.

It's not the end of the world if we can't do it, I can work with that. The current code for native accessibility on Win, Mac & Android is fine with me, it's just the Chrome OS case we'd like to add next that feels unfortunate since we're trying to piece together an accessibility tree outside of the browser process now.

I suppose we could try to sync the mappings between routing ids and FTN ids separately.

- Dominic

[Charlie Reis]

Thanks-- this is a useful example.  It's clear that in this case there's a burden to do the translation.

 

More below:

 

To be clear, I'm not completely opposed to your suggestion of replicating the FTN ID, but there are a few reasons to avoid that if we can:

1) The renderer can't know the FTN ID at all times.  When the frame is first constructed, we use a sync IPC to get a routing ID from the browser's IO thread, but the FTN ID isn't allocated until later on the browser's UI thread.  This would introduce some complexity with having to check whether the FTN ID is known in the renderer or not.

I think it'd be okay if it's unknown for some short time as long as we can register a callback when the FTN id is available in the renderer.

Sure, this would be workable, if not elegant.

 

 

2) There are some security concerns with letting a renderer name a FTN ID, which could refer to any frame in the entire browser process.  The ease of passing them around might be outweighed by having to validate them.

How is this different from a routing ID, though?

The important difference is that a routing ID can only refer to frames within a given process.  That's a much smaller set: just the frames in the tabs that the process already controls.  All other tabs (including other web pages as well as privileged pages like WebUI and apps) can't be named.

FrameTreeNode IDs are browser-global, so an exploited or buggy renderer process can name any frame in any tab.

I suppose we could add an IPC validation step that ensures any FTN IDs received from a renderer process belong to FrameTreeNodes in which the process has a RenderFrameHost.  That would still be an awkward step for the renderer -> extension path on ChromeOS, but I don't think we want to let an exploited renderer name arbitrary FrameTreeNodes.

 

3) It would be nice to keep FTN IDs in a higher layer, only known in the browser process.  The renderer doesn't have much use for them itself, beyond echoing them back to the browser.

Every renderer has to know about the whole frame tree, though - it has to know the whole tree, and the name of each window, and stuff like that. I understand that there's a lot of stuff about FrameTreeNode that should be known only to the browser, but I don't see the harm in having a way to identify nodes in the frame tree that's the same across processes.

Note that it doesn't have to be a FTN ID - if we synced the process id and routing id of the *real* RenderFrame, that'd work just as well for us.

If we do go this route, FTN ID will be better than process ID + routing ID of the current RenderFrame.  FTN ID is stable and doesn't change over time, but the process ID + routing ID could change with each navigation.

 

 

In that sense, making it as easy as possible to map between routing ID and FTN ID in the browser seems desirable to me.

It's not the end of the world if we can't do it, I can work with that. The current code for native accessibility on Win, Mac & Android is fine with me, it's just the Chrome OS case we'd like to add next that feels unfortunate since we're trying to piece together an accessibility tree outside of the browser process now.

I suppose we could try to sync the mappings between routing ids and FTN ids separately. 

- Dominic

What would you think about replicating FTN IDs but requiring a security check in the browser process before passing it in IPCs to other code (including on Chrome OS)?

Charlie

[Carlos Knippschild]

How about sending to the renderer only hashed versions of FTN IDs salted with randomness generated at startup?

We'd still need to keep a mapping from hashed ID to FTN ID but we'd solve two problems:
- The hashed IDs would be as stable as their FTN ID counterparts and would uniquely identify FTNs for all child processes.
- It would be really hard for renderers to guess FTN IDs they are not informed of.

The map could be avoided if we should choose a 2-way operation (encryption, permutation) instead of hashing but I think the extra complications and possible performance implications could make it a worse solution. I'm not a security expert though.

[Charlie Reis]

That seems like unnecessary complexity to me.  If we're going to use a mapping, we already have routing IDs that we can use.

(The hashing approach lets each renderer use the same ID for the frame, but I don't think we need that property here.)

Charlie

[Dominic Mazzoni]

On Tue, Jan 27, 2015 at 6:08 PM, Charlie Reis <cr...@chromium.org> wrote:

That seems like unnecessary complexity to me.  If we're going to use a mapping, we already have routing IDs that we can use.

(The hashing approach lets each renderer use the same ID for the frame, but I don't think we need that property here.)

Actually I think that's exactly the property I do want for accessibility - I want an unique ID for each frame independent of the process.

This discussion has given me some ideas, though. It sounds like we could solve half of the concerns just by using a separate frame accessibility ID rather than the frame tree node ID. Also, I hadn't thought about the fact that the render process makes a Sync IPC to the browser to get a routing id - maybe we could use the same trick.

So instead of using frame tree node ids, let's have a separate frame accessibility ID for each frame. This wouldn't be a security issue because there's nothing a renderer could do if it "guessed" the accessibility ID of a different frame. There are no accessibility APIs for posting messages or retrieving data from other frames on the renderer side.

When we encounter an iframe on the renderer side, we could make an IPC to the browser once to convert the iframe's routing id to its accessibility ID. It wouldn't even have to be a sync IPC. Then we could cache the mapping from (process-local) routing ID to accessibility ID and we wouldn't have to pay the cost of the IPC again from that process.

Any potential concerns about this idea, did I explain it well enough?

I'll play with it and see how it works out.

- Dominic

[Daniel Cheng]

Some thoughts after catching up on this thread.

1) Earlier, it was mentioned that Mojo might allow a renderer to talk directly to an extension process. Is this an explicit goal for the Mojo project to allow direct IPCs between renderers? We were explicit that renderers should not talk to other renderers when working out the design for OOPIF, and I think that routing events via the browser is a good thing for security and auditability.

2) My understanding of accessibility is somewhat incomplete, but it seems like a malicious page that compromises the renderer could still do some shenanigans with accessibility IDs. For example:

    - Create an iframe and point it at a phishing page.

    - window.open something and navigate it to the login page of something important

    - Use the routing ID -> accessibility ID IPC to get the accessibility ID of the login page.

    - Tell the browser that the malicious page has an iframe with the accessibility ID of the login page.

Daniel

[Dominic Mazzoni]

On Tue, Jan 27, 2015 at 11:30 PM, Daniel Cheng <dch...@chromium.org> wrote:

Some thoughts after catching up on this thread.

1) Earlier, it was mentioned that Mojo might allow a renderer to talk directly to an extension process. Is this an explicit goal for the Mojo project to allow direct IPCs between renderers? We were explicit that renderers should not talk to other renderers when working out the design for OOPIF, and I think that routing events via the browser is a good thing for security and auditability.

I think a goal of Mojo is that direct IPCs should be *possible* - that's very different from saying we should necessarily allow IPCs from one particular pair of processes.

 

2) My understanding of accessibility is somewhat incomplete, but it seems like a malicious page that compromises the renderer could still do some shenanigans with accessibility IDs. For example:

    - Create an iframe and point it at a phishing page.

    - window.open something and navigate it to the login page of something important

    - Use the routing ID -> accessibility ID IPC to get the accessibility ID of the login page.

    - Tell the browser that the malicious page has an iframe with the accessibility ID of the login page.

5. ???

6. Profit

I can't figure out a way that they could do something malicious. All they could do is potentially get the browser to construct a composed accessibility tree with the wrong iframe as the child of a compromised parent frame. The effect would be no different than if the parent frame just inserted an <iframe> element. It wouldn't give the compromised frame the ability to reach into that iframe at all, it simply allows an external accessibility client to walk from one frame to another frame as if they're part of the same tree.

Remember, an accessibility client is like an agent of the user. It can already steal your keystrokes and read your passwords.

Furthermore, it seems like we could mitigate this type of maliciousness without needing to query the browser process at all - if every frame reports its own accessibility id and the accessibility id of its parent frame and child frames, then we could easily detect if these are being used inconsistently - like if frame 7 is a child of frame 6, but then subsequently we get a request for frame 7 to be a child of frame 8 too, we could reject that.

- Dominic

[Daniel Cheng]

On Wed Jan 28 2015 at 9:24:23 AM Dominic Mazzoni <dmaz...@google.com> wrote:

On Tue, Jan 27, 2015 at 11:30 PM, Daniel Cheng <dch...@chromium.org> wrote:

Some thoughts after catching up on this thread.

1) Earlier, it was mentioned that Mojo might allow a renderer to talk directly to an extension process. Is this an explicit goal for the Mojo project to allow direct IPCs between renderers? We were explicit that renderers should not talk to other renderers when working out the design for OOPIF, and I think that routing events via the browser is a good thing for security and auditability.

I think a goal of Mojo is that direct IPCs should be *possible* - that's very different from saying we should necessarily allow IPCs from one particular pair of processes.

 

2) My understanding of accessibility is somewhat incomplete, but it seems like a malicious page that compromises the renderer could still do some shenanigans with accessibility IDs. For example:

    - Create an iframe and point it at a phishing page.

    - window.open something and navigate it to the login page of something important

    - Use the routing ID -> accessibility ID IPC to get the accessibility ID of the login page.

    - Tell the browser that the malicious page has an iframe with the accessibility ID of the login page.

5. ???

6. Profit

I can't figure out a way that they could do something malicious. All they could do is potentially get the browser to construct a composed accessibility tree with the wrong iframe as the child of a compromised parent frame. The effect would be no different than if the parent frame just inserted an <iframe> element. It wouldn't give the compromised frame the ability to reach into that iframe at all, it simply allows an external accessibility client to walk from one frame to another frame as if they're part of the same tree.

It's not quite the same. X-Frame-Options might prevent framing of that login page normally, but because the renderer forged the accessibility ID, the accessibility client believes it to be in the iframe. And perhaps this causes the user to enter sensitive information in the iframe, believing it to be something different from what it actually is.

 

Remember, an accessibility client is like an agent of the user. It can already steal your keystrokes and read your passwords.

Furthermore, it seems like we could mitigate this type of maliciousness without needing to query the browser process at all - if every frame reports its own accessibility id and the accessibility id of its parent frame and child frames, then we could easily detect if these are being used inconsistently - like if frame 7 is a child of frame 6, but then subsequently we get a request for frame 7 to be a child of frame 8 too, we could reject that.

If we need to validate anyway, it seems like we might as well using routing ID.

[Dominic Mazzoni]

Hey, I'd like to revisit this thread, because I've got another accessibility use-case (http://crbug.com/472704) that's difficult to solve as currently architected, and I still haven't come up with a solution for creating a composed accessibility tree on Chrome OS.

Please re-read this thread for context, but here's a brief restatement of the problem:

Given the accessibility tree for a bunch of different frames, we need to be able to reconstruct a single *composed* accessibility tree.

It's not sufficient to just look at the frame tree in the browser process, because that just says which frames are children of which other frames - it doesn't say which node in the tree each frame hangs off of, which we need

What we've implemented currently is that in the render process while generating the accessibility tree for a parent frame, for each iframe element it reports the routing id of that child frame. Then in the browser process we map those routing ids to frame tree node ids when constructing the native accessibility tree.

The number one problem we haven't solved yet is that for Chrome OS accessibility we need to be able to construct that composed accessibility tree in a different process - in the ChromeVox extension process, not the browser process. Currently that's impossible because the routing id of a remote frame is useless without any way to map that to the routing id of the local frame in the process of that frame.

The secondary issue - less important than the Chrome OS one but still relevant - is that we're trying to move most accessibility code out of content/ - it'd be nice if the code to construct a composed accessibility tree didn't require making calls into content/browser/frame_host to map routing ids. Ideally we should be able to get a bunch of accessibility trees that just reference each other by some unique id, and it'd be trivial to compose them based on those ids alone, from any process and from any part of the codebase (not only in content/). This is relevant to the bug I mentioned above, 472704.

My proposal, again, is for each RenderFrame to query the frame tree node id (or some globally-unique frame accessibility id) for any frame in its own frame tree, from the browser process. Then the accessibility tree it generates can encode its own id, its parent frame's id, and the ids of any child frames.

See below for one more follow-up to this old thread:

On Wed, Jan 28, 2015 at 9:54 AM, Daniel Cheng <dch...@chromium.org> wrote:

2) My understanding of accessibility is somewhat incomplete, but it seems like a malicious page that compromises the renderer could still do some shenanigans with accessibility IDs. For example:

    - Create an iframe and point it at a phishing page.

    - window.open something and navigate it to the login page of something important

    - Use the routing ID -> accessibility ID IPC to get the accessibility ID of the login page.

    - Tell the browser that the malicious page has an iframe with the accessibility ID of the login page.

5. ???

6. Profit

I can't figure out a way that they could do something malicious. All they could do is potentially get the browser to construct a composed accessibility tree with the wrong iframe as the child of a compromised parent frame. The effect would be no different than if the parent frame just inserted an <iframe> element. It wouldn't give the compromised frame the ability to reach into that iframe at all, it simply allows an external accessibility client to walk from one frame to another frame as if they're part of the same tree.

It's not quite the same. X-Frame-Options might prevent framing of that login page normally, but because the renderer forged the accessibility ID, the accessibility client believes it to be in the iframe. And perhaps this causes the user to enter sensitive information in the iframe, believing it to be something different from what it actually is.

Remember, an accessibility client is like an agent of the user. It can already steal your keystrokes and read your passwords.

Furthermore, it seems like we could mitigate this type of maliciousness without needing to query the browser process at all - if every frame reports its own accessibility id and the accessibility id of its parent frame and child frames, then we could easily detect if these are being used inconsistently - like if frame 7 is a child of frame 6, but then subsequently we get a request for frame 7 to be a child of frame 8 too, we could reject that.

If we need to validate anyway, it seems like we might as well using routing ID.

The reason routing IDs don't work is because multiple sets of (process ID, routing ID) pairs refer to the same frame - one set for each process that's involved in that frame tree. The only way to get them all straight is by querying code in the content module in the browser process. My goal is that we can encode that information at the time we build the accessibility tree in the render process, so that anyone who receives that accessibility tree later has all of the information they need to reconstruct the full composed tree.

Anyway, I think that validating both parents and children should address any potential concerns about malicious use. If an attacker has compromised a parent frame and they try to insert a child frame into the accessibility tree by using its ID, we'd know this is invalid because the child frame would also have an encoding of its parent frame's ID and we'd see that they don't match.

Still, it might be prudent to use a separate accessibility frame id rather than reusing the frame tree node id. That way we don't have to worry about any other potential harm that could be done by knowing a frame tree node id in a renderer.

[Charlie Reis]

[+alexmos, lazyboy]

Sorry for the delay; I wanted to read through the thread again.

Your current proposal sounds promising to me: we can replicate a frame accessibility ID across all processes that know about the frame, so that they all have the same ID for it.  That would let you compose the tree from subtrees in either the browser process or an extension process.  It does sound plausible that we could validate the IDs by ensuring both the parent and child agree about any embedding relationship.

A few notes and questions:

1) Alex has recently added FrameReplicationState for this basic purpose: keeping a set of common state across a RenderFrame and all of its RenderFrameProxies in other processes.  This seems like the right mechanism for storing the frame accessibility ID.  We haven't needed to expose it outside content yet, but it's intended for general use and we were expecting to expose some APIs for it eventually.  You can chat with Alex about how this might look.

2) Is there a need for the frame accessibility to be globally unique, or can it be somehow scoped to an overall page?  It doesn't seem like a frame would need the ability to point to a frame in a completely different tab, for example.  (I would suggest having it specific to a WebContents, but lazyboy@ is working on having one WebContents be a child of another for building <webview> on top of OOPIF.)  Scoping the IDs isn't a strict requirement thanks to the ability to validate them, but it seems like it might avoid certain types of bugs.

Hope that helps,

Charlie

[Dominic Mazzoni]

On Fri, Apr 10, 2015 at 9:39 AM, Charlie Reis <cr...@chromium.org> wrote:

1) Alex has recently added FrameReplicationState for this basic purpose: keeping a set of common state across a RenderFrame and all of its RenderFrameProxies in other processes.  This seems like the right mechanism for storing the frame accessibility ID.  We haven't needed to expose it outside content yet, but it's intended for general use and we were expecting to expose some APIs for it eventually.  You can chat with Alex about how this might look.

FrameReplicationState looks like it could work, thanks. I don't need the whole state outside content, I just need to sync a globally-unique accessibility frame id within content and then expose just that id outside of content.

I'll give this a try.

 

2) Is there a need for the frame accessibility to be globally unique, or can it be somehow scoped to an overall page?  It doesn't seem like a frame would need the ability to point to a frame in a completely different tab, for example.  (I would suggest having it specific to a WebContents, but lazyboy@ is working on having one WebContents be a child of another for building <webview> on top of OOPIF.)  Scoping the IDs isn't a strict requirement thanks to the ability to validate them, but it seems like it might avoid certain types of bugs.

Keep in mind that there are lots of uses of WebContents outside of a tab, too - app windows, pop-ups, etc. - so at first glance this seems like it'd add a lot of complication.

For now I think that global IDs plus checking both the child and parent direction every time makes the most sense to me.

Thanks!

[Daniel Cheng]

Some thoughts inline. Sorry for the late reply.

On Fri, Apr 10, 2015 at 10:00 AM Dominic Mazzoni <dmaz...@google.com> wrote:

On Fri, Apr 10, 2015 at 9:39 AM, Charlie Reis <cr...@chromium.org> wrote:

1) Alex has recently added FrameReplicationState for this basic purpose: keeping a set of common state across a RenderFrame and all of its RenderFrameProxies in other processes.  This seems like the right mechanism for storing the frame accessibility ID.  We haven't needed to expose it outside content yet, but it's intended for general use and we were expecting to expose some APIs for it eventually.  You can chat with Alex about how this might look.

FrameReplicationState looks like it could work, thanks. I don't need the whole state outside content, I just need to sync a globally-unique accessibility frame id within content and then expose just that id outside of content.

I'll give this a try.

 

2) Is there a need for the frame accessibility to be globally unique, or can it be somehow scoped to an overall page?  It doesn't seem like a frame would need the ability to point to a frame in a completely different tab, for example.  (I would suggest having it specific to a WebContents, but lazyboy@ is working on having one WebContents be a child of another for building <webview> on top of OOPIF.)  Scoping the IDs isn't a strict requirement thanks to the ability to validate them, but it seems like it might avoid certain types of bugs.

Keep in mind that there are lots of uses of WebContents outside of a tab, too - app windows, pop-ups, etc. - so at first glance this seems like it'd add a lot of complication.

For now I think that global IDs plus checking both the child and parent direction every time makes the most sense to me.

Thanks!

 

Hope that helps,

Charlie

On Wed, Apr 8, 2015 at 11:57 PM, Dominic Mazzoni <dmaz...@google.com> wrote:

Hey, I'd like to revisit this thread, because I've got another accessibility use-case (http://crbug.com/472704) that's difficult to solve as currently architected, and I still haven't come up with a solution for creating a composed accessibility tree on Chrome OS.

Please re-read this thread for context, but here's a brief restatement of the problem:

Given the accessibility tree for a bunch of different frames, we need to be able to reconstruct a single *composed* accessibility tree.

It's not sufficient to just look at the frame tree in the browser process, because that just says which frames are children of which other frames - it doesn't say which node in the tree each frame hangs off of, which we need

What we've implemented currently is that in the render process while generating the accessibility tree for a parent frame, for each iframe element it reports the routing id of that child frame. Then in the browser process we map those routing ids to frame tree node ids when constructing the native accessibility tree.

The number one problem we haven't solved yet is that for Chrome OS accessibility we need to be able to construct that composed accessibility tree in a different process - in the ChromeVox extension process, not the browser process. Currently that's impossible because the routing id of a remote frame is useless without any way to map that to the routing id of the local frame in the process of that frame.

My understanding is that this still has to flow from renderer -> browser -> extension today. So this doesn't seem like a blocking issue.

 

The secondary issue - less important than the Chrome OS one but still relevant - is that we're trying to move most accessibility code out of content/ - it'd be nice if the code to construct a composed accessibility tree didn't require making calls into content/browser/frame_host to map routing ids. Ideally we should be able to get a bunch of accessibility trees that just reference each other by some unique id, and it'd be trivial to compose them based on those ids alone, from any process and from any part of the codebase (not only in content/). This is relevant to the bug I mentioned above, 472704.

Having a frame tree is a content concept. If you're assembling accessibility trees for different frames, it makes sense that you'd need to go into content.

 

My proposal, again, is for each RenderFrame to query the frame tree node id (or some globally-unique frame accessibility id) for any frame in its own frame tree, from the browser process. Then the accessibility tree it generates can encode its own id, its parent frame's id, and the ids of any child frames.

How will the accessibility frame ID be validated? Isn't the browser process is the only thing that can do this check reliably, unless you plan on mirroring all the frame trees into ChromeVox as well?

[Dominic Mazzoni]

On Fri, Apr 10, 2015 at 10:37 AM, Daniel Cheng <dch...@chromium.org> wrote:

The number one problem we haven't solved yet is that for Chrome OS accessibility we need to be able to construct that composed accessibility tree in a different process - in the ChromeVox extension process, not the browser process. Currently that's impossible because the routing id of a remote frame is useless without any way to map that to the routing id of the local frame in the process of that frame.

My understanding is that this still has to flow from renderer -> browser -> extension today. So this doesn't seem like a blocking issue.

That's true, but doing that mapping on the browser side would mean unpacking a data structure, mapping ids, and packing it again, and it'd mean that the AX tree data structure would have to include a field for the routing id and for the mapped frame id. So, it's not impossible to implement but it feels like a poor design in terms of leaky abstractions and performance - especially considering that in the future a mojo IPC might be able to take us directly from renderer -> extension.

The secondary issue - less important than the Chrome OS one but still relevant - is that we're trying to move most accessibility code out of content/ - it'd be nice if the code to construct a composed accessibility tree didn't require making calls into content/browser/frame_host to map routing ids. Ideally we should be able to get a bunch of accessibility trees that just reference each other by some unique id, and it'd be trivial to compose them based on those ids alone, from any process and from any part of the codebase (not only in content/). This is relevant to the bug I mentioned above, 472704.

Having a frame tree is a content concept. If you're assembling accessibility trees for different frames, it makes sense that you'd need to go into content.

The concept of embedding one accessibility tree in another exists outside of frames within content, though. We have an accessibility tree for the views in each window, and the views::WebView node within that tree references the accessibility tree for the web frame.

My proposal, again, is for each RenderFrame to query the frame tree node id (or some globally-unique frame accessibility id) for any frame in its own frame tree, from the browser process. Then the accessibility tree it generates can encode its own id, its parent frame's id, and the ids of any child frames.

How will the accessibility frame ID be validated? Isn't the browser process is the only thing that can do this check reliably, unless you plan on mirroring all the frame trees into ChromeVox as well?

Yes, all of the frame trees will be mirrored into ChromeVox. The validation will be checking that when following a link from a parent to a child, the child's parent id matches.

[Daniel Cheng]

On Fri, Apr 10, 2015 at 11:03 AM Dominic Mazzoni <dmaz...@google.com> wrote:

On Fri, Apr 10, 2015 at 10:37 AM, Daniel Cheng <dch...@chromium.org> wrote:

The number one problem we haven't solved yet is that for Chrome OS accessibility we need to be able to construct that composed accessibility tree in a different process - in the ChromeVox extension process, not the browser process. Currently that's impossible because the routing id of a remote frame is useless without any way to map that to the routing id of the local frame in the process of that frame.

My understanding is that this still has to flow from renderer -> browser -> extension today. So this doesn't seem like a blocking issue.

That's true, but doing that mapping on the browser side would mean unpacking a data structure, mapping ids, and packing it again, and it'd mean that the AX tree data structure would have to include a field for the routing id and for the mapped frame id. So, it's not impossible to implement but it feels like a poor design in terms of leaky abstractions and performance - especially considering that in the future a mojo IPC might be able to take us directly from renderer -> extension.

What does this data structure look like currently?

 

The secondary issue - less important than the Chrome OS one but still relevant - is that we're trying to move most accessibility code out of content/ - it'd be nice if the code to construct a composed accessibility tree didn't require making calls into content/browser/frame_host to map routing ids. Ideally we should be able to get a bunch of accessibility trees that just reference each other by some unique id, and it'd be trivial to compose them based on those ids alone, from any process and from any part of the codebase (not only in content/). This is relevant to the bug I mentioned above, 472704.

Having a frame tree is a content concept. If you're assembling accessibility trees for different frames, it makes sense that you'd need to go into content.

The concept of embedding one accessibility tree in another exists outside of frames within content, though. We have an accessibility tree for the views in each window, and the views::WebView node within that tree references the accessibility tree for the web frame.

My proposal, again, is for each RenderFrame to query the frame tree node id (or some globally-unique frame accessibility id) for any frame in its own frame tree, from the browser process. Then the accessibility tree it generates can encode its own id, its parent frame's id, and the ids of any child frames.

How will the accessibility frame ID be validated? Isn't the browser process is the only thing that can do this check reliably, unless you plan on mirroring all the frame trees into ChromeVox as well?

Yes, all of the frame trees will be mirrored into ChromeVox. The validation will be checking that when following a link from a parent to a child, the child's parent id matches.

I'd be a lot more comfortable if we only did this validation in one place (e.g. the browser).

In any case, I'll reserve further comments for the implementation. Maybe my concerns are unfounded =)

Daniel 

[Dominic Mazzoni]

On Fri, Apr 10, 2015 at 11:13 AM, Daniel Cheng <dch...@chromium.org> wrote:

That's true, but doing that mapping on the browser side would mean unpacking a data structure, mapping ids, and packing it again, and it'd mean that the AX tree data structure would have to include a field for the routing id and for the mapped frame id. So, it's not impossible to implement but it feels like a poor design in terms of leaky abstractions and performance - especially considering that in the future a mojo IPC might be able to take us directly from renderer -> extension.

What does this data structure look like currently?

Search for ui::AXTreeUpdate - on initial page load, it can sometimes contain tens of thousands of nodes; after that it's mostly small incremental updates.

Yes, all of the frame trees will be mirrored into ChromeVox. The validation will be checking that when following a link from a parent to a child, the child's parent id matches.

I'd be a lot more comfortable if we only did this validation in one place (e.g. the browser).

In any case, I'll reserve further comments for the implementation. Maybe my concerns are unfounded =)

Sounds good, I'll give it a try and reply here so everyone can weigh in.