Sharing FTN IDs

[Rob Wu]

Hi all,

I'm working on updating the frameId (used in the several extension APIs) for OOPIF (crbug.com/432875), which has the following characteristics:

1. The frameId must uniquely identify a frame within a tab, even across cross-process navigation.

2. The ID of the main frame is not important (it is always 0 for backwards-compatibility).

3. The frameId of the current frame and the parent frame must be known (webRequest / webNavigation API).

Currently frameId is based on the render frame ID, but it will be changed to the FTN ID. The FTN ID is currently not public yet, but to facilitate this change, the site isolation team has changed the type from int64 to int32 and expressed approval towards exposing the ID via RFH (crbug.com/432875#c7). I'll therefore add two methods (to retrieve a FTN ID from a RFH and vice versa).

Requests in the ResourceDispatcherHost can be identified by process ID + routing ID, but these cannot be mapped to a RFH because the RFH::FromID method is only useful on the IO thread. Consequently, this pair of IDs cannot be mapped to a FTN ID either. To solve this issue, I'd like to synchronize the FTN IDs with the RenderFrame in the renderer. This was brought up before at Sync frame tree node ids? [1], and was received with some (mild) opposition. With the publication of the FTN ID, I was wondering whether sharing the FTN ID with the renderer is still a no-go. Some points/concerns at that post are addressed below:

> 1. FTN ID is not always known.
This remark seems only applicable to child frames, where the renderer synchronously gets a routing ID from the IO thread, and the FTN (and RFH) is created later on the UI thread (FTN ID starts counting at 1).

This issue could be resolved by allocating FTN IDs on the IO thread as well, by starting at INT_MAX and counting backwards. This ID is included in the existing message to the UI thread, where the FTN (&RFH) is constructed.

> 2. Security concerns due to the browser-global nature of FTN IDs.
Valid, but any spoofing concerns do not magically disappear by forcing the use of other IDs. Making it easier to validate the FTN ID in the browser would be a stronger solution
(e.g. a means to verify that the sender can reply with a given FTN ID).
(e.g. checking that the routing id and process id matches of the RFH corresponding to the FTN ID matches the routing ID and process ID that is sent along the message.)

If synchronizing FTN IDs with RenderFrame acceptable now, or do you see a better way to map resource requests to FTN IDs?

(Incidentally, I saw that requests initiated from the PlzNavigate project are already associated with FTN IDs, but no renderer ID. I wonder how that interacts with the webRequest extension APi, which currently uses renderframe IDs. In the context of PlzNavigate, here is another ticket that might benefit from having a FTN ID in the renderer: crbug.com/376003.)

Kind regards,

 Rob

 https://robwu.nl

[1] https://groups.google.com/a/chromium.org/d/msg/site-isolation-dev/zPoc1i7YKdQ/2R6jl03-OkcJ

[Charlie Reis]

[+rdsmith,davidben,carlosk]

On Sun, Aug 2, 2015 at 4:02 PM, Rob Wu <r...@robwu.nl> wrote:

Hi all,

I'm working on updating the frameId (used in the several extension APIs) for OOPIF (crbug.com/432875), which has the following characteristics:

1. The frameId must uniquely identify a frame within a tab, even across cross-process navigation.

2. The ID of the main frame is not important (it is always 0 for backwards-compatibility).

3. The frameId of the current frame and the parent frame must be known (webRequest / webNavigation API).

Currently frameId is based on the render frame ID, but it will be changed to the FTN ID. The FTN ID is currently not public yet, but to facilitate this change, the site isolation team has changed the type from int64 to int32 and expressed approval towards exposing the ID via RFH (crbug.com/432875#c7). I'll therefore add two methods (to retrieve a FTN ID from a RFH and vice versa).

Requests in the ResourceDispatcherHost can be identified by process ID + routing ID, but these cannot be mapped to a RFH because the RFH::FromID method is only useful on the IO thread. Consequently, this pair of IDs cannot be mapped to a FTN ID either.

rdsmith@ is updating the ResourceDispatcherHost IDs in https://crbug.com/482049, and there's some discussion in the "Site Isolation needs help with the network stack" thread.  In conjunction with PlzNavigate, I think there might some need for FTN IDs there already.

To solve this issue, I'd like to synchronize the FTN IDs with the RenderFrame in the renderer. This was brought up before at Sync frame tree node ids? [1], and was received with some (mild) opposition. With the publication of the FTN ID, I was wondering whether sharing the FTN ID with the renderer is still a no-go.

I'd still really like to avoid giving FTN IDs to the renderer process.

 

Some points/concerns at that post are addressed below:

> 1. FTN ID is not always known.
This remark seems only applicable to child frames, where the renderer synchronously gets a routing ID from the IO thread, and the FTN (and RFH) is created later on the UI thread (FTN ID starts counting at 1).

This issue could be resolved by allocating FTN IDs on the IO thread as well, by starting at INT_MAX and counting backwards. This ID is included in the existing message to the UI thread, where the FTN (&RFH) is constructed.

> 2. Security concerns due to the browser-global nature of FTN IDs.
Valid, but any spoofing concerns do not magically disappear by forcing the use of other IDs. Making it easier to validate the FTN ID in the browser would be a stronger solution
(e.g. a means to verify that the sender can reply with a given FTN ID).
(e.g. checking that the routing id and process id matches of the RFH corresponding to the FTN ID matches the routing ID and process ID that is sent along the message.)

Validating them would be required if we added it, but this creates opportunities to forget to validate.  Chrome team has already pushed back strongly against telling the renderer process its child process ID for this reason-- it's too easy for new code to accept an IPC and assume that the reported process ID is correct, when the browser process should know it already.  FTN ID falls into the same category.

Plus, the validation would likely be difficult on the IO thread, which is the reason we'd be sending it through the renderer.  I don't think we should do this.

 

If synchronizing FTN IDs with RenderFrame acceptable now, or do you see a better way to map resource requests to FTN IDs?

(Incidentally, I saw that requests initiated from the PlzNavigate project are already associated with FTN IDs, but no renderer ID. I wonder how that interacts with the webRequest extension APi, which currently uses renderframe IDs. In the context of PlzNavigate, here is another ticket that might benefit from having a FTN ID in the renderer: crbug.com/376003.)

Yes, this is the same work I was referring to above.  I think it's better to revise the network stack to be aware of FTN IDs than to pass them through the renderer process.

Randy and David, can you comment on your plans here, and whether there's a way forward?

Thanks!

Charlie

[Randy Smith]

On Mon, Aug 3, 2015 at 12:40 PM, Charlie Reis <cr...@chromium.org> wrote:
> [+rdsmith,davidben,carlosk]
>
> On Sun, Aug 2, 2015 at 4:02 PM, Rob Wu <r...@robwu.nl> wrote:
>>
>> Hi all,
>>
>> I'm working on updating the frameId (used in the several extension APIs)
>> for OOPIF (crbug.com/432875), which has the following characteristics:
>> 1. The frameId must uniquely identify a frame within a tab, even across
>> cross-process navigation.
>> 2. The ID of the main frame is not important (it is always 0 for
>> backwards-compatibility).

Meaning that the frameId will not uniquely identify a frame within a
tab if it refers to a main frame--main frames will need the additional
context of the tab?

>> 3. The frameId of the current frame and the parent frame must be known
>> (webRequest / webNavigation API).
>>
>> Currently frameId is based on the render frame ID, but it will be changed
>> to the FTN ID. The FTN ID is currently not public yet, but to facilitate
>> this change, the site isolation team has changed the type from int64 to
>> int32 and expressed approval towards exposing the ID via RFH
>> (crbug.com/432875#c7). I'll therefore add two methods (to retrieve a FTN ID
>> from a RFH and vice versa).
>>
>> Requests in the ResourceDispatcherHost can be identified by process ID +
>> routing ID, but these cannot be mapped to a RFH because the RFH::FromID
>> method is only useful on the IO thread.

Sorry, I'm missing something. RDH runs on the IO thread; why does the
restriction of RFH::FromID to the IO thread inhibit the mapping of
process ID + routing ID -> RFH? Did you mean UI thread?

>> Consequently, this pair of IDs
>> cannot be mapped to a FTN ID either.
>
>
> rdsmith@ is updating the ResourceDispatcherHost IDs in
> https://crbug.com/482049, and there's some discussion in the "Site Isolation
> needs help with the network stack" thread. In conjunction with PlzNavigate,
> I think there might some need for FTN IDs there already.

My work so far has been just trying to switch over to using render
frame ids from using render view ids. I have some awareness of FTN
IDs, but not a good sense of how it's used; mostly when I've
encountered FTN IDs I've left them alone and just switched the RVIDs
to RFIDs.

I'm hoping David can do the heavy lifting on this request, as I don't
feel I have the context to comment intelligently. If you want an
unintelligent comment, I think we need some way to map from process id
+ render frame id -> FTN ID on the IO thread in the
ResourceDispatcherHost. And yes, there's a concern about duplication
of information between an IO thread and UI thread DS in doing that but
if we're moving over to FTN ID as the one tree id (???), maybe we
could deprecate lookup of the various data structures by process
id/render frame id?

-- Randy

[David Benjamin]

I'll try to send out a document at some point this week (I keep promising one and getting pulled away for something else), but I would like to decouple the ResourceLoader from the RDH and somewhat decouple the RDH from WebContents and replace all these RVH/RFH IDs stapled to requests. This is largely to resolve the PlzNavigate RFH ID issues around network auth dialogs and all our other action-at-a-distance id-carrying IO -> UI net stack thread hops, but probably is relevant here.

Specifically, my plan was that, when RFHs are created, we register a client with the RDH which maintains a map of (child_id, frame_id) -> some kind of delegate interface. And then, when the RDH is destroyed, we unregister the client and cancel all requests under that client. (Requests from unregistered clients get rejected.) This would be used for routing network auth dialogs back up through the WebContentsDelegate, which PlzNavigate would also funnel into.

There would be some timing subtleties to work out due to what's effectively synchronizing some state between UI and IO threads. But I think this is unavoidable. Both extensions and WebView *already* maintain these kinds of maps today. //content is in a much better position to get the ordering right once rather than let embedders do it ad-hoc. (Granted, Service Worker makes both of these features inherently problematic anyway.)

In other words, this means that the CreateNewRenderFrame() API that the renderer conceptually implements (across processes, IPCs, and friends) gains a "ResourceLoaderInterface" parameter that this particular RenderFrame is expected to use for resource requests. That particular implementation would have the owning WebContents and FrameTreeNode bound to it, so that callbacks have the right context.

Then we can unexpose child ID and RFH ID from ResourceRequestInfo altogether, since those do weird things with downloads and PlzNavigate. Plus RFH IDs aren't very meaningful from a web platform perspective. They aren't stable across the entire frame due to cross-process navigations, but they don't correspond to a single document due to in-process ones either.

I actually wasn't intending to expose FTN ID, but it sounds like extensions need it, so we can set it up as part of this RDH client context. FTNs are meaningful to the platform, so that seems reasonable.

David

 

[Rob Wu]

Randy's reply was missing from this thread, I've added some of his replies back.

2015-08-03 19:19 GMT+02:00 David Benjamin <davi...@chromium.org>:

On Mon, Aug 3, 2015 at 12:40 PM Charlie Reis <cr...@chromium.org> wrote:

[+rdsmith,davidben,carlosk]

On Sun, Aug 2, 2015 at 4:02 PM, Rob Wu <r...@robwu.nl> wrote:

Hi all,

I'm working on updating the frameId (used in the several extension APIs) for OOPIF (crbug.com/432875), which has the following characteristics:

1. The frameId must uniquely identify a frame within a tab, even across cross-process navigation.

2. The ID of the main frame is not important (it is always 0 for backwards-compatibility).

> Meaning that the frameId will not uniquely identify a frame within a

> tab if it refers to a main frame--main frames will need the additional
> context of the tab?

Yes. All extension APIs that use frames also require a tab ID to be specified.

3. The frameId of the current frame and the parent frame must be known (webRequest / webNavigation API).

Currently frameId is based on the render frame ID, but it will be changed to the FTN ID. The FTN ID is currently not public yet, but to facilitate this change, the site isolation team has changed the type from int64 to int32 and expressed approval towards exposing the ID via RFH (crbug.com/432875#c7). I'll therefore add two methods (to retrieve a FTN ID from a RFH and vice versa).

Requests in the ResourceDispatcherHost can be identified by process ID + routing ID, but these cannot be mapped to a RFH because the RFH::FromID method is only useful on the IO thread.

> Sorry, I'm missing something.  RDH runs on the IO thread; why does the
> restriction of RFH::FromID to the IO thread inhibit the mapping of
> process ID + routing ID -> RFH?  Did you mean UI thread?

Eh, yes. I meant to say that RFH::FromID can only be used on the UI thread, while RDH runs on the IO thread.

 

Consequently, this pair of IDs cannot be mapped to a FTN ID either.

rdsmith@ is updating the ResourceDispatcherHost IDs in https://crbug.com/482049, and there's some discussion in the "Site Isolation needs help with the network stack" thread.  In conjunction with PlzNavigate, I think there might some need for FTN IDs there already.

To solve this issue, I'd like to synchronize the FTN IDs with the RenderFrame in the renderer. This was brought up before at Sync frame tree node ids? [1], and was received with some (mild) opposition. With the publication of the FTN ID, I was wondering whether sharing the FTN ID with the renderer is still a no-go.

I'd still really like to avoid giving FTN IDs to the renderer process.

 

Some points/concerns at that post are addressed below:

> 1. FTN ID is not always known.
This remark seems only applicable to child frames, where the renderer synchronously gets a routing ID from the IO thread, and the FTN (and RFH) is created later on the UI thread (FTN ID starts counting at 1).

This issue could be resolved by allocating FTN IDs on the IO thread as well, by starting at INT_MAX and counting backwards. This ID is included in the existing message to the UI thread, where the FTN (&RFH) is constructed.

> 2. Security concerns due to the browser-global nature of FTN IDs.
Valid, but any spoofing concerns do not magically disappear by forcing the use of other IDs. Making it easier to validate the FTN ID in the browser would be a stronger solution
(e.g. a means to verify that the sender can reply with a given FTN ID).
(e.g. checking that the routing id and process id matches of the RFH corresponding to the FTN ID matches the routing ID and process ID that is sent along the message.)

Validating them would be required if we added it, but this creates opportunities to forget to validate.  Chrome team has already pushed back strongly against telling the renderer process its child process ID for this reason-- it's too easy for new code to accept an IPC and assume that the reported process ID is correct, when the browser process should know it already.  FTN ID falls into the same category.

Plus, the validation would likely be difficult on the IO thread, which is the reason we'd be sending it through the renderer.  I don't think we should do this.

 

If synchronizing FTN IDs with RenderFrame acceptable now, or do you see a better way to map resource requests to FTN IDs?

(Incidentally, I saw that requests initiated from the PlzNavigate project are already associated with FTN IDs, but no renderer ID. I wonder how that interacts with the webRequest extension APi, which currently uses renderframe IDs. In the context of PlzNavigate, here is another ticket that might benefit from having a FTN ID in the renderer: crbug.com/376003.)

Yes, this is the same work I was referring to above.  I think it's better to revise the network stack to be aware of FTN IDs than to pass them through the renderer process.

Randy and David, can you comment on your plans here, and whether there's a way forward?

I'll try to send out a document at some point this week (I keep promising one and getting pulled away for something else), but I would like to decouple the ResourceLoader from the RDH and somewhat decouple the RDH from WebContents and replace all these RVH/RFH IDs stapled to requests. This is largely to resolve the PlzNavigate RFH ID issues around network auth dialogs and all our other action-at-a-distance id-carrying IO -> UI net stack thread hops, but probably is relevant here.

Specifically, my plan was that, when RFHs are created, we register a client with the RDH which maintains a map of (child_id, frame_id) -> some kind of delegate interface. And then, when the RDH is destroyed, we unregister the client and cancel all requests under that client. (Requests from unregistered clients get rejected.) This would be used for routing network auth dialogs back up through the WebContentsDelegate, which PlzNavigate would also funnel into.

The current implementation of PlzNavigate omits the ID for the parent frame. Will your proposed changes offer a way to retrieve the FTN ID for the parent frame?
 

There would be some timing subtleties to work out due to what's effectively synchronizing some state between UI and IO threads. But I think this is unavoidable. Both extensions and WebView *already* maintain these kinds of maps today. //content is in a much better position to get the ordering right once rather than let embedders do it ad-hoc. (Granted, Service Worker makes both of these features inherently problematic anyway.)

In other words, this means that the CreateNewRenderFrame() API that the renderer conceptually implements (across processes, IPCs, and friends) gains a "ResourceLoaderInterface" parameter that this particular RenderFrame is expected to use for resource requests. That particular implementation would have the owning WebContents and FrameTreeNode bound to it, so that callbacks have the right context.

Then we can unexpose child ID and RFH ID from ResourceRequestInfo altogether, since those do weird things with downloads and PlzNavigate. Plus RFH IDs aren't very meaningful from a web platform perspective. They aren't stable across the entire frame due to cross-process navigations, but they don't correspond to a single document due to in-process ones either.

I actually wasn't intending to expose FTN ID, but it sounds like extensions need it, so we can set it up as part of this RDH client context. FTNs are meaningful to the platform, so that seems reasonable.

That's good news. Which issue tracks this specific feature?

[David Benjamin]

That can be stapled to the context too I suppose. *shrug*
 

There would be some timing subtleties to work out due to what's effectively synchronizing some state between UI and IO threads. But I think this is unavoidable. Both extensions and WebView *already* maintain these kinds of maps today. //content is in a much better position to get the ordering right once rather than let embedders do it ad-hoc. (Granted, Service Worker makes both of these features inherently problematic anyway.)

In other words, this means that the CreateNewRenderFrame() API that the renderer conceptually implements (across processes, IPCs, and friends) gains a "ResourceLoaderInterface" parameter that this particular RenderFrame is expected to use for resource requests. That particular implementation would have the owning WebContents and FrameTreeNode bound to it, so that callbacks have the right context.

Then we can unexpose child ID and RFH ID from ResourceRequestInfo altogether, since those do weird things with downloads and PlzNavigate. Plus RFH IDs aren't very meaningful from a web platform perspective. They aren't stable across the entire frame due to cross-process navigations, but they don't correspond to a single document due to in-process ones either.

I actually wasn't intending to expose FTN ID, but it sounds like extensions need it, so we can set it up as part of this RDH client context. FTNs are meaningful to the platform, so that seems reasonable.

That's good news. Which issue tracks this specific feature?

None right now. I'm still in the "write down a concrete plan with bounded scope and make sure people don't think this is a terrible idea" stages. :-) The ResourceDispatcherHost stack is kind of a mess.

David

[Rob Wu]

What is the status of FTN ID exposure in RDH? The extensions team would like to launch site isolation for extensions in Q3 2015, and the absence of FTN ID in the RDH is blocking that.

Of course, I could add a stop-gap solution by introducing a (RPH ID + RFH) -> FTN ID mapping, but if you have somethng close to completion that is usable, the efforts would be a waste.

Kind regards,

 Rob

 https://robwu.nl

[David Benjamin]

I'll try to get around to sending a broader email today. The status is I wrote this document:

https://docs.google.com/document/d/1OpgSP0qtHzweXjTpcHTMg6n2XtB2zSg6YIqbGpmxJNY/edit?usp=sharing

There were concerns about adding IO protrusions of RFH and things because we try to keep IO thread state simple. I'm currently toying with getting the easier parts of this cleanup through (getting network auth dialogs in shape).

For other consumers, the thought is to see how we could involve the UI thread more. I want to do a field trial to see how expensive a UI hop on every network request would be. Failing that, we could try to do it just for things that need it.

But getting the FTN ID onto the IO thread is actually very challenging even with IO protrusions because of how child RFHs get created. I beileve a request may come in before a UI -> IO PostTask even. I think it's possible, but it would be pretty complex. For extensions, I think it may be best to pay the upfront cost of moving chrome.webRequest logic to the UI thread. This is an IPC followed by running JS in a renderer, so I doubt the thread hops matter. I'm not familiar with the extensions code though, so I haven't explored how that would look.

David