FrameTreeNode* frame_tree_node =Philip Pfaffecould this cross the CDP target boundary and reach an OOPIF?
I suppose so. Would you expect that to be problematic? We'd call a tool in the wrong frame, assuming that new frame declares a tool by that name and schema. Same for same-process frames though.
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
WebContentsObserver::Observe(host_ ? WebContents::FromRenderFrameHost(host_)Could we observe/unobserve in enable/disable?
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
std::is_same<T, protocol::WebMCPHandler>,Philip Pfaffeis it listed two times here?
Done
std::is_same<T, protocol::WebMCPHandler>,Philip Pfaffeis it listed two times here?
Done
WebContentsObserver::Observe(host_ ? WebContents::FromRenderFrameHost(host_)Could we observe/unobserve in enable/disable?
Done
WebContentsObserver::Observe(host_ ? WebContents::FromRenderFrameHost(host_)Could we observe/unobserve in enable/disable?
Done
// Invokes a tool in the frame. Returns true if the tool was found andPhilip Pfaffe```suggestion
// Invokes a WebMCP tool in the frame. Returns true if the tool was found and
```
I think the name WebMCP isn't really used in the codebase, I'm calling it Script Tool now.
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
FrameTreeNode* frame_tree_node =Philip Pfaffecould this cross the CDP target boundary and reach an OOPIF?
I suppose so. Would you expect that to be problematic? We'd call a tool in the wrong frame, assuming that new frame declares a tool by that name and schema. Same for same-process frames though.
It would be problematic in extensions for instance where connecting to an OOPIF might not be allowed. Can we ensure that the frame_id belongs to the current CDP target?
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
base::JSONWriter::Write(*input, &input_arguments);Does this work? I think the protocol types are usually (always? compatible with base types but I think the AI suggestion might be good?
AI-suggested:
`input` is of type `std::unique_ptr<protocol::DictionaryValue>` (which is based on `crdtp`), not a `base::Value`. Passing it to `base::JSONWriter::Write` will cause a compilation error.
You should either convert it to a `base::Value` first, or serialize it using `crdtp::json` similar to how it was done in the Blink implementation:
```cpp
std::vector<uint8_t> cbor;
input->AppendSerialized(&cbor);
crdtp::json::ConvertCBORToJSON(
crdtp::span<uint8_t>(cbor.data(), cbor.size()), &input_arguments);
```
FrameTreeNode* frame_tree_node =Philip Pfaffecould this cross the CDP target boundary and reach an OOPIF?
Alex RudenkoI suppose so. Would you expect that to be problematic? We'd call a tool in the wrong frame, assuming that new frame declares a tool by that name and schema. Same for same-process frames though.
It would be problematic in extensions for instance where connecting to an OOPIF might not be allowed. Can we ensure that the frame_id belongs to the current CDP target?
Ack. added a check below, is that the best way to check this?
Does this work? I think the protocol types are usually (always? compatible with base types but I think the AI suggestion might be good?
AI-suggested:
`input` is of type `std::unique_ptr<protocol::DictionaryValue>` (which is based on `crdtp`), not a `base::Value`. Passing it to `base::JSONWriter::Write` will cause a compilation error.
You should either convert it to a `base::Value` first, or serialize it using `crdtp::json` similar to how it was done in the Blink implementation:
```cpp
std::vector<uint8_t> cbor;
input->AppendSerialized(&cbor);
crdtp::json::ConvertCBORToJSON(
crdtp::span<uint8_t>(cbor.data(), cbor.size()), &input_arguments);
```
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
| Code-Review | +1 |
FrameTreeNode* frame_tree_node =Philip Pfaffecould this cross the CDP target boundary and reach an OOPIF?
Alex RudenkoI suppose so. Would you expect that to be problematic? We'd call a tool in the wrong frame, assuming that new frame declares a tool by that name and schema. Same for same-process frames though.
Philip PfaffeIt would be problematic in extensions for instance where connecting to an OOPIF might not be allowed. Can we ensure that the frame_id belongs to the current CDP target?
Ack. added a check below, is that the best way to check this?
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Rakina: PTAL at fake_local_frame
dcheng: PTAL at the local frame mojo changes
Thanks!
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
dom@: Any general thoughts on the approach? Any issues with drilling these extra holes for the inspector?
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
From googleclient/chrome/chromium_gwsq/ipc/config.gwsq:
IPC: antonio...@chromium.org, dch...@chromium.org, d...@chromium.org
📎 It looks like you’re making a possibly security-sensitive change! 📎 IPC security review isn’t a rubberstamp, so your friendly security reviewer will need a fair amount of context to review your CL effectively. Please review your CL description and code comments to make sure they provide context for someone unfamiliar with your project/area. Pay special attention to where data comes from and which processes it flows between (and their privilege levels). Feel free to point your security reviewer at design docs, bugs, or other links if you can’t reasonably make a self-contained CL description. (Also see https://cbea.ms/git-commit/).
IPC reviewer(s): antonio...@chromium.org, dch...@chromium.org, d...@chromium.org
Reviewer source(s):
antonio...@chromium.org is from context(googleclient/chrome/chromium_gwsq/ipc/config.gwsq)
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
This seems very similar to https://chromium-review.googlesource.com/c/chromium/src/+/7748304 in some ways. What's the difference and why do we need both?
base::flat_set<base::UnguessableToken> initiated_invocations_;Nit: unless you need ordering, please use absl::flat_hash_set.
if (auto* window = DomWindow()) {This null check shouldn't be needed? At the very least, I hope we're not handling Mojo calls for detached frames.
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
This seems very similar to https://chromium-review.googlesource.com/c/chromium/src/+/7748304 in some ways. What's the difference and why do we need both?
The only real difference is where the tool calls are coming from. In that CL they originate from actor in //chrome, this here is in //content. Other than that they do pretty much the same thing. This change is required because we can't have the cdp handler in //chrome or it wouldn't work in headless.
base::flat_set<base::UnguessableToken> initiated_invocations_;Nit: unless you need ordering, please use absl::flat_hash_set.
Done
This null check shouldn't be needed? At the very least, I hope we're not handling Mojo calls for detached frames.
Done
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
InvokeScriptToolForInspector(mojo_base.mojom.UnguessableToken invocation_id,
string tool_name,
string input_arguments) => (bool success);is this formatted? I would have expected all parameters to have the same indentation level.
mojo_base.mojom.UnguessableToken invocation_id) => (string? result);are you using this result anywhere? The only invocation I see is discarding the result I believe. Also, if you are using it, do you need it to be optional? I think you are always returning a string (maybe empty).
InvokeScriptToolForInspector(mojo_base.mojom.UnguessableToken invocation_id,
string tool_name,
string input_arguments) => (bool success);is this formatted? I would have expected all parameters to have the same indentation level.
Yes. I don't think presumbmit would pass this otherwise, would it?
mojo_base.mojom.UnguessableToken invocation_id) => (string? result);are you using this result anywhere? The only invocation I see is discarding the result I believe. Also, if you are using it, do you need it to be optional? I think you are always returning a string (maybe empty).
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
InvokeScriptToolForInspector(mojo_base.mojom.UnguessableToken invocation_id,
string tool_name,
string input_arguments) => (bool success);Philip Pfaffeis this formatted? I would have expected all parameters to have the same indentation level.
Yes. I don't think presumbmit would pass this otherwise, would it?
Oh I suppose it wasn't. Formatted like you suggested. Formatting everything produces a huge diff.
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
// document, and requests the result of that invocation.this is now outdated
// Notifies the frame that a tool invocation resulted in a navigation to this
// document, and requests the result of that invocation.Now that I think about of this, this sounds kind of weird. What is the document supposed to do in response of this "notification"? This can will reach the renderer some time in the future, and the document might already have been renderer and interacted with...?
I think I am missing the point of this call, but it might just be that I don't have enough context here.
GetCrossDocumentScriptToolResultForInspector(the name should also be updated, something like "Notify..."?
// Notifies the frame that a tool invocation resulted in a navigation to this
// document, and requests the result of that invocation.Now that I think about of this, this sounds kind of weird. What is the document supposed to do in response of this "notification"? This can will reach the renderer some time in the future, and the document might already have been renderer and interacted with...?
I think I am missing the point of this call, but it might just be that I don't have enough context here.
At this point, the renderer doesn't really do anything with it except trigger the call to the devtools probe, at least that's the intended purpose. In practice it'll call the regular tool result retrieval on modelcontext the same way it would for a call from actor via //chrome's render frame: https://source.chromium.org/chromium/chromium/src/+/main:chrome/common/chrome_render_frame.mojom;drc=6250dbaff56fb8aad8ede90a7ec8e9a1a3c4dd05;l=152
void LocalFrameMojoHandler::InvokeScriptToolForInspector(This seems like a wrapper around ModelContext::ExecuteTool, which already implements a mojo interface. Do we need this wrapper and the additional mojo messages on LocalFrame? Could we somehow reuse the existing ModelContext interface and call it directly?
// document, and requests the result of that invocation.Philip Pfaffethis is now outdated
Done
the name should also be updated, something like "Notify..."?
Done
void LocalFrameMojoHandler::InvokeScriptToolForInspector(This seems like a wrapper around ModelContext::ExecuteTool, which already implements a mojo interface. Do we need this wrapper and the additional mojo messages on LocalFrame? Could we somehow reuse the existing ModelContext interface and call it directly?
The problem with that interface is that it only gets connected once the page accesses navigator.modelContext. We need to call the new IPC on navigation, at which point the interface is not yet connected.
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
| Code-Review | +1 |
// Notifies the frame that a tool invocation resulted in a navigation to this
// document, and requests the result of that invocation.Philip PfaffeNow that I think about of this, this sounds kind of weird. What is the document supposed to do in response of this "notification"? This can will reach the renderer some time in the future, and the document might already have been renderer and interacted with...?
I think I am missing the point of this call, but it might just be that I don't have enough context here.
At this point, the renderer doesn't really do anything with it except trigger the call to the devtools probe, at least that's the intended purpose. In practice it'll call the regular tool result retrieval on modelcontext the same way it would for a call from actor via //chrome's render frame: https://source.chromium.org/chromium/chromium/src/+/main:chrome/common/chrome_render_frame.mojom;drc=6250dbaff56fb8aad8ede90a7ec8e9a1a3c4dd05;l=152
Thanks. I think elaborating a bit in the comment could be helpful. I still don't have enough context to understand exactly why this is needed, but I think this is fine.
void LocalFrameMojoHandler::InvokeScriptToolForInspector(Philip PfaffeThis seems like a wrapper around ModelContext::ExecuteTool, which already implements a mojo interface. Do we need this wrapper and the additional mojo messages on LocalFrame? Could we somehow reuse the existing ModelContext interface and call it directly?
The problem with that interface is that it only gets connected once the page accesses navigator.modelContext. We need to call the new IPC on navigation, at which point the interface is not yet connected.
Ok. I don't have the full picture here, so I'll defer to you. I think it would be beneficial from a code health pov to understand whether it's possible to deduplicate these mojo calls, but I don't think this is blocking, so LGTM.
| Code-Review | +1 |
Philip PfaffeThis seems very similar to https://chromium-review.googlesource.com/c/chromium/src/+/7748304 in some ways. What's the difference and why do we need both?
The only real difference is where the tool calls are coming from. In that CL they originate from actor in //chrome, this here is in //content. Other than that they do pretty much the same thing. This change is required because we can't have the cdp handler in //chrome or it wouldn't work in headless.
Friendly ping, does @pfaffe's answer resolve the comment? This CL would fix the declarative tools not working for automation client so it would be great if we could land it. PTAL.
| Code-Review | +1 |
| Code-Review | +1 |
Philip PfaffeThis seems very similar to https://chromium-review.googlesource.com/c/chromium/src/+/7748304 in some ways. What's the difference and why do we need both?
Alex RudenkoThe only real difference is where the tool calls are coming from. In that CL they originate from actor in //chrome, this here is in //content. Other than that they do pretty much the same thing. This change is required because we can't have the cdp handler in //chrome or it wouldn't work in headless.
Friendly ping, does @pfaffe's answer resolve the comment? This CL would fix the declarative tools not working for automation client so it would be great if we could land it. PTAL.
The updated CL seems to have much less overlap so it's better from that perspective.
But it still feels a bit weird and like we're possibly missing some pieces somewhere. Having `NotifyInspectorOfCrossDocumentScriptToolResult` still means we IPC twice to the renderer to notify of a cross-document tool result... but that still seems a bit redundant. It seems like maybe //content should expose the APIs that //chrome might need, or the IPC returning the `result` itself ought to be in //content, with some way for //content to pass it back to //chrome if that makes sense... it doesn't really make sense for both layers to do the work.
LGTM but I think maybe there's a bit more followup and investigation involved here. The interfaces, as written, don't quite make sense to me still.
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Philip PfaffeThis seems very similar to https://chromium-review.googlesource.com/c/chromium/src/+/7748304 in some ways. What's the difference and why do we need both?
Alex RudenkoThe only real difference is where the tool calls are coming from. In that CL they originate from actor in //chrome, this here is in //content. Other than that they do pretty much the same thing. This change is required because we can't have the cdp handler in //chrome or it wouldn't work in headless.
Daniel ChengFriendly ping, does @pfaffe's answer resolve the comment? This CL would fix the declarative tools not working for automation client so it would be great if we could land it. PTAL.
The updated CL seems to have much less overlap so it's better from that perspective.
But it still feels a bit weird and like we're possibly missing some pieces somewhere. Having `NotifyInspectorOfCrossDocumentScriptToolResult` still means we IPC twice to the renderer to notify of a cross-document tool result... but that still seems a bit redundant. It seems like maybe //content should expose the APIs that //chrome might need, or the IPC returning the `result` itself ought to be in //content, with some way for //content to pass it back to //chrome if that makes sense... it doesn't really make sense for both layers to do the work.
Thanks I filed http://crbug.com/515541906 In general, we do not want to have the CDP handle to invoke WebMCP in //chrome because it does not work in headless and that was agreed to be in scope for WebMCP as a platform feature. I believe there was interest in moving the feature logic that is currently in //chrome to //content but it requires additional work on the WebMCP end.
LGTM but I think maybe there's a bit more followup and investigation involved here. The interfaces, as written, don't quite make sense to me still.
I filed http://crbug.com/515541906
| Code-Review | +1 |
// Notifies the frame that a tool invocation resulted in a navigation to this
// document, and requests the result of that invocation.Philip PfaffeNow that I think about of this, this sounds kind of weird. What is the document supposed to do in response of this "notification"? This can will reach the renderer some time in the future, and the document might already have been renderer and interacted with...?
I think I am missing the point of this call, but it might just be that I don't have enough context here.
Antonio SartoriAt this point, the renderer doesn't really do anything with it except trigger the call to the devtools probe, at least that's the intended purpose. In practice it'll call the regular tool result retrieval on modelcontext the same way it would for a call from actor via //chrome's render frame: https://source.chromium.org/chromium/chromium/src/+/main:chrome/common/chrome_render_frame.mojom;drc=6250dbaff56fb8aad8ede90a7ec8e9a1a3c4dd05;l=152
Thanks. I think elaborating a bit in the comment could be helpful. I still don't have enough context to understand exactly why this is needed, but I think this is fine.
I agree with the need for more documentation here, but since pfaffe@ is out, for us to add it in this CL we'd end up changing the owner and probably alexrudenko@'s LGTM would be wiped, so I vote we add this in a follow-up CL that he can upload and own directly. Otherwise, LGTM.
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
// Notifies the frame that a tool invocation resulted in a navigation to this
// document, and requests the result of that invocation.Philip PfaffeNow that I think about of this, this sounds kind of weird. What is the document supposed to do in response of this "notification"? This can will reach the renderer some time in the future, and the document might already have been renderer and interacted with...?
I think I am missing the point of this call, but it might just be that I don't have enough context here.
Antonio SartoriAt this point, the renderer doesn't really do anything with it except trigger the call to the devtools probe, at least that's the intended purpose. In practice it'll call the regular tool result retrieval on modelcontext the same way it would for a call from actor via //chrome's render frame: https://source.chromium.org/chromium/chromium/src/+/main:chrome/common/chrome_render_frame.mojom;drc=6250dbaff56fb8aad8ede90a7ec8e9a1a3c4dd05;l=152
Dominic FarolinoThanks. I think elaborating a bit in the comment could be helpful. I still don't have enough context to understand exactly why this is needed, but I think this is fine.
I agree with the need for more documentation here, but since pfaffe@ is out, for us to add it in this CL we'd end up changing the owner and probably alexrudenko@'s LGTM would be wiped, so I vote we add this in a follow-up CL that he can upload and own directly. Otherwise, LGTM.
Thanks for review. I will send a CL with the extended comment next week!
9 is the latest approved patch-set.
No files were changed between the latest approved patch-set and the submitted one.
Add a browser-side WebMCP.invokeTool handler in //content
This allows us to detect cross-document results and trigger the event
for them.
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |