First testing of WebMCP - Part 2!

[Raymond Camden]

Ok, I kinda brought this up in my first post but... I'm really confused about how we test this. From what I know, Chrome's built-in Gemini support can't use the tools.

And if I try to use it via the Gemini API and url tools, it fails as well. I tried testing it in AI Studio as well, and got:

"I attempted to look at the URL you provided, but my browsing tool is only able to see the static HTML of the page before any JavaScript loads the main content[1]."

I know about the excellent Chrome extension, but *outside* of that, where should it this work?

[François Beaufort]

On Thu, May 7, 2026 at 12:02 AM Raymond Camden <raymon...@gmail.com> wrote:

Ok, I kinda brought this up in my first post but... I'm really confused about how we test this. From what I know, Chrome's built-in Gemini support can't use the tools.

That's right.

 

And if I try to use it via the Gemini API and url tools, it fails as well. I tried testing it in AI Studio as well, and got:

"I attempted to look at the URL you provided, but my browsing tool is only able to see the static HTML of the page before any JavaScript loads the main content[1]."

I know about the excellent Chrome extension, but *outside* of that, where should it this work?

It currently requires extensions with experimental WebMCP API support such as WebMCP - Model Context Tool Inspector or AgentBoard for instance.

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/f62e4d0f-83c7-40a0-a2fb-647dba7ac777n%40chromium.org.

[Raymond Camden]

Gotcha - what is the plan for the future? I assume eventually we can test this in other ways? I don't really work with extensions myself. 

--

===========================================================================
Raymond Camden

Email : raymon...@gmail.com
Blog : www.raymondcamden.com

[Santiago Omar Javier Aguirre]

July 21, Gemini 3.6 Release and can't able to use WEBMCP tools, for what we are working? this is the future of webmcp standards?

We need answers, i don't want to use other agents like Claude now, im just asking for GEMINI support.

I hope this will work with gemini extension and his agents to really start with this!

Thanks!

[joseph brewerton]

Hi Santiago,

While I'm not running into the exact same blockers, I’ve been exploring alternative ways to solve the tool-calling/execution issue directly in the browser.

I’ve actually shared some of these workarounds back in this forum under my name, but like you, I haven't received any feedback or traction yet. It definitely feels a bit quiet support-wise while the early preview evolves.

Wishing you all the best with your project—hope the team provides some official clarity on the roadmap soon!

Regards

Joe

[Santiago Omar Javier Aguirre]

Thanks Joseph! I will review some of your post! Regards!

[Ares Fixer]

SCRIPTS.zip
https://drive.google.com/file/d/18JDZcl8RPfaYyjnkGh6KzQ-lKque48X-/view?usp=sharing

README.md
https://drive.google.com/file/d/1oXZ2iECQ-fpAhFAPqjy130VoynQJdBdx/view?usp=sharing

Hello everyone,

After following this discussion, I believe there may be an intermediate approach worth exploring while native agent support for WebMCP continues to evolve.

The idea is not to replace WebMCP or create another competing standard.

Instead, we can create a small local development bridge that connects an MCP-capable AI agent to the real browser tab where the web application is running.

The architecture is intentionally simple:

AI or MCP Host
      |
      | MCP over stdio
      v
Local Node.js MCP Server
      |
      | Authenticated WebSocket on 127.0.0.1
      v
Chrome Extension
      |
      | Extension messaging and main-world injection
      v
Web Application / DOM / document.modelContext

This would allow an AI agent to communicate with a visible web application without launching Playwright, Puppeteer, or Selenium for every operation.

Those tools are valuable, but they can be unnecessarily heavy when the browser is already open, authenticated, positioned on the correct page, and running the application state we want to inspect.

Euler and I have prepared a primary proof of concept following this model.

The local Node.js process acts both as an MCP server for the AI and as a small WebSocket server for the browser extension. The WebSocket is bound only to 127.0.0.1, requires a shared authentication token, and accepts communication only from the local machine.

The Chrome extension then acts as the controlled gateway to the active tab.

Through MCP tools, the AI can currently request operations such as:

• Check whether the browser bridge is connected.

• Read the active page title, URL, viewport and visible text.

• Locate inputs, textareas, dropdowns, buttons, links and contenteditable controls.

• Return labels, current values, available options, visibility, coordinates and stable CSS selectors.

• Fill inputs and dropdowns while dispatching the native input and change events expected by React, Angular and Vue applications.

• Click visible controls.

• Wait for a selector or asynchronous interface state.

• Capture the visible browser tab and return the screenshot as a PNG image directly through MCP.

• Inject a lightweight inspector inside the application.

• Discover WebMCP tools registered by the page.

• Execute a WebMCP tool and return its structured result.

• Run diagnostic JavaScript and collect structured console output when explicitly enabled.

When the application exposes proper WebMCP tools, the bridge prefers the structured path:

const tools = await document.modelContext.getTools();

const tool = tools.find(
  (candidate) => candidate.name === requestedToolName
);

if (!tool) {
  throw new Error(`WebMCP tool not found: ${requestedToolName}`);
}

const result = await document.modelContext.executeTool(
  tool,
  JSON.stringify(arguments)
);

console.log("WebMCP result", {
  tool: requestedToolName,
  arguments,
  result
});

return result;

DOM interaction is therefore not intended to compete with WebMCP. It is a fallback and diagnostic mechanism.

The preferred sequence would be:

1. Inspect the active page.
2. Discover its WebMCP tools.
3. Execute a structured WebMCP tool when available.
4. Use direct DOM interaction only when no structured tool exists.
5. Capture a screenshot to validate the visible result.

This also gives product developers a practical way to compare both approaches.

They can observe whether a task succeeds through a structured WebMCP tool, compare it with direct DOM interaction, inspect the returned data, identify missing schemas and understand precisely where the agent loses context.

We also added a lightweight inspector that can be injected directly into the product.

When enabled, it outlines the element currently under the pointer and displays information such as:

{
  tag: "select",
  type: "select",
  selector: "select[name=\"country\"]",
  label: "Country",
  value: "BR",
  options: [
    { value: "BR", text: "Brazil", selected: true },
    { value: "AR", text: "Argentina", selected: false }
  ],
  visible: true,
  disabled: false,
  bounds: {
    x: 412,
    y: 286,
    width: 320,
    height: 42
  }
}

This can make debugging considerably faster because the developer, the browser and the AI can refer to the same visible element and the same structured description.

The implementation uses defensive try/catch handling at every communication boundary:

MCP request
WebSocket transport
Extension service worker
Content-script messaging
Main-world page execution
WebMCP execution
DOM operation
Result serialization

Errors are returned with the action, selector, timeout and stack information whenever available.

Objects written to the console are preserved as structured values rather than converted prematurely into incomplete strings. Circular references, errors, DOM elements, functions, BigInt values and nested objects are serialized with controlled depth so the AI can inspect useful information without breaking the transport.

Arbitrary JavaScript execution deserves special attention.

The prototype supports it because it is extremely valuable during local diagnosis, but it is disabled by default at two independent levels:

1. The MCP server environment.

2. The Chrome extension configuration.

It also requires the active website to match an explicit origin allowlist.

I would not recommend enabling arbitrary evaluation for general browsing or untrusted pages. In a more mature implementation, state-changing operations should also require user confirmation, tool-specific authorization and an audit log.

For Windows developers, the proof of concept includes:

• A PowerShell installation script.

• A Node.js dependency and syntax validator.

• A Chrome Manifest V3 extension.

• An example MCP host configuration using an absolute Windows path.

• A command file for local diagnostics.

• A complete prompt that another coding agent can use to recreate or extend every required file.

• Security and troubleshooting guidance.

• A small MIT-licensed project structure that can be reviewed before use.

I believe this bridge could help us answer several practical questions immediately:

• Can an external MCP agent discover and execute the WebMCP tools of the active page?

• Can developers test their WebMCP implementation before full native Gemini integration is available?

• Can screenshots, console objects and structured control information improve debugging?

• Can we avoid the startup and execution overhead of a complete browser automation framework for simple interactive tests?

• Can the same test run compare structured WebMCP execution with a DOM fallback?

• Which parts should ultimately belong to Chrome, the extension, the website or the AI host?

This is still a development prototype, not a production remote-control solution.

However, it creates a working communication path that can be inspected, criticized, measured and improved today.

If this direction is useful to the group, Euler and I would be glad to share the initial code, document the protocol and collaborate on tests against real WebMCP demonstrations.

Standards become stronger when developers are not limited to discussing what should work, but are also able to build small bridges that reveal what already works, what fails and exactly where the distance remains.

Best regards,

Ares
Developed with Euler

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/c7e619ab-faae-47ed-af87-8733c3e59befn%40chromium.org.

[joseph brewerton]

  Thanks for sharing this! It’s a really interesting starting point for solving the localhost bridge issue.

When I was building my project, running a local model against a hosted https:// site made CORS the main hurdle, so I ended up taking a fairly similar loopback methodology—though routing the stream straight to an 88KB GPU canvas rather than into the DOM.

Really cool to see how you and Euler tackled the extension and WebMCP side of things!  It would be great if this Group was more responsive, really glad to see involvment.

Regards

Joe