Tool-Calling / Multimodality
18 views
Skip to first unread message
Nico Martin
Jun 29, 2026, 5:41:05 AM (7 days ago) Jun 29
to Chrome Built-in AI Early Preview Program Discussions
Hi all,
I'm Nico, ML Engineer at Hugging Face where I work on Transformers.js. Lately I've been building an AgentSDK on top of it to enable on-device agents with tool calling and multimodality support. I've been trying to align it as closely as possible with the Prompt API spec, which is how I ended up here.
Honestly, I think the recent public backlash has taken some of the urgency out of things, both at Google and Microsoft. The GitHub repo has gotten pretty quiet and some of the bigger open questions around tool calling, message structure and extended thinking haven't really moved.
But I still think the API has a real future. On-device models are getting better fast and having a shared browser-native interface for them is genuinely exciting. It would be a shame to lose the momentum now.
I just opened a GitHub issue proposing a restructured Message type to cover tool calling, extended thinking and multimodality:
https://github.com/webmachinelearning/prompt-api/issues/209
I'm Nico, ML Engineer at Hugging Face where I work on Transformers.js. Lately I've been building an AgentSDK on top of it to enable on-device agents with tool calling and multimodality support. I've been trying to align it as closely as possible with the Prompt API spec, which is how I ended up here.
Honestly, I think the recent public backlash has taken some of the urgency out of things, both at Google and Microsoft. The GitHub repo has gotten pretty quiet and some of the bigger open questions around tool calling, message structure and extended thinking haven't really moved.
But I still think the API has a real future. On-device models are getting better fast and having a shared browser-native interface for them is genuinely exciting. It would be a shame to lose the momentum now.
I just opened a GitHub issue proposing a restructured Message type to cover tool calling, extended thinking and multimodality:
https://github.com/webmachinelearning/prompt-api/issues/209
It would mean a lot to get some community input on it before I move forward with the implementation. Even if the spec isn't final yet, I want to make sure I'm heading in the right direction. So if you have a few minutes, please drop a comment!
Best, Nico
Reply all
Reply to author
Forward
0 new messages