Error enabling Audio in Chrome Prompt API (Multimodal) [Uncaught NotAllowedError: Model capability is not available]

[Alan Corn]

I am currently testing the multimodal capabilities of the Prompt API on Chrome [Version 143.0.7499.193 (Official Build) (64-bit)]. When I attempt to use audio-related features, I encounter the following error:

Uncaught NotAllowedError: Model capability is not available.

I have researched this extensively and I am confident that I have enabled the necessary experimental flags:

#optimization-guide-on-device-model: Enable Bypass...
#prompt-api-for-gemini-nano: Enabled
#prompt-api-for-gemini-nano-multimodal-input: Enabled
When I try to add audio support by including type: "audio" in expectedInput during model initialization, I receive the error Uncaught NotAllowedError: Model capability is not available, which prevents me from using the audio functionality.


System Environment:
- OS:  Windows 11 64-bit
- Chrome Version: 143.0.7499.193 (Official Build) (64-bit)
- CPU: 13th Gen Intel(R) Core(TM) i7-13700 (2.10 GHz)
- RAM: 32GB
- GPU (if applicable): none

[Felipe Marcos de abreu Aquino]

After deep investigation and multiple controlled tests, I can confirm that this is not a configuration or permission issue.

At the moment (Chrome 143), the Prompt API with on-device Gemini Nano does not expose audio capabilities (audio input or output) to JavaScript, even when all experimental flags are enabled:

#prompt-api-for-gemini-nano

#prompt-api-for-gemini-nano-multimodal-input

#optimization-guide-on-device-model

These flags only allow usage if the capability is already available, but they do not unlock unsupported modalities. When expectedInput: [{ type: "audio" }] is provided, the runtime correctly rejects the request with:

NotAllowedError: Model capability is not available

This error is thrown by the model capability validator, not by browser permissions, HTTPS policy, or user media access.

Currently, Gemini Nano on-device supports text-only inference via the Prompt API. Audio pipelines exist internally in Chrome (e.g., Live Caption, Recorder), but they are not yet exposed to developers through the Prompt API.

The correct and supported approach today is a hybrid architecture:

Browser-native Speech Recognition (Web Speech API or equivalent) for STT

Prompt API (Gemini Nano) for on-device text inference

Browser-native SpeechSynthesis for TTS

This is a known platform limitation rather than a bug, and there is no workaround on the local Prompt API side until audio capabilities are officially exposed in a future Chrome release.

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/4c8237b9-322e-4194-a9fd-fa2f11c6d6c6n%40chromium.org.

[Alan Corn]

Thank you for your reply. However, I still have a few points of confusion.

When using chrome://on-device-internals/ with Chrome 143 [Version 143.0.7499.193 (Official Build) (64-bit)], I am able to invoke the audio capabilities normally. However, when I navigate to https://web-ai.studio/, I receive the following error: NotAllowedError: Model capability is not available.

Conversely, when using Chrome Canary [Version 145.0.7620.0 (Official Build) canary (64-bit)], everything works fine upon entering https://web-ai.studio/, and I can use the audio-related capabilities normally. This phenomenon confuses me. I asked a few colleagues to test this on their computers using the same latest version of Chrome Canary, but they encountered the NotAllowedError: Model capability is not available error on https://web-ai.studio/.

For your reference, https://web-ai.studio/ is an official demo page that I found in the official documentation: https://developer.chrome.com/docs/ai/prompt-api#demos.

[Felipe Marcos de abreu Aquino]

Thank you for the confirmation.

That aligns with my findings and understanding as well. Given that this is a known platform limitation rather than a misconfiguration, the current hybrid approach using browser-native STT/TTS alongside the Prompt API for on-device text inference is indeed the only viable and supported solution at this stage.

From an architectural standpoint, this separation of concerns is acceptable for now, although native audio exposure through the Prompt API would significantly improve latency, privacy guarantees, and developer ergonomics once available.

I’ll proceed with the hybrid implementation and will monitor future Chrome releases and related origin trials for any official exposure of audio capabilities in the Prompt API. If there are roadmap updates or experimental channels where audio support becomes available, I’d be very interested in evaluating them.

Thanks again for the clarification.

[Kenji Baheux]

Audio & Image inputs are supported in the Prompt API for the web (Origin Trial) and with the flags mentioned by the OP.

The fact that it works in chrome's internals page but not when using the API suggests that there is a bug or some other issue at play.

I've forwarded this thread to the team working on the Prompt API for further investigation.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/CAFr0sBiDnV1%2BQ5iKnv6EzXzN%2BwyJDixEXV4%2BJ3fC1A0ACf9%2BeQ%40mail.gmail.com.

--

Kenji BAHEUX (my how-to)

Product Manager - Chrome

Google Japan

[Alan Corn]

Thank you for your attention and response! Currently, we have tested the audio functionality on several computers with dedicated graphics cards, and it works properly. According to the official documentation: "GPU or CPU: Built-in models can run with GPU or CPU. GPU: Strictly more than 4 GB of VRAM. CPU: 16 GB of RAM or more and 4 CPU cores or more." We hypothesized that the audio functionality would still work properly with integrated graphics, but it appears that this is not the case. If the audio functionality has specific hardware requirements, could you please let us know the minimum required configuration? Thank you once again for your attention and response!

[Thomas Steiner]

Hi Alan,

Older Intel-based MacBooks don't support the audio modality either. My understanding is that you are using Windows without a dedicated GPU. This seems to be the same limitation as on those MacBooks. Once we hear the authoritative response from engineering about the limitation details, we'll update our documentation.

Cheers,

Tom

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/a4167546-b8eb-4504-a182-0101df1782b8n%40chromium.org.

--

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Spain, S.L.U.

Torre Picasso, Pl. Pablo Ruiz Picasso, 1, Tetuán, 28020 Madrid, Spain

CIF: B63272603

Inscrita en el Registro Mercantil de Madrid, sección 8, Hoja M­-435397 Tomo 24227 Folio 25

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.8 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----