Browser Built-in AI Takes Too Long To Give Results

[C K.K]

Hi Google Chrome Team,

Celestino here. Integrated Prompt API and Summarizer API, but the time to output takes too long, specifically the Summarizer API takes a total of 133 seconds when I ask it to summarize a product page for the extension's user on Amazon for example. 

I think this is something you can work on, unless there is a way I can optimize this further on Chrome settings? I need it to take very few seconds, like the native experience I get when using Gemini Web App or ChatGPT

Kindly advise.

Please see screenshots

Kind Regards,

Celestino

[C K.K]

It takes now up to 4 minutes to successfully call the Prompt API 

[Connie Leung]

Will streaming response improve the user experience?

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/6da675fd-d4b7-4ca2-8881-98f6886219afn%40chromium.org.

[Thomas Steiner]

Hi Celestino,

It's a known challenge that the API has a bit of a "cold start" problem where the first prompt would take a couple of seconds until the first token is emitted, but four minutes sounds excessive. Does this reproduce on another device? If you're using the `summarize()` method, can you try the `summarizeStreaming()` method (even if you end up just concatenating the chunks and work with the final result)? I'm interested in hearing your results for when the first token is emitted. Happy to take a look at the code as well, feel free to share it in private if it can't be shared publicly.  

Cheers,

Tom

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Spain, S.L.U.

Torre Picasso, Pl. Pablo Ruiz Picasso, 1, Tetuán, 28020 Madrid, Spain

CIF: B63272603

Inscrita en el Registro Mercantil de Madrid, sección 8, Hoja M­-435397 Tomo 24227 Folio 25

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.8 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/CA%2Borrmmu96XR89B3D2kADrqS0RKfCkyqbJpKeKoS_n7du1o-Kg%40mail.gmail.com.

[Kenji Baheux]

Hi Celestino,

Sorry to hear about such high latency.

Would you mind sharing details about your device (OS, CPU, GPU, Memory RAM and/or VRAM, SSD or HDD, etc) so we can investigate this further?

Also, a GPU report downloaded from chrome://gpu could be helpful. Probably best to send this directly to me instead of the group.

Thanks in advance!

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/CALgRrLmC7dMK5%2BdWpfsnFTZR5Z9qeF5ZmWLR5QhjLgVhqzaoGg%40mail.gmail.com.

--

Kenji BAHEUX (my how-to)

Product Manager - Chrome

Google Japan

[C K.K]

Hi Connie, 

I have tried adopting the streaming experience and still no improvements

[C K.K]

Hi Thomas,

Still lots of problems, sharing code in an hour or less

[C K.K]

Attached is the GPU report, and system information(dxdiag file)

[Kenji Baheux]

From a quick look, there seems to be a problem with your GPU AMD Vega 10 (error code 43) which results in having it disabled in Chrome.

An online search suggests that it might be a driver problem. Can you uninstall and reinstall your AMD drivers?

On Tue, Oct 28, 2025 at 2:36 AM C K.K <kcelestin...@gmail.com> wrote:

Attached is the GPU report, and system information(dxdiag file)

[C K.K]

My display driver has a problem and HP nor AMD has updated the drivers. So do you mean the display driver affects Chrome AI API compatibility?

[C K.K]

Hi Kenji, Tom & Connie,

I'm going to have to submit it anyways. even when I install and uninstall the drivers, no luck, I wish we can decouple the AI APIs and how they work from many of the resources and operations at the OS level like drivers.

[C K.K]

I'll still be building after the hackathon :) 

[Kenji Baheux]

In general, driver issues can make the GPU unavailable or unstable and therefore not being used in Chrome (AI or not).

In the logs, there is a fallback "GPU" in the form of Microsoft Basic Render Driver but it's blocked because it crashes too often or is too slow.

That said, if you have at least 16GB of RAM then you should automatically fallback to the CPU inferencing / model.

Can you try Chrome canary?