Things seemed to have changed in the latest Gemini Nano install in Chrome.

[Colin Mummery]

I spent recent weeks adding optional Nano based features to my free HTML/JS screenwriting app 'MovieScripter' for when it runs in Chrome. Everything has gone great and even though I've been working on an older i7 PC with no VRAM, Nano performance  has been acceptable for the Prompt and Rewriter APIs. But yesterday I was testing Nano installation with the Chrome Dev channel and there were two changes  that I noticed:

1. The Prompt API Nano install is bigger, around 5GB (closer to Phi-4-mini size) and when using the Rewriter API there was no separate download required. It seems the Prompt API Nano model can now also service the Rewriter API, is that correct?

2. On my PC with no VRAM, I got the dreaded 'generic error' when the app tried to use the Prompt API even though Nano was successfully installed. I tried again on a PC with VRAM and this time, Nano responded correctly to Prompt API calls. When I looked for problems I opened chrome://on-device-internals/ on the PC with no GPU and found that Nano had set its 'Backend Type' parameter to 'GPU (highest quality)' whereas my previous working Nano installation on the no-GPU PC has that backend parameter correctly set to 'CPU' . I assume this is the source of the problem with using the Prompt API. A known bug? Deliberate behavior?

 Footnote:  In flags 'Prompt API for Gemini Nano' is always enabled and  'Enables optimization guide on device' is set to 'Enabled BypassPrefRequirement'. 

[Seriki Olalekan]

I see no reason why everytime i make changes to what I am building privately people will always have access to it 

If I am building for google just let me know 

I can’t be working and cracking my brain for another organisation without acknowledgment or compensation, 

This is not fair 

Inline images
AURA
Ecosystem, Qubuhub
gravatar.com/167559384web4application
 

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/bef9834d-2d1e-4e7e-8e3d-1ce6042ee0d7n%40chromium.org.

[Colin Mummery]

Update on original post: I create a Chromium issue for the 'Backend Type' problem : https://issues.chromium.org/issues/467298238 and it seems to getting some attention.

[Thomas Steiner]

Hi Colin,

Thanks for filing the bug, it certainly looks like one :-) To answer your questions, see inline… 

1. The Prompt API Nano install is bigger, around 5GB (closer to Phi-4-mini size) and when using the Rewriter API there was no separate download required. It seems the Prompt API Nano model can now also service the Rewriter API, is that correct?

Internally, the Gemini Nano model serves the Rewriter API as well using a set of system prompt. You can see this if you use the Rewriter API while you debug Gemini Nano (you can just open the DevTools Console and type in `await (await Rewriter.create()).rewrite('foo')`).

 

2. On my PC with no VRAM, I got the dreaded 'generic error' when the app tried to use the Prompt API even though Nano was successfully installed. I tried again on a PC with VRAM and this time, Nano responded correctly to Prompt API calls. When I looked for problems I opened chrome://on-device-internals/ on the PC with no GPU and found that Nano had set its 'Backend Type' parameter to 'GPU (highest quality)' whereas my previous working Nano installation on the no-GPU PC has that backend parameter correctly set to 'CPU' . I assume this is the source of the problem with using the Prompt API. A known bug? Deliberate behavior?

Looks like a bug, as mentioned above.

 

 Footnote:  In flags 'Prompt API for Gemini Nano' is always enabled and  'Enables optimization guide on device' is set to 'Enabled BypassPrefRequirement'. 

The flag `chrome://flags/#optimization-guide-on-device-model` has a value "Enabled BypassPerfRequirement" (Not "Pref", but "Perf"), so you're force-enabling the model despite it not fully meeting the performance checks. 

Cheers,

Tom

 

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/bef9834d-2d1e-4e7e-8e3d-1ce6042ee0d7n%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/7fec29e7-83da-4ef5-a66c-d8d01e3444b6n%40chromium.org.

--

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Spain, S.L.U.

Torre Picasso, Pl. Pablo Ruiz Picasso, 1, Tetuán, 28020 Madrid, Spain

CIF: B63272603

Inscrita en el Registro Mercantil de Madrid, sección 8, Hoja M­-435397 Tomo 24227 Folio 25

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.8 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----

[Colin Mummery]

Thanks for the reply Thomas!

[Colin Mummery]

Just a update on the 'Backend Type' issue. The Chromium folks asked me to run the install again but without changing any flags and it worked! Backend was correctly set to 'CPU' and I could get prompt responses from Nano. I asked what I should recommend to my app's users and to quote their reply: " In general, end-users should never modify chrome://flags. Doing so is very likely to put them in an unsupported or broken combination of configuration settings."

[Thomas Steiner]

Yes, that's very good advice! End users should never touch any flags. Glad it's working as expected now :-)