Support for Gemini Nano on 'regular' Chrome?

[Rishi Gorantala]

Hi! I've been conducting my experiments for Chrome's Built-in AI on Chrome Canary so far, but I just noticed the flags to enable Gemini Nano on my regular Chrome instance on my Mac - Version 131.0.6778.86 (Official Build) (arm64)

However it doesn't seem to work. Is this something that's supposed to work? If not, is it something that's coming soon?

Thanks!

-rishi

[Thomas Steiner]

Hi Rishi,

What once was Canary finally has become Stable, so in theory it should work if you flip the flags. Note that it's still opt-in, and also remember that any bug fixes (and there were many) will hit Canary, and then slowly trickle down over time. Long story short, I recommend for any testing you keep using Canary. This is also how we can be sure newly introduced bugs get caught as soon as possible, so please report it when you see something weird.

Cheers,

Tom

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/7652c198-e767-475c-b114-1ea1625efe5fn%40chromium.org.

--

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Germany GmbH, ABC-Str. 19, 20354 Hamburg, Germany
Geschäftsführer: Paul Manicle, Liana Sebastian
Registergericht und -nummer: Hamburg, HRB 86891

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.3 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----

[Rishi Gorantala]

Hi Thomas! Thanks, so I flipped the flags and tried to get it working but am having trouble. Is it required that I uninstall Canary for this to work?

I understand Canary will have the latest but am trying to see if there's a way I can share my chrome extension easily with friends etc. without asking them to download dev/Canary.

--

Regards,
-rishi

[Thomas Steiner]

Hi Rishi,

On Fri, Dec 6, 2024, 20:51 Rishi Gorantala <rishi...@gmail.com> wrote:

Hi Thomas! Thanks, so I flipped the flags and tried to get it working but am having trouble. Is it required that I uninstall Canary for this to work?

No, it's not required. Can you try forcing it with `chrome://flags/#optimization-guide-on-device-model` set to bypass perf requirements check?

I understand Canary will have the latest but am trying to see if there's a way I can share my chrome extension easily with friends etc. without asking them to download dev/Canary.

For the extension, you can take part in the origin trial: https://developer.chrome.com/docs/extensions/ai/prompt-api#participate_in_the_origin_trial. 

[Rishi Gorantala]

Thanks Thomas. So after some tinkering got it working on Chrome Version 131.0.6778.109 (Official Build) (arm64) for Mac OS:

• I think running both Canary and regular Chrome at the same time was causing issues somehow. I closed Canary.
• I then set the flag for  Enables optimization guide on device to Enabled BypassPerfAndTextSafety first and it started working. I then flipped it back to Enabled BypassPerfRequirement.

Now I have one challenge/question. Is it required that I change the extension code to use chrome.aiOriginTrial.languageModel or can I continue to use ai.languageModel api for origin trials? I'm having trouble with the chrome.aiOriginTrial API. See screenshot below:

--

Regards,
-rishi

[Thomas Steiner]

Hi Rishi,

Glad to hear it worked. In an extension contact, you need to closely follow the origin trial instructions for the  chrome.aiOriginTrial.languageModel namespace to work. 

Cheers,

Tom

[Rishi Gorantala]

Not sure what I am missing here because that's what I did. It seems like the new namespace functions don't work. Any ideas? I tried the functions from with a chrome extension's content.js and not just dev console too, but still get the same error.

--

Regards,
-rishi

[Thomas Steiner]

It's most likely something with the key in the extension manifest, especially since you have the API working in a web app context. Closely follow the instructions and links in the article, it's a bit fiddly, but should work. 

Cheers,

Tom

[Rishi Gorantala]

So I tried everything all over again, but no luck with namespace chrome.aiOriginTrial. The older ai.languageModel API still works fine in my Chrome extension locally. Either I'm missing something trivial or there's an issue with the new namespace API working when the chrome extension is just content.js (no html?)

I checked the key, chrome extension id, trial token are all matching everywhere and are not corrupted. Anyone have any ideas?

--

Regards,
-rishi

[Thomas Steiner]

Hey Rishi,

Does the example extension work for you? https://github.com/GoogleChrome/chrome-extensions-samples/tree/main/functional-samples/ai.gemini-on-device 

Cheers,

Tom

[Rishi Gorantala]

So I built the example extension and it works fine on my browser. The only key difference I see is that I don't use sidePanel; mine is just content.js. Does that matter?

--

Regards,
-rishi

[Sebastian Benz]

Content scripts work slightly different: https://developer.chrome.com/docs/web-platform/origin-trials#:~:text=Content%20scripts%20are,your%20extension%20ID.

It might be easier to send a message to your service worker, run the prompt API there and return the result to the content script.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/CAEMEAspsB%3DREWAn1SQVg_u0MDf1JQJ022EJCVw4KDV3upKACiQ%40mail.gmail.com.

Dr. Sebastian Benz

Developer Relations Engineer

Google Germany GmbH

Erika-Mann-Straße 33

80636 München

Geschäftsführer: Paul Manicle, Liana Sebastian

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls Sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde. 

     

This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

[Rishi Gorantala]

Thanks Sebastian, I saw that and followed the steps in there on how to programatically add to script. Let me try what you just said.

--

Regards,
-rishi

[Rishi Gorantala]

Looks like moving it to a service worker and using Chrome's runtime message passing did the trick. Thanks Thomas and Sebastian for your help.

One final question: If I'm distributing the extension under origin trials, do I still ask end users to enable those 2 flags on their Chrome browsers as per built in AI documentation?

--

P.S. I did spend more time trying to get it  to work from content.js directly but ran into CSP errors etc. so maybe this is an exercise for another day.

--

Regards,
-rishi

[Thomas Steiner]

On Tue, Dec 10, 2024 at 12:59 AM Rishi Gorantala <rishi...@gmail.com> wrote:

Looks like moving it to a service worker and using Chrome's runtime message passing did the trick. Thanks Thomas and Sebastian for your help.

Yay, glad it's working now!

 

One final question: If I'm distributing the extension under origin trials, do I still ask end users to enable those 2 flags on their Chrome browsers as per built in AI documentation?

Not having your users go through these hoops is the point of the origin trial. Once everything is properly set up, Chrome will just enable the API for your users. 

 

P.S. I did spend more time trying to get it  to work from content.js directly but ran into CSP errors etc. so maybe this is an exercise for another day.

As Sebastian said, this is most likely not the best way forward, also for the reason with CSP that you mentioned, but also because of the need to set the HTTP header. You can communicate the responses of the AI model between the various components of your extension if you need to. 

Cheers,

Tom

[Rishi Gorantala]

Thanks Thomas. 

> Once everything is properly set up

If the extension works locally when loaded unpacked in Dev mode, with the new chrome.aiOriginTrial API, is that an indication things are setup fine wrt origin trials; or is there more I should look for before it's published to the Chrome store?

--

Regards,
-rishi

[Thomas Steiner]

If it works with the chrome.aiOriginTrial namespace, you can be certain that it will work for external users who download the extension. Just remember that they will need to download the model, so make sure you have a decent UI that conveys the downloading state. The highlighted bold part isn't that UI ;-) but it shows you where you need to plug it.

const session = await chrome.aiOriginTrial.languageModel.create({
  monitor(m) {
    m.addEventListener("downloadprogress", e => {
      console.log(`Downloaded ${e.loaded} of ${e.total} bytes.`);
    });
  }
});

[Rishi Gorantala]

Thanks Thomas!

--

Regards,
-rishi

[Extension Rocks]

Hi, Thomas, I have a follow-up question to this. After I successfully downloaded the 2555207866 bytes (2.38 GB) of Gemini Nano for the first time, how can I delete it so that I can re-download it in order to test the "download progress bar" feature?

Thanks

Andre

[Thomas Steiner]

Hi Andre,

This was recently discussed in another thread. Hope this works for you!

Cheers,

Tom