First-time Chrome Extension developer's journey setting up Gemini Nano and Prompt API

[Lars H]

As a first-time Chrome Extension developer, I encountered several challenges while setting up the Prompt API with Gemini Nano.

This guide documents my journey step-by-step and the issues I faced along the way.

I found that additional steps were needed to get everything working properly.

I hope this guide can help others.

- -

1. My starting point: Artificial Intelligence Chrome for Developers

https://developer.chrome.com/docs/ai

2. I clicked on the Prompt API

https://developer.chrome.com/docs/extensions/ai/prompt-api

3. I decided to test the Prompt API using the demo extension available on GitHub:

https://github.com/GoogleChrome/chrome-extensions-samples/tree/main/functional-samples/ai.gemini-on-device

4. I followed the instructions as described in the section "Running this extension" on the Github repo page:

"Clone this repository.
Run npm install in the project directory.
Run npm run build in the project directory to build the extension.
Load the newly created dist directory in Chrome as an unpacked extension.
Click the extension icon.
Interact with the Prompt API in the sidebar."

5. I did the steps in point 4 but it didn't work. After some research I found out that I first had to do several other steps first. I guess this would be obvious for others who had developed Chrome extensions before.

I following this documentation:
https://developer.chrome.com/docs/extensions/ai/prompt-api#participate_in_the_origin_trial

6. Obtain the Origin Trial token (for later use in the manifest.json file)

Sign up for the "origin trial" at:

https://developer.chrome.com/docs/extensions/ai/prompt-api

Link is in the section "Availability":

https://developer.chrome.com/origintrials/#/view_trial/320318523496726529

I had some trouble to figure out what to put in the "Web Origin" field.

I found out that the format should be: chrome-extension://<extension-id>

To get the <extension-id> I had to load the extension into the browser first.

So I went back to the terminal to create a build of the extension.

7. Make an initial build:    npm run build

8. Load the extension in Chrome to get the extension-id:

Go to:    chrome://extensions/

Enable "Developer mode" (upper right corner)

Click "Load unpacked" (upper left corner) and navigate to your local project folder where you have the prompt-ai demo app and select the 'dist' directory (that was created by the build command)

Now copy the extension-id shown in the "Chrome Prompt AI Demo" extension card (it will look something like "abcdefghijklmnopqrstuvwxyz").

9. Go back to the Chrome Origin Trial sign up page

Paste the Extension ID in Web Origin field so it looks like this: chrome-extension://<your-extension-id>

Click "REGISTER"

Copy the token.

10. Open the manifest.json file in the project directory in the IDE.

Update the trial_tokens array with your new token: "trial_tokens": ["YOUR_GENERATED_TOKEN"]

Delete the existing "key" field:
"key": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvf0O/bR3JULoj6dOpG7sDif4BNVgootUIfSybh2a7jX47BglfZFNH/aRUgDjNtcTBPinXdGbljMVIudQ7w6LiwVq9b1Ht6ZXFVtHTKOsDWtVh/rVKE/AGue9eQ7xCncHFl4zLJUaDRUIRqe5zvjHtaMr8p92I3c/6k43LmTUp1QHz0NooDJRYKRPLS77YVDX8hZc2yopIH5NIY25Ned3wxZ/NWV70GZkYqFRN+UzvMS8bJUEY23L1AMSX7YQjMThY0BCZ/MBLo8UBLs8vN11EphMpLxnBhF2Zwwj2sCPR0jn0ev8HYCtKmGx8nzOl79oK24RFIsW8YWFB2fd28fBLwIDAQAB",

11. Build the extension again:    npm run build

12. Go to Chrome and reload the extension by clicking the refresh icon on the "Chrome Prompt AI Demo" extension card

13. Click the "Chrome Prompt AI Demo" extension icon in Chrome. The extension should open as a sidebar in the right side of the browser. The extension will not work (yet) as you didn't install the Gemini Nano model and Prompt API yet. 

(You can activate the Inspect / Dev Tool window in the extension side panel to run some commands in the command line. See later steps)

14. Enable Gemini Nano and Prompt API in Chrome

In Chrome navigate to: chrome://flags/#optimization-guide-on-device-model
Find "Enables optimization guide on device"
Select "Set Enabled BypassPerfRequirement" in the drop down menu.


Navigate to: chrome://flags/#prompt-api-for-gemini-nano
Find "Prompt API for Gemini Nano"
Select "Enabled" in the drop down menu.

These settings configure Chrome to download the Nano model. Download will be initiated when Chrome has restarted.

15. Quit Chrome
(I also tried "Relaunch" but for some reason it didn't seem to work for me).

16. Start Chrome again.

17. Go to: chrome://components and look for "Optimization Guide On Device Model"

It should say:

- -
Optimization Guide On Device Model - Version: 0.0.0.0
Status - Downloading
Check for update
- -

In other words - the model has started to download. It can take some time. 

18. Track download progress (optional)

I clicked on the "Chrome Prompt AI Demo" icon to open the extension in the side panel.

Open the Inspect window by right clicking in the extension in the side panel.

In the command line execute this code:

const session = await chrome.aiOriginTrial.languageModel.create({
  monitor(m) {
    m.addEventListener("downloadprogress", (e) => {
      console.log(`Downloaded ${e.loaded} of ${e.total} bytes.`);
    });
  },
});

You should now see this rolling output that shows the status on the download process:

Downloaded 821290896 of 2555207866 bytes.
VM207:4 Downloaded 821307280 of 2555207866 bytes.
VM207:4 Downloaded 821323664 of 2555207866 bytes.
VM207:4 Downloaded 821336444 of 2555207866 bytes.
VM207:4 Downloaded 821340048 of 2555207866 bytes.
...
...
...
Downloaded 2555207866 of 2555207866 bytes.

It took approx 10 min to finalize the download.

(Size of model: 2555207866 bytes = 2.38 GB)


19. Check Chrome's AI model download status in "components"

Go to: chrome://components/

The status for the "Optimization Guide On Device Model" component has now changed from "Downloading" to "Updated":

- - -
Optimization Guide On Device Model - Version: 2024.9.25.2033
Status - Updated
- - -

When I clicked the "Check for update" button the status changed to:

- - -
Optimization Guide On Device Model - Version: 2024.9.25.2033
Status - Up-to-date
- - -


20. Verify Gemini model is ready

To verify Gemini model availability in Chrome, you can check both the main browser context and your extension's context separately.

First, open Chrome's main DevTools (press F12 or right-click anywhere and select 'Inspect'). In the Console tab, paste this command to check if the Gemini model is available in your browser:

const capabilities = await window.ai.languageModel.capabilities();
console.log('Model status:', capabilities.available);

Output: readily

Secondly, open DevTool in Inspect window for the extension panel and run the same command:

const capabilities = await ai.languageModel.capabilities();
console.log(capabilities.available);

Output: readily


21. Verify Prompt AI API access is available

Now check that the extension has access to the AI API.

Click the extension icon (if the extension is not already open) to open its side panel, then right-click the side panel and select 'Inspect'. Paste these commands into the console:

console.log('Chrome version:', /Chrome\/([0-9.]+)/.exec(navigator.userAgent)[1]);
console.log('AI API present:', !!chrome.aiOriginTrial);
console.log('Language model present:', !!chrome.aiOriginTrial?.languageModel);

Output:

Chrome version: 132.0.0.0
AI API present: true
Language model present: true

This verifies that your extension has proper access to the API.

Try to run the same commands the main browser's DevTools context and you will get this output:

Chrome version: 132.0.0.0
AI API present: false
Language model present: false

They appear to be using different APIs (window.ai vs chrome.aiOriginTrial) because they're accessing the AI capabilities through different contexts.

Try running this instead:

console.log('Chrome version:', /Chrome\/([0-9.]+)/.exec(navigator.userAgent)[1]);
console.log('AI API present:', !!window.ai);
console.log('Language model present:', !!window.ai?.languageModel);

Chrome version: 132.0.0.0
AI API present: true
Language model present: true

I cannot exactly explain how this works. But I guess it is important to be aware of.


22. You can try to run this test to simulate a session creation to interact (chat) with the Gemini Nano model.

try {
    // Create a session
    const session = await ai.languageModel.create({
        systemPrompt: 'You are a helpful and friendly assistant.',
        temperature: 0.8,
        topK: 3
    });
    console.log('Session created successfully');

    // Use the session to chat
    const response = await session.prompt('Hello, how are you?');
    console.log('Model response:', response);

} catch (error) {
    console.error('Error:', {
        name: error.name,
        message: error.message,
        stack: error.stack
    });
}

After some seconds you should see this output:

Session created successfully

Model response: Hello! I'm doing great, thank you for asking. How are you?  I'm ready to assist you with anything you need.  Let's chat! 😊

- - -

Thats it !

Please add comments if something is missing or clarification is required.

[Lars H]

My environment:

MacOS 14.6.1

Google Chrome Canary: Version 133.0.6865.0 (Official Build) canary (x86_64)

[Thomas Steiner]

Hi Lars,

The extension from GitHub already has an origin trial token and a key in the manifest. After downloading it from GitHub, you can just add it to Chrome in developer mode, and Chrome will then take care of downloading the model itself. Only if you build your own extension you have to go through the steps that you have described. 

Cheers,

Tom

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/9c80de43-13a5-4154-904c-908e984f11bbn%40chromium.org.

[Lars H]

Thanks Thomas ! 👍

That would have helped quite a lot during my initial testing 😊

My intention is to build my own extensions and follow the development of the Gemini Nano quite closely.

L

[bhanusupraja99]

Thank you Lars and Thomas for sharing the details. 
It saves a lot of time for a beginners like me who are trying to set up.

Thanks,

Bhanu

[bhanusupraja99]

Hi Thomas,

Thank you for your earlier guidance! I have a quick question: While building my own extension, do I need to have a separate key in addition to the origin trial token? If yes, could you please let me know how to generate or include it?

Appreciate your help!

Best regards,
Bhanu

[Vinayak Nigam]

Hey  bhanusupraja99,

You might have noticed that you have an Extension ID which you have to input in Origin Trial while issuing a Origin Token.

This ID is basically randomly generated and may change when you change the location of the extension and if you want an absolute fixed ID then you can use the "key" in `manifest.json` file to fix that key and the steps are explained below in the link
This is explained in https://developer.chrome.com/docs/extensions/ai/prompt-api#participate_in_the_origin_trial, search for "key"