initialPrompts size limits

[Raymond Camden]

The docs for initialPrompts say:

If the combined token length of all the initial prompts (including the separate systemPrompt, if provided) is too large, then the promise will be rejected with a "QuotaExceededError" DOMException.

Is there any kind of rough idea of when a user may expect to hit this? 100 items? 1000? 

I understand the size of each item has an impact, but I'm just curious about the base expectation here.

[Raymond Camden]

Duh, that makes complete sense now that you've said it. :) 

On Tue, Sep 3, 2024 at 3:27 PM Timo Witte <timo....@gmail.com> wrote:

This is probably a hardcoded limit which is the max number of tokens the model will accept. A token is loosly a word, but not always. Depends on the tokenization strategy they use internally.
They might have a limit on the number of Vectors after embeedding these tokens as well, as this is typically the limiting factor regarding the context window the LLM can process.

--

===========================================================================
Raymond Camden

Email : raymon...@gmail.com
Blog : www.raymondcamden.com
Twitter: raymondcamden

[Timo Witte]

This is probably a hardcoded limit which is the max number of tokens the model will accept. A token is loosly a word, but not always. Depends on the tokenization strategy they use internally.
They might have a limit on the number of Vectors after embeedding these tokens as well, as this is typically the limiting factor regarding the context window the LLM can process.

On Tuesday, September 3, 2024 at 8:06:31 PM UTC+2 Raymond Camden wrote:

[Cuyler Stuwe]

According to Google (

https://ai.google.dev/gemini-api/docs/tokens):

“For Gemini models, a token is equivalent to about 4 characters. 100 tokens is equal to about 60-80 English words.”

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/CAK2avF-Qgma9HQdEziGHpZx21dK7b6QYnuGtFutAUpjkJgR3rg%40mail.gmail.com.

[eyal...@gmail.com]

That's a real bammer, each time I ooen it I get 1 response and that's it for that day. 

The models are downloaded to the client so, why is there a limitation on number of API calls? 

בתאריך יום ג׳, 3 בספט׳ 2024, 23:30, מאת Timo Witte ‏<timo....@gmail.com>:

--

You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/2bfaaf4e-68d5-4c1f-b517-2b6aa05a1b7fn%40chromium.org.

[Sebastian Benz]

With the latest API update you can query the token limit and the number of tokens used. You can play around with the limits in Tom's demo. Afaik there's no limit on API calls.