Does Chrome’s Prompt API initialPrompts obey the 1,024-token limit or full 4,096-token context?

[Pius Shedrach]

For the built-in Prompt API, there is a documented per-prompt input limit of about 1,024 tokens, and a session context window of up to ~4,096 tokens. 

If I have a single text input that exceeds 1,024 tokens (but is still under 4,096 tokens) and I want to use it as the initial prompt in a session, say in 

const session = await LanguageModel.create({

  initialPrompts: [
    { role: 'system', content: '…' },
    { role: 'user',   content: {very long text} }
  ]
});

will the API allow me to pass that long text in one go (i.e., exceed the per-prompt limit) and still take advantage of the full 4,096-token session context window? 

[Thomas Steiner]

Hi Pius,

I'm not sure where you're looking, but the current inputQuota is 9216 tokens (and we don't document this, as it has changed in the past already). Instead, users should rely on the reported `inputQuota` and `inputUsage`. For example, Edge's implementation of the Prompt API has a different `inputQuota`. I'm not aware of a per-prompt token limit. Please only ever rely on information from the articles linked in this overview, or for the shipped APIs (Summarizer, Translator, Language Detector) on the information on MDN (example). LLMs have their knowledge cut off too early, so there's a lot of outdated information in various LLMs right now. This will eventually improve, but right now, I can't recommend working with LLMs, unless you make them aware of the updated documentation. 

Cheers,

Tom 

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/67958a90-8543-450f-b05c-089e6a546fcen%40chromium.org.

--

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Spain, S.L.U.

Torre Picasso, Pl. Pablo Ruiz Picasso, 1, Tetuán, 28020 Madrid, Spain

CIF: B63272603

Inscrita en el Registro Mercantil de Madrid, sección 8, Hoja M­-435397 Tomo 24227 Folio 25

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.8 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----