How do you handle Large 'system' messages (PromptAPI)

[Arinze Ogbonna]

For user prompt messages that are large, the messages can be chunked into smaller messages, but what if the system message is large?

At the moment, I only receive the error from the PromptAPI and send it to the user. 

If I calculate the tokens and they exceed the required tokens, how best can I handle the leftovers? 

And as per the documentation, there should only be one "system" message in the initialPrompt.

[Thomas Steiner]

Hi Arinze,

You can measure the input usage of a prompt with the `session.measureInputUsage()` function. The system prompt consumes tokens as well. You can see the consumed tokens by looking at `session.inputUsage`. Hope this helps. 

Cheers,

Tom 

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Spain, S.L.U.

Torre Picasso, Pl. Pablo Ruiz Picasso, 1, Tetuán, 28020 Madrid, Spain

CIF: B63272603

Inscrita en el Registro Mercantil de Madrid, sección 8, Hoja M­-435397 Tomo 24227 Folio 25

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.8 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/7edb9bcb-7cae-4be8-9945-0f9c1914dd90n%40chromium.org.

[Arinze Ogbonna]

That's what I'm doing at the moment, but it still far exceeds. 

I want to parse and make sense of large terms of service documents on each page a user visits. 

The content are always big and most times exceeds the limit, with the Summarize API, I was batching and processing them (a chunk of valid token size at a time), the overall output was poor compared to the PromptAPI, so I switched and I can't seem to do same with it. 

Any recommendations for a way forward for me? 

My requests will always exceed the limits (on most sites)

[Thomas Steiner]

On Tue, Oct 28, 2025 at 1:13 AM Arinze Ogbonna <arinz...@gmail.com> wrote:

That's what I'm doing at the moment, but it still far exceeds. 

I want to parse and make sense of large terms of service documents on each page a user visits. 

The content are always big and most times exceeds the limit, with the Summarize API, I was batching and processing them (a chunk of valid token size at a time), the overall output was poor compared to the PromptAPI, so I switched and I can't seem to do same with it. 

Any recommendations for a way forward for me? 

My requests will always exceed the limits (on most sites)

Sounds like you're really hitting the limits of the API's context window. Would it be possible to send an example of such a site to @Kenji Baheux so he can see if there's a reasonable way for us to enlarge the context window? Just in case, it also helps if you just send the raw text (via Element.innerText), not an HTML structure. 

--