prefix doesn't work for JSON output?

[Die4Ever2005]

I'm trying the new prefix API like

{role:'assistant', content: prefix, prefix: true}

but it seems like it's being ignored for JSON output? I was trying to do a prefix like 

{"firstProperty":"

but it's completely ignored. Maybe internally you're already doing this? Maybe even always prefilling property names and only prompting the LLM for the values of them?

[Thomas Steiner]

Are you pairing this with a `responseConstraint`? Can you share (publicly or privately) what you're trying to achieve?

Cheers,

Tom

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Spain, S.L.U.

Torre Picasso, Pl. Pablo Ruiz Picasso, 1, Tetuán, 28020 Madrid, Spain

CIF: B63272603

Inscrita en el Registro Mercantil de Madrid, sección 8, Hoja M­-435397 Tomo 24227 Folio 25

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.8 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/3024c6c8-c716-417c-9197-198652d14ad8n%40chromium.org.

[Die4Ever2005]

Yes, with responseConstraint. I was basically just wondering as a performance optimization to specify the prefix as

{"firstProperty":"

Or even just

{

But maybe this is already done internally. It could even be skipping generation for all of the property names? You could have code looking for the closing quote of a value or ending comma, and then inject the next characters for valid JSON without generating it through the LLM. Like if the LLM outputs the closing quote, you can then inject like ,"secondProperty":"

[Thomas Steiner]

Thanks for clarifying. If you're already using a `responseConstraint`, the `prefix` option won't be needed. It's more meant for things you can't (easily) constrain via JSON Schema or a regular expression. 

Cheers,

Tom

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Spain, S.L.U.

Torre Picasso, Pl. Pablo Ruiz Picasso, 1, Tetuán, 28020 Madrid, Spain

CIF: B63272603

Inscrita en el Registro Mercantil de Madrid, sección 8, Hoja M­-435397 Tomo 24227 Folio 25

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.8 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----

[Die4Ever2005]

Ok thanks, just making sure I wasn't leaving performance on the table.

[Scott Fortmann-Roe]

+1 to this

One use case we have is where we're building a spell checker and we want the response to be of the form:

User:  {"input: "[Input to correct]"}

Assistant: {"corrected": "[The corrected output]"}

We can use responseConstraint to guarantee the correct output form.

Now, if we correct some content for the user and then the user adds new content at the end of it, we want to correct the new content without having to re-correct the old content.

To do this, we want to be able to do the following:

User:  {"input: "[Already corrected input][Added input]"}

Assistant Prefix: {"corrected": "[Already corrected output]     <-- we include the already corrected output as a prefix so it's not recomputed, we don't terminate the JSON object

Assistant Generated: [Corrected added input]"}    <-- the assistant just generates the corrected form of the added text and then closes the JSON object

Thanks! Scott

[Thomas Steiner]

Adding in @Domenic Denicola and @Clark Duvall for thoughts on how (or if) `responseConstraint` and `prefix` should interact. Currently, it seems like the `prefix` is ignored if a `responseConstraint` is set. 

--

[Clark Duvall]

I think this could be possible to support, but curious what the expected behavior would be if the prefix given does not match the constraint. Would an error be thrown in that case?

On Sun, Jul 27, 2025 at 9:20 PM Domenic Denicola <dom...@google.com> wrote:

This is pretty interesting. It seems like we should be able to make this work, and so we can treat this as a Chrome bug. But, Clark would be the real expert here.

If we can't make it work, we should throw an exception to make it clearer.

[Scott Fortmann-Roe]

When the `responseConstraint` can't be satisfied for the specified `prefix`, I think throwing an error makes sense.

Cheers, Scott

On Mon, Jul 28, 2025 at 6:12 AM Domenic Denicola <dom...@google.com> wrote:

That makes the most sense to me. (Although if the developers on this thread have a different opinion, I'd be eager to hear it.)

[Clark Duvall]

Filed crbug.com/434766400 to track this.