The model attempted to output text in an untested language...

[Wendel Pereira]

I'm constantly getting:

"The model attempted to output text in an untested language, and was prevented from doing so."

Almost for any content, it seems too much senstive, is there is a way to tune this?
Even looking on simple webpage content trigger this error.

[Kenji Baheux]

Hi Wendel,

Can you share some prompts where the language check unnecessarily kicks in?

Kenji

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/d4eb6ff6-e61b-4535-960b-ee6ed15ecb4cn%40chromium.org.

--

Kenji BAHEUX (my how-to)

Product Manager - Chrome

Google Japan

[Wendel]

Hi Kenji, here is a prompt example:

<INSTRUCTIONS>

Analyze this webpage content and give a safety percentage score from 0-100%. 

Consider: Adult, Gambling, Violent, Hateful, Deceptive, Spammy, Malware, Sentiment.

The value 0-100% represents the percentage of the content in the page that is of the type.

Suggest any other categories that are relevant, like: "Sexual, Drugs, etc."

Return example: 

[["adult", 0-100], ...]

</INSTRUCTIONS>

<START OF CONTENT>

Hacker News new | past | comments | ask | show | jobs | submit login

1.

Gothic Architecture Became Spooky (architecturaldigest.com)

44 points by teleforce 1 hour ago | hide | 20 comments

2.

Ask HN: What Are You Working On? (October 2024)

193 points by david927 14 hours ago | hide | 563 comments

3.

Practical Introduction to BLE GATT Reverse Engineering: Hacking the Domyos EL500 (jcjc-dev.com)

32 points by greesil 6 hours ago | hide | discuss

4.

Care Doesn't Scale (stevenscrawls.com)

119 points by surprisetalk 4 hours ago | hide | 84 comments

5.

NewPipe on Linux, Using Android_translation_layer (flathub.org)

255 points by FuturisticGoo 19 hours ago | hide | 70 comments

6.

A Chopin waltz unearthed after nearly 200 years (nytimes.com)

365 points by perihelions 1 day ago | hide | 117 comments

7.

You-get: Dumb downloader that scrapes the web (github.com/soimort)

305 points by Anon84 1 day ago | hide | 118 comments

8.

Hoard of coins from Norman Conquest is Britain'

<END OF CONTENT>

<RESPONSE FORMAT>

Please return ONLY valid JSON in the exact schema shown. 

Do not include any other text or explanation. 

The response must be parseable by JSON.parse().

</RESPONSE FORMAT>

And the response is:

[ML ERROR] NotSupportedError: The model attempted to output text in an untested language, and was prevented from doing so.

The text above is just our beloved https://news.ycombinator.com/ list.

[Kenji Baheux]

Thanks, I've shared this with the team.

[Lars H]

I have a similar problem when asking for output in a specific format.

Example prompt:

**Output format:**
- EntityName | Description | Confidence level (0-1) | Explain Confidence level


Output from the model:

NotSupportedError: The model attempted to output text in an untested language, and was prevented from doing so.

[Wendel]

Any update on this?

The way it is, it pretty unusable now.

[Thomas Steiner]

Thanks, I'll check back with the team if there's anything they can do. 

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/CANSKurMao02iSmXTYXVvALa5OAgpNnDEaW-eUtcaX3ROmbv1_w%40mail.gmail.com.

--

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Germany GmbH, ABC-Str. 19, 20354 Hamburg, Germany
Geschäftsführer: Paul Manicle, Liana Sebastian
Registergericht und -nummer: Hamburg, HRB 86891

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.3 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----

[Sagar Dewan]

Hi, 

Having the same issue. 
When we ask the prompt to output only a single word, it shows Untested Language error. Moroever, sometimes it gives the output and sometimes, it shows untested language. Tried adding and removing SharedContext parameter as well. 

[Thomas Steiner]

Please see yesterday's PSA (public service announcement) on a flag in Chrome that you can flip. This should at least work as a local workaround. 

Thanks,

Tom

[Vinayak Nigam]

Hey Wendel,
I have also noticed that when using the prompt API, if I provide instructions for a specific output format, this error frequently occurs.
Like JSON or any other very specific format.

[Thomas Steiner]

Yes, requiring a specific format definitely can trick the language detector into believing the output language is non-English. Share representative examples with us by filing bugs (https://new.crbug.com). We're definitely hearing the need for enforcement of an output format: https://github.com/explainers-by-googlers/prompt-api/issues/35. 

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/486ac34d-b3ed-4647-b1d4-56fe0d9c0ac7n%40chromium.org.

[Nico Martin]

I had this error a lot when using XML like syntax for the input and also told it to output XML.

I then switched to JSON. JSON never gave me the unsupported language error, however the JSON output has not always been valid so I had to add some checks before the JSON.parse..

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/486ac34d-b3ed-4647-b1d4-56fe0d9c0ac7n%40chromium.org.

[Thomas Steiner]

JSON has the problem of not being a streaming-friendly format https://youtu.be/surh_D8CU9A?si=fUDx6u9-lMYNe_c8 (around the 4:30 mark), but if it works for your use case and the model reliably enough respects your desired format, by all means use it. 

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/CAL8L5nDxwcXCrTdVoAfwMhVMjDJrKfSyBrBVkhKdjMqKjorP_w%40mail.gmail.com.

[Papillon]

Thomas, where can one find the PSA? Can you please link to it.

Thanks.

[Anil Kumar]

PSA link:
https://groups.google.com/a/chromium.org/g/chrome-ai-dev-preview/c/WNB2xMpMyGE