model with expectedInputs: image crashes in measureInputUsage

[Die4Ever2005]

Just pasting code here since jsfiddle doesn't work anymore :(

await (await LanguageModel.create()).measureInputUsage('test string')
10

await (await LanguageModel.create({expectedInputs: [ { type:'text', languages: ['en'] } ]})).measureInputUsage('test string')
10

await (await LanguageModel.create({expectedInputs: [ { type:'text', languages: ['en'] }, { type: "image" } ]})).measureInputUsage('test string')
Uncaught OperationError: The usage cannot be calculated.Understand this error

await (await LanguageModel.create({expectedInputs: [ { type:'text', languages: ['en'] }, { type: "audio" } ]})).measureInputUsage('test string')
10

This code used to work for me. Also the crash limit is really annoying, forcing me to restart Chrome after 3 of these crashes.

await LanguageModel.create();
Uncaught NotAllowedError: The model process crashed too many times for this version.

Crashing in Version 140.0.7322.0 (Official Build) canary (64-bit)

Model Name: v3Nano
Version: 2025.06.30.1229

[Clark Duvall]

I'm not able to repro this error, can you send any crash report IDs you have in chrome://crashes from around the time you saw this crash?

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/7c869698-211c-4508-bd71-fbd6caacaeccn%40chromium.org.

[Die4Ever2005]

Nothing shows up in chrome://crashes/ for these errors

[Clark Duvall]

Ok one more thing to check: after this happens, can you go to chrome://histograms and see if you have anything recorded for OnDeviceModel.GpuErrorReason?

[Die4Ever2005]

The histogram is a lot lol, I switched it to monitoring mode and ran the code in the console. Nothing for GpuErrorReason, but I found some suspicious errors

+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.ClientSidePhishing recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.ClientSidePhishingImageEmbedder recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.GeolocationPermissions recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.GeolocationPermissionsV3 recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.LanguageDetection recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.ModelExecutionFeaturePromptAPI recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.ModelExecutionFeatureScamDetection recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.ModelExecutionFeatureSummarize recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.ModelExecutionFeatureWritingAssistanceApi recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.NotificationPermissions recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.NotificationPermissionsV3 recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.OmniboxOnDeviceTailSuggest recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.OmniboxUrlScoring recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.PageTopicsV2 recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.PageVisibility recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.PassageEmbedder recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.SegmentationChromeLowUserEngagement recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.SegmentationComposePromotion recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.SegmentationDeviceSwitcher recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.SegmentationFedCmUser recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.SegmentationSearchUser recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.SegmentationShoppingUser recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.TextSafety recorded 1 samples (flags = 0x41) [#]
+ Histogram: OptimizationGuide.PredictionModelFetcher.GetModelsResponse.NetErrorCode.URLVisitResumptionRanker recorded 1 samples (flags = 0x41) [#]

[Clark Duvall]

Can you download the full histograms.txt file after reproducing the issue and attach it? There should be a "Download" button near the top of the histograms page.

[Die4Ever2005]

attached

[Clark Duvall]

Thanks! Just to make sure, are you still able to repro the issue and was that histograms file from right after reproducing?

[Die4Ever2005]

yes, that was after repro

[Die4Ever2005]

Wait sorry, the histogram page needs a refresh button? does that affect the download too? I will make a new one, with a refresh this time

[Die4Ever2005]

attached, this time with a refresh

[Clark Duvall]

Hmm, nothing in the histograms looks suspicious but it does show there was a crash. I would have expected the crash to show up in chrome://crashes. What chrome version did you see the latest crash, and do you know what GPU your device has?

[Die4Ever2005]

Version 140.0.7322.0 (Official Build) canary (64-bit)

Model Name: v3Nano

Version: 2025.06.30.1229

RTX 2080

[Clark Duvall]

Would you be able to check if you see the same issue in the latest Chrome dev and beta? That might help narrow down when the problem started. Thanks so much for helping debug this!

[Die4Ever2005]

It works fine in stable Version 138.0.7204.169 (Official Build) (64-bit)

Model Name: v2Nano
Version: 2025.05.05.2308

I will check dev and beta later today.

[Die4Ever2005]

Broken in beta

Version 139.0.7258.52 (Official Build) beta (64-bit)

Model Name: v3Nano
Version: 2025.06.30.1229

[Clark Duvall]

Is there anything in chrome://crashes for beta? Also are you on Windows or Linux?

[Die4Ever2005]

Windows 10. Nothing in chrome://crashes/ in beta

Only the function crashes (throws), not the browser, so I guess that's why there's no crash dump for it.

[Clark Duvall]

Very strange... If you go to http://clarkduvall.com/ai/walkthrough.html, does the "Multi-modal Input" example work for you?

[Die4Ever2005]

#10 fails, the other tests seem to all work

the web page shows:

An error occurred:
undefined

the console shows:

UnknownError: Other generic failures occurred.
(anonymous) @ walkthrough.html:1187

[Clark Duvall]

Thanks for trying! My best guess is you may be running out of GPU memory when loading the image model. Can you try shutting down all chrome windows you have open or any other possibly GPU intensive applications, then attempt to repro again? If you have a way of monitoring GPU memory usage, that would be useful as well. If you can still reproduce, can you also send the histograms.txt file one more time?

Thanks again for working through this!

[Die4Ever2005]

I closed everything and Task Manager says it peaks at about 4GB used, still failed.

[Clark Duvall]

Ok this definitely looks like a crash of the process running the model that should show up in chrome://crashes. Do you have crash reporting turned on? Can you make sure it's turned on (explained here), and then try going to chrome://crash (no "es" at the end of that URL) which should intentionally crash the tab. This should definitely show up in chrome://crashes. Then repro the bug, and see if it gets another entry in chrome://crashes.

[Die4Ever2005]

"Help improve Chrome's features and performance" is enabled

opening chrome://crash did add to chrome://crashes

but running the measureInputUsage doesn't show up in the crashes list

[Clark Duvall]

The last way I can think of to debug this would be turning on debug logging and sending the chrome_debug.log file that gets generated in your user data directory. If that doesn't have anything interesting I can try adding some more debug logs for future canary versions.

[Die4Ever2005]

log generated from beta attached

[Clark Duvall]

The debug log does show there is a crash but for some reason the crash reporting is not happening, I'll add some more debug logging soon. If you go to chrome://on-device-internals/ in Canary and click "Load Default", does it load successfully? Are you then able to use the "Add image" button with some basic prompt (e.g. "describe this image") and get a result?

[Die4Ever2005]

Clicking "Load Default" says:

Model loaded from C:\Users\die4e\AppData\Local\Google\Chrome SxS\User Data\OptGuideOnDeviceModel\2025.6.30.1229 in 2857ms [images enabled][audio enabled]

But then after like 2 seconds it changes to say:

Service crashed, please reload the model.

It doesn't really give me a chance to try adding an image or running a prompt. I attached the log file, this is from Canary

[Die4Ever2005]

--no-sandbox fixes the issue, but the issue persists without that flag

Version 140.0.7327.1 (Official Build) canary-dcheck (64-bit)

Model Name: v3Nano
Version: 2025.06.30.1229

[Clark Duvall]

Update on this bug: this should be fixed with the latest model update. You can force update the model by going to chrome://components -> "Optimization Guide On Device Model" and clicking "Check for update". The fixed model version will start with 2025.8.11.