[Intent to Experiment] Prompt API Sampling Parameters
Deepti Bogadi
Contact emails
mem...@google.com, dbo...@google.com, m...@google.com
Explainer
https://github.com/webmachinelearning/prompt-api?tab=readme-ov-file#sampling-parameters
Specification
https://webmachinelearning.github.io/prompt-api/
Summary
Adds sampling parameters to the Prompt API. These control how tokens are sampled from the model, giving developers control over the "creativeness" or "randomness" of the output. Additionally, it adds attributes to the LanguageModel instance to read the set values, as well as a static LanguageModel function to get the default and max values of these parameters. The first implementation adds `temperature` and `topK` parameters.
Blink component
Web Feature ID
https://github.com/web-platform-dx/web-features/issues/3530
TAG review
https://github.com/w3ctag/design-reviews/issues/1093
TAG review status
Lack of consensus
Goals for experimentation
topK and temperature parameters were excluded from the initial Prompt API launch due to interoperability concerns. Developers have expressed value in tuning parameters, for testing and use-case specific optimization. Our goal for experimentation is to explore different params or options that satisfy developer requirements and mitigate interoperability concerns.
Risks
Interoperability and Compatibility
We currently only support top-k and temperature while other models may use other sampling parameters such as top-p. This experiment is part of an exploration to get these sampling parameters right.
Gecko: No signal
WebKit: No signal
Web developers: Several partners are using non-default temperatures (e.g. low temperature for classification use cases or better adherence to structured output constraints), and we also observed how adjusting temperature can improve accuracy for specific use cases.
Other signals:
WebView application risks
Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?
None
Ongoing technical constraints
None
Debuggability
It is possible that giving DevTools more insight into the nondeterministic states of the model, e.g. random seeds, could help with debugging. See related discussion at https://github.com/explainers-by-googlers/prompt-api/issues/9.
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?
No, for the initial stages of the Prompt API, we will support Windows, Mac, Linux, and ChromeOS.
Is this feature fully tested by web-platform-tests?
No; The API shape is fully tested, but it is difficult to test the effects it has on the model’s response as it is non-deterministic.
Flag name on about://flags
None
Finch feature name
AIPromptAPIParams
Requires code in //chrome?
True
Tracking bug
Launch bug
https://launch.corp.google.com/launch/4463387
Estimated milestones
Link to entry on the Chrome Platform Status
https://chromestatus.com/feature/6325545693478912
This intent message was generated by Chrome Platform Status.
Mike Taylor
LGTM to experiment from M148 to M153 inclusive.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJcT_ZhwQ5AbE47ad2jpDHAH5wFKsko0boYjbinvHpANbqF-Jw%40mail.gmail.com.
Dr-Tarek Mohamed
Thanks for the update
Sampling parameters like temperature and topK are essential for real world use cases
Low temperature works well for classification and structured outputs
Higher values improve creativity in generative tasks
It would be useful to also explore support for topP for better control across different model types
Consistency and debuggability are important especially for production use
Having better visibility into sampling behavior inside DevTools would help
Looking forward to testing this in real browser based AI tools
Kenji Baheux
Hi everyone,
We have an important update regarding the Sampling Parameters Origin Trial for the Prompt API.
Based on recent web standards feedback concerning cross-browser interoperability, we are changing how output variety is controlled. Raw numerical scalars (like temperature and top-K) can behave inconsistently across different underlying model families and versions, leading to silent breaks in developer expectations.
To address this, we are removing access to the raw parameters and replacing them with Categorical Sampling Modes. We believe that this is largely aligned with early results of an on-going survey via the Early Preview Program:
45% of respondents shared that they’ve been actively tuning these parameters. For this group:
50% indicated that predefined presets would likely be acceptable.
28% expressed a definitive need for raw parameter access.
We recognize that removing this access is a significant shift for this group, and we want to deeply understand the specific production requirements driving this need (see the end of this email for how).
14% had no strong opinion yet.
Here are the changes happening from Chrome 151:
Removed: The params(), temperature, and topk surfaces are unavailable from web page contexts.
Note that this is only for the API exposed to the web, which means that you can still access those parameters from within Chrome extensions.
Introduced: A new samplingMode semantic enum can be passed during session creation.
The browser will now handle the heavy lifting of mapping these semantic presets to the optimal raw parameters for their specific underlying model.
The New API ShapeDuring this initial experimentation phase, we are testing a 5-tier spectrum to gather real-world usage data and see which modes developers gravitate toward most:
enum AILanguageModelSamplingMode {
"most-predictable", // For strict consistency/factual extraction
"predictable", // For highly focused outputs
"balanced", // The default state for standard prompting
"creative", // For tasks favoring variety over strict facts
"most-creative" // For maximum token diversity and brainstorming
};
dictionary AILanguageModelCreateOptions {
AILanguageModelSamplingMode samplingMode;
};
Code snippet:
const channelYourInnerSelf = await LanguageModel.create({
samplingMode: "most-creative"
});
// A high-variance prompt perfect for testing the "most-creative" setting
const creativeBrainstorm = await channelYourInnerSelf.prompt(
"Give me 3 weird, highly unusual flavor combinations for an ice cream shop that surprisingly taste good."
);
console.log(creativeBrainstorm);
To prevent ambiguity, the new samplingMode parameter is mutually exclusive with legacy raw parameters where they might still exist. Providing both a samplingMode and a raw parameter (like temperature) will result in a TypeError.
FeedbackWe welcome your feedback on these new presets as we evaluate whether to keep this 5-tier spectrum or consolidate it into fewer buckets before final standardization. Feel free to use one of these options:
A private reply (if public channels are not an option; e.g. kenji...@google.com)
For developers whose workflows are anchored in raw parameter tuning: To help us bridge the gap, we need to look at concrete data with commitment rather than hypothetical use cases. Please try these new presets and see if they can work for your production requirements. If they don’t, please share the exact historical data with us:
The specific prompt and required output variance.
The exact numerical settings (temperature/top-K) your production use case relied on.
The specific presets you tested and the exact way the output failed to achieve your required result.
Having these specific, real-world examples of where a preset cannot achieve the production outcome you need is the most effective way for us to explore workable alternatives.
Cheers, Kenji.