[Intent to Experiment] Prompt API Sampling Parameters
Deepti Bogadi
Contact emails
mem...@google.com, dbo...@google.com, m...@google.com
Explainer
https://github.com/webmachinelearning/prompt-api?tab=readme-ov-file#sampling-parameters
Specification
https://webmachinelearning.github.io/prompt-api/
Summary
Adds sampling parameters to the Prompt API. These control how tokens are sampled from the model, giving developers control over the "creativeness" or "randomness" of the output. Additionally, it adds attributes to the LanguageModel instance to read the set values, as well as a static LanguageModel function to get the default and max values of these parameters. The first implementation adds `temperature` and `topK` parameters.
Blink component
Web Feature ID
https://github.com/web-platform-dx/web-features/issues/3530
TAG review
https://github.com/w3ctag/design-reviews/issues/1093
TAG review status
Lack of consensus
Goals for experimentation
topK and temperature parameters were excluded from the initial Prompt API launch due to interoperability concerns. Developers have expressed value in tuning parameters, for testing and use-case specific optimization. Our goal for experimentation is to explore different params or options that satisfy developer requirements and mitigate interoperability concerns.
Risks
Interoperability and Compatibility
We currently only support top-k and temperature while other models may use other sampling parameters such as top-p. This experiment is part of an exploration to get these sampling parameters right.
Gecko: No signal
WebKit: No signal
Web developers: Several partners are using non-default temperatures (e.g. low temperature for classification use cases or better adherence to structured output constraints), and we also observed how adjusting temperature can improve accuracy for specific use cases.
Other signals:
WebView application risks
Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?
None
Ongoing technical constraints
None
Debuggability
It is possible that giving DevTools more insight into the nondeterministic states of the model, e.g. random seeds, could help with debugging. See related discussion at https://github.com/explainers-by-googlers/prompt-api/issues/9.
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?
No, for the initial stages of the Prompt API, we will support Windows, Mac, Linux, and ChromeOS.
Is this feature fully tested by web-platform-tests?
No; The API shape is fully tested, but it is difficult to test the effects it has on the model’s response as it is non-deterministic.
Flag name on about://flags
None
Finch feature name
AIPromptAPIParams
Requires code in //chrome?
True
Tracking bug
Launch bug
https://launch.corp.google.com/launch/4463387
Estimated milestones
Link to entry on the Chrome Platform Status
https://chromestatus.com/feature/6325545693478912
This intent message was generated by Chrome Platform Status.