[Release LIT v1.0] Improved layouts, APIs, demos, and more
Ian Tenney
This is a major release, including enhanced layout options, built-in metrics for question-answering and multi-label classification use cases, new demos for question-answering and generative image models, and many API updates to improve the developer experience and ergonomics. Check the release notes for complete details.
This is a collaborative work made possible thanks to many Googlers - Ryan Mullins, Ian Tenney, Nada Hussein, Minsuk Kahng, Mahima Pushkarna, Crystal Qian, James Wexler, Bin Du, Cibi Arjun, Oscar Wahltinez - plus many more who helped build LIT since the project began in 2020!
Enhanced Layouts
LIT now supports 4 different layout configurations based on the properties defined in a LitCanonicalLayout instance:
Single-panel: Define only the upper= parameter.
Two-panel, upper/lower: Define the upper= and lower= parameters.
Two-panel, left/right: Define the left= and upper= parameters; the upper= section will be shown on the right.
Three-panel: Define the left=, upper=, and lower= parameters; the upper= and lower= sections will be shown on the right.
LIT provides 3 layouts showcasing these configurations:
Simple: A minimalist layout with the examples on top (either individually (selected by default) or in a table) and predictions on the bottom;
Default: The original LIT layout with a single group of modules on top for exploring and selecting data, and a collection of tabs supporting different analytical tasks on the bottom; and
Experimental: A three-panel layout that puts exploratory data visualizations at full-page height on the left, tools for inspecting and manipulating examples and their associated predictions in the upper right, and a collection of tabs supporting different analytical tasks in the lower right. Your feedback on this layout is appreciated.
Enhanced Metrics
Multi-label classification metrics compute exact match, precision, recall, and F1 score for labeled examples.
Exact match metrics support seq2seq generation tasks, such as question-answering.
Metrics have also been promoted in the LitMetadata object, making it easier to see which metrics have been loaded in your LIT instance.
New Demos, brought to you by Google Summer of Code
Multilingual question-answering with TyDi QA.
TyDi QA is a benchmark dataset focused on information-seeking across typologically diverse languages
This demo shows how to evaluate an off-the-shelf FLAX model trained on this dataset, with a focus on exact-match metrics.
Generating images with DALL-E Mini.
DALL-E Mini is a small-scale, open source text-to-image generation model inspired by Open AI’s DALL-E.
This demo provides a small set of canned prompts and a unified model wrapper to enable image generation.
Use the Datapoint Editor to add additional prompts and explore the results.
Note that these demos are provided as isolated units, to avoid potential dependency conflicts with other examples.
Library and API updates
Many dependencies have been updated for this release, notably upgrading to Python 3.10, TypeScript 5.0, and HuggingFace Transformers 4.27.
Consolidating the Dataset.examples representation to a Mapping[str, Any] structure.
Elements of IndexedDataset.examples will have special _id and _meta fields, replacing the functionality previously provided by IndexedExample.
Removing all *_with_metadata() APIs from the Python library to simplify the API surface.
- Improved the performance and hit-rate of LIT’s built-in CachingModelWrapper.