[CFP] NeurIPS 2021 Workshop on ImageNet: Past, Present, Future
SangHyuk Chun
Submission deadline: 18 September 2021, 23:59 AOE
Author notification: 15 October 2021
Workshop date: 13 December 2021
Since its release in 2009, ImageNet has played an instrumental role in the development of deep learning architectures for computer vision, enabling neural networks to greatly outperform hand-crafted visual representations. ImageNet also quickly became the go-to benchmark for model architectures and training techniques which eventually reach far beyond image classification. Today’s models are getting close to “solving” the benchmark. Models trained on ImageNet have been used as strong initialization for numerous downstream tasks. The ImageNet dataset has even been used for tasks going way beyond its initial purpose of the training of classification models. It has been leveraged and reinvented for tasks such as few-shot learning, self-supervised learning and semi-supervised learning. Interesting re-creation of the ImageNet benchmark enables the evaluation of novel challenges like robustness, bias, or concept generalization. More accurate labels have been provided. About 10 years later, ImageNet symbolizes a decade of staggering advances in computer vision, deep learning, and artificial intelligence.
We believe now is a good time to discuss what’s next: Did we solve ImageNet? If not, what is there left to do? What are the main lessons learnt thanks to this benchmark? What should the next generation of ImageNet-like benchmarks encompass? Is language supervision a promising alternative? How can we reflect on the diverse requirements for good datasets and models, such as fairness, privacy, security, generalization, scale, and efficiency?
We welcome any submission on the topics below. The list is non-exhaustive.
ImageNet as a benchmark
- How is ImageNet serving the community as a benchmark?
- Did we solve the ImageNet classification task?
- How can we spot and remedy the remaining errors?
- What are the remaining challenges not captured by ImageNet?
- How can we measure the research progress on robustness, bias, and other out-of-domain generalization issues using ImageNet?
- How to fix errors in ImageNet labels and compare results across different labels?
ImageNet as a pre-training dataset
- Is ImageNet still the best dataset for pre-training?
- What could a better pre-training dataset look like? What are good properties?
- Zero- and few-shot learning on or using ImageNet
- Unsupervised and self-supervised learning
- Semi-supervised learning
- Transfer learning
Explorative and innovative ideas even without state-of-the-art (SOTA) results
- We encourage submissions that are groundbreaking but are difficult to get published at conferences because they do not get SOTA performances.
Beyond ImageNet
- Recipes for building a large-scale dataset
- Socially responsible benchmarks and datasets: fairness, privacy, security, robustness, scalability, efficiency, and environmental friendliness
- Efficiency and quality aspects of annotations and crowdsourcing
- Going beyond classification labels for supervising visual models - e.g. language descriptions for images
- Post-ImageNet benchmarks and datasets
There are three submission tracks. All accepted papers, including the ones for the main track, are non-archival; that is, there are no formally published proceedings.
- Main track: at most 5 pages
- Extended abstract track: at most 3 pages
- Published work track
Zeynep Akata (Univ. of Tübingen)
Lucas Beyer (Google Brain)
Sanghyuk Chun (Naver AI Lab)
A. Sophia Koepke (Univ. of Tübingen)
Diane Larlus (Naver Labs Europe)
Seong Joon Oh (Naver AI Lab)
Rafael Rezende (Naver Labs Europe)
Sangdoo Yun (Naver AI Lab)
Xiaohua Zhai (Google Brain)