| | Prof. Hongyang Li
University of Hong Kong
Hongyang Li is an Assistant Professor at The University of Hong Kong and has led OpenDriveLab since 2021. His research focus is on autonomous driving and embodied AI. He led the end-to-end autonomous driving project, UniAD and won the IEEE CVPR 2023 Best Paper Award. He created the first large-scale real robot ecosystem, Agibot World, that systematically investigated the scaling law principles for robotic manipulation. He served as Area Chair for CVPR, NeurIPS, ICLR, ICCV, ICML, RSS. |
|
|
|
|
Abstract
A generalist robot should perform effectively across various environments. However, most existing approaches heavily rely on scaling action-annotated data to enhance their capabilities. Consequently, they are often limited to a single physical specification and struggle to learn transferable knowledge across different embodiments and environments. To confront these limitations, we propose UniVLA, a new framework for learning cross-embodiment vision-language-action (VLA) policies. Our key innovation is to derive task-centric action representations from videos with a latent action model. This enables us to exploit extensive data across a wide spectrum of embodiments and perspectives.
To mitigate the effect of task-irrelevant dynamics, we incorporate language instructions and establish a latent action model within the DINO feature space. Learned from internet-scale videos, the generalist policy can be deployed to various robots through efficient latent action decoding. We obtain state-of-the-art results across multiple manipulation and navigation benchmarks, as well as real-robot deployments. UniVLA achieves superior performance over OpenVLA with less than 1/20 of pretraining compute and 1/10 of downstream data. Continuous performance improvements are observed as heterogeneous data, even including human videos, are incorporated into the training pipeline. The results underscore UniVLA's potential to facilitate scalable and efficient robot policy learning. |
|
This event is free and open to the public.
The Seminar Series in Modern Artificial Intelligence is held at NYU Tandon School of Engineering and is hosted by the Department of Electrical and Computer Engineering. Organized by Professor Anna Choromanska, the series aims to bring together faculty and students to discuss the most important research trends in the world of AI. The speakers include world-renowned experts whose research is making an immense impact on the development of new machine learning techniques and technologies and helping to build a better, smarter, more-connected world. |
|
| To make sure you keep getting these emails, please add Modern AI Seminar Series Prof. Hongyang Li to your address book or whitelist us. If you are not a member of the NYU Tandon community and wish to be removed from all our mailing lists, unsubscribe.
Can't see this email? View it as a webpage here. |
|
|
|
|
|
|
|
|
|
|
|
