Nvidia enhances the development of robotics with new simulation tools and AI | Generative ai use cases in manufacturing | Microsoft generative ai tools free | Ai in healthcare examples | Turtles AI
NVIDIA has unveiled a series of new simulation and AI tools dedicated to the development of robots, including humanoids. These tools, including Isaac Lab, the Cosmos tokenizer and the GR00T project, aim to simplify the learning and improvement of robot capabilities in various application domains. In collaboration with Hugging Face, NVIDIA has also expanded the possibilities for open source research in robotics.
Key points:
- Isaac Lab: an open source framework for robot learning and simulation, with applications for humanoids, collaborative robots, and quadrupeds.
- Project GR00T: an initiative to develop basic models, libraries and workflows for humanoid robots, focusing on perception, movement and interaction.
- Cosmos Tokenizer: a tool for creating world models with efficient video processing and advanced compression.
- Hugging Face-NVIDIA collaboration: a partnership that aims to improve open source robotics by integrating NVIDIA technologies with Hugging Face’s machine learning solutions.
NVIDIA is stepping up its commitment to robotics with the introduction of new technology tools that promise to accelerate the development of humanoid robots and extend AI capabilities in the field. During the Conference for Robotic Learning in Munich, the company announced a number of new developments ranging from advanced simulation frameworks to innovative solutions for video data management. Key players in this new phase include Isaac Lab, an open source robotic learning framework, and the GR00T project, an initiative that introduces a series of workflows to improve robots’ capabilities in perception, locomotion, control, and interaction. Isaac Lab, built on NVIDIA’s Omniverse platform, represents a complete robot training environment with customizable movement policies that can simulate a wide range of situations, including the movements of humanoids and quadruped robots. This tool is already being used by a number of research organizations and companies, including Boston Dynamics, Agility Robotics, and Unitree Robotics, which are exploiting its potential to develop and refine their own robotic solutions. In parallel, the GR00T project aims to simplify the process of creating and using basic models for humanoid robots. The six planned workflows include the generation of 3D environments using AI, creation of trajectories and motions for manipulation, robot body control, locomotion, and multimodal perception, all of which are key to making robots more autonomous and versatile. To support these developments, NVIDIA also introduced the Cosmos tokenizer, an advanced visual data management tool. This tokenizer, available on both GitHub and Hugging Face, enables the creation of world models with significantly higher processing speed than previous solutions, while reducing temporal and spatial image distortion. This innovation is particularly useful when it comes to training robots to understand and interact with their environment through images and videos. The tokenizer is already being used by companies such as XPENG Robotics, 1X Technologies and Hillbot to improve data quality and optimize the training process. Rounding out this technology landscape, NVIDIA also unveiled NeMo Curator, a video data curation platform that simplifies the orchestration of large-scale processing pipelines by supporting the use of more than 100 petabytes of data efficiently through its linear scaling capability on multi-node and multi-GPU systems. In a broader context, NVIDIA announced a collaboration with Hugging Face to accelerate open source research in robotics by integrating Isaac Lab simulation technologies with Hugging Face’s machine learning solutions, such as the LeRobot platform. This agreement will allow researchers to work on more ambitious projects, such as improving the ability of robots to learn from data, either through demonstration or trial and error. The collaborative workflow, using Isaac Sim and data generated by GR00T, will be able to speed up robot training in simulated scenarios before being applied in the real world, using devices such as NVIDIA Jetson for real-time inference. Finally, at the same conference, NVIDIA presented a series of research papers exploring important topics such as integrating visual language models to improve environmental understanding and execution of complex tasks, marking a further step forward in the innovation of robotic technologies.
The innovations announced by Nvidia open new perspectives for humanoid robotics and for the applied to the physical world, accelerating progress in creating robots capable of interacting independently with their environment.
