NVIDIA introduces FlexiCubes for AI 3D meshes | | | | Turtles AI
NVIDIA introduces FlexiCubes for AI 3D meshes
DukeRem13 August 2023
A new #technique called #FlexiCubes from #Nvidia #AI #Lab enables high-quality optimization of #3D #surface #meshes. Key #advantages are differentiability for #gradient-based fitting and #flexibility to locally adapt vertices. #Experiments show benefits for #photogrammetry, #generative #modeling, and #physics #simulation.
Today we report on a new development in computer graphics research from the Nvidia AI Lab in Toronto. The lab has introduced a novel technique called FlexiCubes for optimizing 3D surface meshes in applications like photogrammetry, generative modeling, and physics simulation. The key insight is representing the mesh as an isosurface extracted from an underlying volumetric field. This allows differentiating through the mesh extraction, so gradient-based optimization can be used to fit the surface to data like images or physics simulations. FlexiCubes incorporates additional flexibility compared to classic techniques like Marching Cubes, letting vertices adjust locally to capture features in the optimized shape. Experiments validate advantages over existing methods. Photogrammetry results are improved by using FlexiCubes in a neural inverse rendering pipeline. It generates higher quality geometry from images than the previous state-of-the-art differentiable mesh extraction technique. FlexiCubes also enables end-to-end generative modeling of 3D shapes from 2D image supervision with dramatically increased mesh quality. Preliminary physics-based reconstruction shows promise for jointly optimizing shapes, materials, and dynamics from video. The FlexiCubes paper will be presented at SIGGRAPH 2023. The authors believe this approach will enable high quality optimization-based 3D reconstruction and modeling in a range of applications.
Highlights:
- - FlexiCubes represents meshes as isosurfaces, enabling gradient optimization.
- - Additional flexibility lets vertices adjust locally to fit features.
- - Experiments validate quality improvements over existing methods.
- - Benefits shown for photogrammetry, generative modeling, physics.