Thread by @mattdeitke

Introducing Objaverse, a massive open dataset of text-paired 3D objects!

Nearly 1 million annotated 3D objects to pave the way to build incredible large-scale 3D generative models: 🧵👇

🤗 Hugging Face: huggingface.co/datasets/allen…
📝ArXiv: arxiv.org/abs/2212.08051

#CVPR2023

In the past, ShapeNet has enabled remarkable progress and benchmarking across 3D computer vision!

But, it lacks visual diversity, realism…

and scale! 🪨

Objaverse is more than an order of magnitude larger and has ~400x more categories! 📈

Objaverse is incredibly diverse and includes a vast amount of visual styles!

✅ Photogrammetry Scanned Objects
✅ 3D Modeled Objects
✅ Interiors
✅ Exteriors
✅ Characters
✅ High-Poly
✅ Low-Poly
✅ Cartoon

And More!

Many objects are also animated! 🦜

We began to show the potential of Objaverse by generating 3D objects! 👜👞🍇🎃

Here we use the amazing work of GET3D from @JunGao33210520 et al. to generate textured meshes!

We’re also thrilled by the potential for Objaverse to impact robotics! 🤖

We introduce a task where the agent’s goal is to navigate to objects as described by language.

🏠 Here’s a house we set up purely with objects in Objaverse:

Objaverse is hosted on 🤗 @huggingface!

We’re thrilled to have worked with their incredible team to make it easily downloadable! 🚀

Individual objects are available under a Creative Commons license.

🤗 Dataset: huggingface.co/datasets/allen…

Rendering images of the 3D objects is very fast with Python in @Blender!

All objects are standardized GLB files.

🚀 GitHub Rendering Scripts: github.com/allenai/objave…

We’ve also worked with @algolia to build a powerful real-time filterable search engine across all objects in Objaverse!

🔍 Search Engine: objaverse.allenai.org/explore/

We’re thrilled to see what you build! 🤩

Just this week, @ruoshi_liu et al. released Zero-1-to-3, incredible work that combines a generative image model and Objaverse, for learning a shape prior, to render novel viewpoints of an object!

https://twitter.com/_akhaliq/status/1637989614349606913

Text2Tex is an exciting paper using Objaverse that also came out just this week by @davech2y et al. for texturing 3D meshes from a text description!

https://twitter.com/_akhaliq/status/1638338037636538368

And even earlier this month, Anchit Gupta, @XiongWenhan, @EasonNie, Ian Jones, Barlas Oğuz at @MetaAI used Objaverse in work on 3DGen for triplane textured mesh generation!

https://twitter.com/_akhaliq/status/1634008015350235136

Objaverse will appear at #CVPR2023!

This was a very enjoyable collaboration between @allen_ai and @uwcse!

With Dustin Schwenk, Jordi Salvador, @LucaWeihs, Oscar Michel, Eli VanderBilt, @lschmidt3, @ehsanik @anikembhavi, Ali Farhadi

Some exciting updates soon 👀