1/ We used NeRF to get this shot. This entire thing was shot with a phone 🤳 No drones were used @jperldev #photogrammetry #instantNeRF #neuralrendering #artificialintelligence
2/ Here's how - walked around me in circles a few times, pointing his phone at me. We fed the video into Instant-NGP which created a NeRF out of the footage.
3/ It's kinda like a 3D model, but instead of creating mesh+textures it’s more like a point cloud that changes color depending on what angle you view it from. This creates beautiful surreal lighting effects - you can see how the light hits differently as we change camera angles.
4/ With the NeRF, you're able to set any camera angle / camera path you want - you're not limited to the camera path you captured from. Which is how we were able to simulate "drone" shots even though we never even shot from above.
5/ I think the most exciting thing happening in VFX right now is everything that's about to be possible with AI-powered video editing. While this technique does require quite a bit of technical knowledge right now (the ability to write and debug code)
6/ I do believe that within a few short years, this will become as easy as pressing a button on an app on your phone.
7/Which means soon everyone and their grandma will be able to generate a video like this, simply from walking around and taking a casual phone video. Over the next few years, AI-powered video editing is gonna empower so many people to make Hollywood-level effects with their phone
8/ @jperldev and I did a lot of experimenting with this technique - we tested over 100 clips over the last few months and we're gonna share a few more of our favorites over the next month nvlabs.github.io/instant-ngp/
• • •
Missing some Tweet in this thread? You can try to
force a refresh
