This week I will present at @NVIDIAGTC about our work with @weights_biases and @Steadysun on leveraging diffusion models to predict cloud movement on satellite imagery.
Let me tell you about the problem we are trying to solve:
The idea is using a conditional diffusion pipeline to "outpaint" the future, one frame at a time. To test our approach's work, we first solve for the much "simpler" Moving MNIST dataset.
Given three previous frames, the model predicts the 4th.