Thread by @Warvito

🎉New preprint! We used Diffusion Models (same one from #dalle2 and @StableDiffusion ) to generate 3D MRI images of the brain conditioned on several covariates and make 100k synthetic brains openly available
Paper: arxiv.org/abs/2209.07162
Dataset: tinyurl.com/32p4hu7d
1/n

with @jessdafflon @pfdacosta @PTudosiu Virginia Fernandez, Parashkev Nachev, Sebastien Ourselin
@mjorgecardoso
In a collaboration with @NVIDIA Cambridge-1 supercomputer

2/n

We adapted latent diffusion models (arxiv.org/abs/2112.10752 developed by @robrombach and @andi_blatt) to learn to generate 3D high-resolution medical images. Due to its scalability, we were to train these models on images with millions of voxels!

3/n

Our diffusion models were able to beat previous GAN-based approaches generating diverse and realistic images. They are also much easier to train, without problems of stability, convergence or mode collapse!

4/n

With our conditioning method, we can control the generation brains for age, sex, ventricular volume, and brain volume. By analysing the ventricular volume and age, we find a high correlation between the inputted values and the observed ones

5/n

We also can extrapolate to never seen conditioning values, creating massive ventricles or completely removing them!

6/n

We hope our study will help regarding one of the current limitations of medical imaging projects: Lack of availability of large datasets. For that, we are collaborating with @ProjectMONAI to make our training code and models easily accessible to the medical imaging community! 7/n

Also, our paper was accepted @dgm4miccai ! So, if you are coming to #MICCAI2022 in Singapore this year, come checkout our presentation on the 22nd of September, and let’s discuss the method and its future applications! 8/n

Check out our @Gradio application using Diffusion Models to the generated 3D brain! Using this online tool, you can generate new brains by yourself and download them!

Temporary fast version (GPU): 14365.gradio.app
@huggingface Spaces (CPU): huggingface.co/spaces/Warvito…

n/n

Thanks to @EMostaque 's open source initiative with #stableDifusion and the amazing apps created artistic community, we can also borrow some tricks when generating medical images, for example, using a fixed seed when generating images and changing the ventricular size ^^ n+1/n