Will scale pay dividends in diffusion models? Here is Alibaba's Composer with 5B parameters. Dall-e 2 has 3.5B parameters, and StableDiffusion has 890m parameters.
https://twitter.com/_akhaliq/status/1629305281392832513
Browsing through the examples, there are only a few things that #stablediffusion with #controlnet cannot do. There aren't any Color palette and intensity ControlNet networks. But certainly this can be trained. 
The generality of modular networks appears to trump scale. medium.com/intuitionmachi…
It is still too early to tell if Composer reveals new emergent phenomena. But it is indeed exciting that there's a 5G parameter image AI that will soon be available publicly to test.
But if larger diffusion models are not significantly superior to smaller ones. Then we must ask if we have reached the limits of a specific kind of architecture and need to invest in a new kind. What does Composer not do differently enough from StableDiffusion?
This is an absolutely fascinating development because we can now contrast a large model like Composer that is trained to compose multiple input sources and compare them with #ControlNet where you can manually achieve the same thing through the right sequencing!
Absent access to Composer, I'm guessing here that a larger network affords the parallel satisfaction of multiple constraints. This is in contrast with multiple #controlnet that do well via sequential application.
The question here is in the 5x larger network worth the parallel constraint satisfaction? What new emergent capabilities does this make possible that is absent in a more modular network like #controlnet? We will discover this quite soon!
I expect the next advance to involve the replacement of the UNet convolution network with a ViT transformer network. There are semantics not captured in UNet that is essential in image composition. I'm just waiting for a firm to foot the training bill!
This question of scale versus modularity is a critical question to answer. This is because it will determine if AI monopolies can build their defensive moats around their cathedrals or whether a cottage industry of bespoke solutions will generate markets of diversity.
@threadreaderapp unroll
• • •
Missing some Tweet in this thread? You can try to
force a refresh
