Emad Profile picture
Aug 21, 2022 13 tweets 6 min read Read on X
Some thoughts on #StableDiffusion vs #DALLE2 vs #midjourney

They are actually all different (complementary?).

First this whole field was catalysed by @OpenAI releasing CLIP publicly last year (!) which @RiversHaveWings, @advadnoun and others built on openai.com/blog/clip/
This allowed image generation to be guided, the output of which was surprising and led to a burst of developer and artist creativity from @ClaireSilver12 and many others.

Proud to say we supported & funded most of the open tools in this space.

It is now good enough, fast enough
So #dalle2 is a model and a service.

It is focused on a certain usage subset that will broaden.

Inpainting is it’s best feature but by default it is random and best used for ideation and more corporate usage, hence it’s clear training on licensed stock images
I am sure lots more features are in the pipeline and more efficient, even higher quality versions are in the pipeline.

Expect the cost to drop 10x by the new year and we should see API access.

I do not expect the model to be open sourced given the training data
If you want to see how it works dig into the research paper here arxiv.org/abs/2204.06125

And you can check the open source variant of those research concepts we supported here: github.com/lucidrains/DAL…

@OpenAI is focused on general intelligence, not product.

And that is fine.
Now #midjourney.

David Holz is a visionary technologist focused on understanding how humans interact with computers.

MidJourney is not a VC-backed product company but a research lab seeing how people interact and can be improved by this new tech, see his latest interviews
MidJourney has a very distinctive painterly style on purpose.

It uses the same raw open source model as most other services (for now! That will change soon) but a huge amount of work has gone into consistency and coherence.

Output is random tilted but there is some control
The front end code is not open, nor is anything else about it, but David has open sourced millions of lines of code i his career.

And that is fine. Not everything needs to be open source and it is a great service that will evolve in perhaps surprising ways 🙃
Now #stablediffusion is a model built through a series of collaborations that will be released very 🔜

It is a file that is part of foundational infrastructure for knowledge of all types of image, from art to product to whatever you can imagine

It is a general foundation model
Now there are services that will be built around it as it is being released open source as the first of our benchmark/foundational models.

We will shortly announce our #DreamStudio prosumer service but our focus is on our API to drive down costs of accessing this & future models
So that a billion people can communicate better.

These models will need to reflect every culture and work with creators and we are doing a lot of work in this area with the brightest and most big names in the space.

As a general model outputs are wider, but what you have seen
is raw output from our beta model tests, no pre or post processing.

It is much better with these and we have focused on fine grained control.

But

As an open source model anyone can use it. Code is already available as is dataset.

So everyone will improve and build on it
Or take elements of it & make even better things

Which is fantastic.

More tools more choice but ultimately a new way of communicating for all

Brand new market and segment, nobody is competing as going from millions to billions using this

Look forward to collaborating with all

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Emad

Emad Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @EMostaque

Apr 20, 2023
This is our business model for folk that keep asking:

Stable models are benchmark open source of every modality based on open data.

We'll have sectoral/commercially licensed ones via our partners eg AWS to your data.

We also build custom models for largest cos & govts
We will be the largest supporter of open source innovation via individuals, collaboratives and academia.

That is the Linux to our Red Hat as it were.

The Stable series of models will be the building blocks for turning the private data of the world into intelligence to help
This is a good and solid infrastructure play similar to ASML and the best researchers should join and collaborate with us.

We have loads of announcements to come but by bringing stability to a chaotically growing technology we will make it easier & safer for folk to use
Read 6 tweets
Apr 19, 2023
Proud of team after dealing with melted GPUs and more starting to release these (currently very alpha) models

Loads of details, inner workings & more coming soon now that this is out, should view this as a continuous process now vs a one and done, loads of complexity in LLMs
We have been honing down on our exact position in the ecosystem and it is this.

We will be the biggest supporter of the open ecosystem, academic, independent and otherwise.

The stable series of models is the equivalent of Red Hat, done deliberatively & fully open & auditable
We will make these available across every modality, sector and nationality to provide the foundation to activate humanity's potential, building blocks you can take to your private data and own, on device or on prem or in the cloud via new services such as @awscloud Bedrock
Read 4 tweets
Mar 29, 2023
The argument private companies should hyperscale large models they themselves say could flip our society & kill us all because otherwise China or Russia will get there first is specious

If proponents actually believe that then large trainings should be nationalized by military
Fear of the other is no reason to do what is effectively gain of function research into models that exhibit crazy emergent properties and we have basically zero idea how to align.

Alignment is orthogonal to freedom as you must take the liberty of a entity more capable than you
To ensure it does as you say.

However, large emergent models can be very dangerous without achieving AGI or sentience.

I don't think a pause will do much but signed in the spirit of the letter that transparency & coordination & better governance is 💯 needed
Read 6 tweets
Mar 11, 2023
To all impacted by #svb it will be ok.

The best analysis on this is (as usual) by Matt Levine at Bloomberg and I would agree with the below as the base case.

There will still be plenty of knock on effects but this is not the big systemic issue.. yet.

bloomberg.com/opinion/articl…
The bulk of the assets are not complex and the market is functioning well so expect a rapid resolution versus prior bank failures which still had depositors pretty much made whole.

The book will likely be bought by just a few entities also speeding things up absent takeover.
The purchase of long dated bonds was a stupid trade causing a mismatch, but their are plenty of folk who can hold them to maturity, including by basically putting them to the Fed (which svb couldn’t given the mess they got in).

So plenty of buyers and nothing to complex in there
Read 4 tweets
Aug 22, 2022
Delighted to announce the public open source release of #StableDiffusion!

Please see our release post and retweet! stability.ai/blog/stable-di…

Proud of everyone involved in releasing this tech that is the first of a series of models to activate the creative potential of humanity
We are also delighted to announce the public beta release of #DreamStudio, our reference implementation to allow anyone to create anything.

The public API will be available here shortly to extend your creativity and enable new experiences.

beta.dreamstudio.ai
We have more to come and expect an explosion of originality in UI/UX and new ways to create both from our efforts and those of the community, which everyone should join at discord.gg/stablediffusion !

Developers, creatives and those inspired by this, all are welcome to come & dream
Read 11 tweets
Mar 12, 2020
Hi all, I am back and I know a lot of you are scared.
This is a thread on why this will end up ok.
I say this having warned what would happen/get bad at the end of January saying it would go exponential over the period it did ()
1. As humans our greatest ability is to come together to form something @waitbutwhy calls the "Human Colossus".
The Colossus can split the atom or reach the stars.
Little versions of it can also do big things, some good, some bad.
2. We are tied together by our common stories, from family to sports team to nations.
Even money is just a common story that allows us to store and exchange value at scale, helping connect between and across communities
Read 18 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(