Share this page!

Sharon Zhou Profile picture
Twitter logo
Sep 6 • 6 tweets • 4 min read
I really wanted to illustrate long stories with Stable Diffusion! So, hacked together this pipeline:

Long text ->
GPT-3 suggests illustration ideas ->
GPT-3 translates from English to "prompt-English" ->
Stable Diffusion outputs images

Open source 👇🏿
github.com/sharonzhou/lon…
#stablediffusion #StableDiffusionDemo #aiart #AI #midjourneyAi #midjourney #dalle #dalle2 #gpt3
So here's the gist of "promptgen". Without any code.

1. Copy-paste these good "prompt-English" examples, from this file in the repo: github.com/sharonzhou/lon…
2. On a newline, write your prompt in plain English
3. Generate prompt-English suffixes and use it in your fave model!
Can you guess which is pre-promptgen and which are using promptgen? Literally just using the prompts in the mini-video above.

Yeah, the two images from promptgen are better
It’s out!! Illustrations from this script, accompanying this short story (written by GPT3)

storiesby.ai/p/never-hire-a…
Added example files to the repo for the classic, Three Little Pigs.

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Sharon Zhou

Sharon Zhou Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @realSharonZhou

Sharon Zhou Profile picture
Jun 29
The OpenAI Minecraft paper is a great push to getting AI to work in Photoshop, Figma, or any software product — using just the keyboard & mouse, like a person would.

Steps in the paper, explained 🧵

1/
 openai.com/blog/vpt/
1. First, hire people to play Minecraft, who are OK at it. Record their screen and keyboard & mouse strokes. This costs $2k for 2k hrs of video in total.

This is your small dataset.

2/
2. Train a model on this small dataset. Let the model to look a little bit in the past and a little bit in the future in the videos. Let it predict the key & mouse strokes the person used, aligned to the video.

This is your small model.

2/
Read 11 tweets
Sharon Zhou Profile picture
Jan 15, 2021
Excited to share our #ICLR2021 paper w/ CS & math depts @Stanford 🎊

Evaluating the Disentanglement of Deep Generative Models through Manifold Topology!

w/ @ericzelikman Fred Lu @AndrewYNg Gunnar Carlsson @StefanoErmon. Acknowledging @torbjornlundh Samuel Bengmark.

Thread 🧵
Before I start: camera-ready 📸 & math-inclined R5 burn 🔥 are here
openreview.net/forum?id=djwS0…

Huge appreciation for all reviewers esp R5 in making our work better.

My goal in 🧵: Explain our work in my simplest terms to you. Don't worry if you get lost, it's admittedly dense :)
Disentanglement in your generative model means dimensions in its latent space can change a corresponding feature in its data space, e.g. adapting just 1️⃣ dim can make the output "sunnier" ☁️→🌥→⛅️→🌤→☀️ Contrast w/ this entangled mess ☁️→🌥→🌩→🌪→☀️
Read 17 tweets
Sharon Zhou Profile picture
Jan 13, 2021
Lasting collaborations can come from transient places.

One of my collaborations came out of a train ride 🚊 where I traded notes with a mathematician, en route to Stockholm. Followed by making these corgis happen.

Story 🧵👇🏿 Image
On the 4 hr 🚊ride, I talked about generative models and neural networks. He talked about fractals and Mobius transformations — and even how all this ties into making better compression socks 🧦.

Hours 1-2: Just a pen and a few loose pages.
Mobius transformations generalize affine transformations
(see cute dogs). They are found naturally in biology. Maybe... we could use these for data augmentation, without much tuning across a ton of different augmentations.

Hour 3-4: Hacking on a Python Notebook together. Image
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(