Thread by @Stephen_Parker

The Frost

Hey friends, we've just completed a short film, 'The Frost', every frame of which was generated with #dalle2. Thread for detail and process 🧵

technologyreview.com/2023/06/01/107…

Here are some pre-production stills I created with #dalle2 this winter, sort of 'Mountains of Madness' meets 'Color Out of Space'. They served as an inspirational starting point for the film.

@bigkickcreative (Josh Rubin) our director, took the images and put together a script filled with themes of AI, global warming, and human vs nature. For him, it wasn't really about making something with pretty pictures, but about the creation of a story that might resonate.

The bulk of my contribution then ended up not being individual image creation, but rather teaching the team to guide DALL-E through the use of 'contextual prompt wrappers'. These provided a structure the team could use for more consistent results. BTS featurette with the team:

Rather than striving for photographic accuracy, Josh wanted to prioritze the emotion of the scene. He became heavily influenced by Kurosawa's 'The Blizzard', which has a kind of oddly stilted movement to it. We took that and leaned into the weirdness of Dalle's outputs.

One of the biggest challenges was maintaining character continuity. We ended up building a narrative that suited a group of people, allowing us to introduce new characters without disrupting the story.

For recurring characters like Ulrich, the team prompted Dalle with archetypes like 'grey-haired, Wily-esque, scientist, in a white lab coat'. This got us close enough to move through shots and suspend disbelief in extended sequences.

While DALL-E played a major role, it wasn't the only tool we used. We also utilized @Adobe After Effects, @D_ID_ , and Mixamo, (big time!) and more to bring 'The Frost' to life. It's absolutely a blend of traditional and AI tools that made this project possible.

We also started the project well before Gen-1 and 2 from @runwayml, but did manage a Gen-2 Easter egg: the video that Ulrich "decodes for metadata."

A big shout-out to all the creators who contributed to this project. There were 3 main Dalle artists, an animator, an editor, and of course Josh running point on direction, writing, and production. Their creativity and dedication made 'The Frost' possible.

I also want to give special thanks to Alex Persky-Stern (our CEO) who green lit the project, Maggie Hughes (our Chief of Staff) who stood up in the annual planning session and said "let's do this!" And my friend and mentor @tyler_thompson who worked on the logo with me.

Hit me up with any questions or criticism. And thanks for checking out The Frost.