Thread by @RyanMichero

Hi! My name is Ryan, and I was a Lighting Lead in charge of Soul World Assets on #PixarSoul. My main responsibilty was to be part of the look development team and the Lighting department liason for the Soul world characters. So, SoulJoe, Soul22, Soul mystics, Terry, the Jerries..

as well as all of the crowd souls, the cat soul, etc. All of these characters were particularly difficult to figure out, as they were unlike any characters Pixar had done before. There were some precedents at Pixar (Joy in Inside Out, the clouds in Partly Cloudy), but tech has...

changed a lot at the studios since those days. There was no approved single "design" for the soul characters when I started, so we weren't figuring out how the look of the souls would be achieved, we were figuring out what they were supposed to look like on screen. #PixarSoul

And now we're in the Great Beyond, first look at the soul chraracters! #PixarSoul This movie's preproduction played out as if we were working on two completely different movies at the same time: the NYC human world movie and the soul world movie.

The soul characters are made almost entirely of volume primitives (at Pixar, saved in the f3d format), with the sole (ahem) exception of the eyes. We did experiments with volumetric eyes, but we were concerned we would never be able to get the eyes looking "right" #PixarSoul

I want to call out my brilliant co-workers in the lighting department who worked on soul world lighting. I will probably forget people, but...Don Bui and Jesse Hollander did the Great Beyond scene; You Seminar/Great Before was Sungyeon Joh, Jose Ramos, Amber Lunderville, Andy Lin

#PixarSoul Emmanuel Maniez for the Great Beyond scene with Terry and a Jerry; Jonathan Kiker and Katie Bickley for the You Seminar scene with Terry talking to the Jerrys; Brian Boyd and Josee Lajoie for the You Seminar theater scene. Magnificent, brilliant work by all of them.

#PixarSoul Josee Lajoie worked hard to make the You Seminar instructional video look like an old, worn out VHS tape. Notice the 4:3 aspect ratio of the You Seminar video, snow, tracking errors...

#PixarSoul The Hall of You was lit by @MaxBickley, Vandana Sahrawat, Emmanuel Maniez, with the mentor interludes in the You Seminar lit by Edward Chen.

One of the most difficult things about the soul characters is that they were volumes, so partially transparent. But wait, shouldn't we see teeth and tongue inside the characters' closed mouths? Yes, but we don't because we cut away what would be hidden by cheeks/skin. #PixarSoul

Molly Meyer lit a lot of the You Seminar shots just seen, like the shot with the ball of young souls. #PixarSoul

#PixarSoul Alfonso Caparini, Laura Grieve, and Jae Kim lit much of the shots in the Astral Plane.

#PixarSoul Back to the souls' mouths--I feel like this is where I spent a significant amount of my time working on the movie. So we cut out the inner mouth pieces when the mouths were closed. But, guess what? Light still leaks through the volumes and does thing like illuminate...

#PixarSoul ...the insides of the teeth on the opposite side of the light. This looks weird, and would not be a problem with normal characters because the head would completely shadow the inner mouths. We had to come up with a lot of different strategies for fixing light leaks.

#PixarSoul One quick shot in the Great Beyond for the soul of Mr. Mittens! It's the only shot of Mr. Mittens in the whole movie. Because of this, Mr. Mittens was not so heavily scrutinized by the shot cameras and we were able to use a surface-based model instead of volumes.

#PixarSoul The Mr. Mittens shot was lit and composited by Chia-chi Hu (with a little help from me), our Compositing Supervisor and look development partner in crime.

#PixarSoul Another note about the Great Beyond... There is actually more visual information if you are able to watch the film in HDR. On normal televisions the ball of souls flattens out to pure white near the edges. In HDR, you will see a wider range of values and more souls.

#PixarSoul That one off shot of Soul22 freaking out over pizza was lit by Don Bui. I used that shot as an experimental test bed for mouth lighting because it had teeth problems. Characters made a piece of animated geometry invisible to camera that shadowed the inner mouth.

#PixarSoul Let's talk about Terry and the counselors. There was not a lot of "lighting" going on there--they glow! They basically are lights. The main thing we had to worry about was that the counselors were outputting the proper control channels for compositing...

#PixarSoul And that their bodies were actually operating as lights themselves. We used mesh lights for the individual pieces that made up the line characters: the line, the right side and left side interior skin, and the particles. Each mesh light could be dialed up and down...

#PixarSoul which was most important in the scenes where Terry is lurking around NYC (coming up soon). The light cast from Terry on to the realistic environments were a essential to integrating the weird soul-world line characters in a completely different world.

#PixarSoul As a fan of A Tribe Called Quest for decades, it warms my heart to hear Check the Rhyme in the barber shop. Was that @Powerkeni's request?

#PixarSoul So the soul characters' eyes are surfaces embedded in a volume. Again, shouldn't we see them through semitransparent volumes? Yep, but it looks weird so we had to correct for it. We used a transparency map to cut out the outsides of the eyes. Here's before/after.

#PixarSoul Another essential part of the souls' look: the ethereal helmet. This is essentially a glowy halo around the soul characters' heads that adds additional color and life to the volumetric body. Here's a gif comparing with and without.

#PixarSoul I want to shout out to my colleagues I worked with in other depts getting these characters on screen: Junyi Ling, Michael Comet, @gnguyen, Markus Kranzler, @lingding85, @TieryasXu, Thomas Jordan, Laura Murphy, Hiroaki Narita, Tolga Goktekin, Patrick Coleman, & more...

#PixarSoul More lighters: Stefan Gronsky for where the souls are caught by Terry, and Miguel Zozaya Marquez for the You Seminar scene where 22 and Joe are arguing. I'm so grateful for the talent and hard work of all of the lighters that helped make these images.

I was beyond thrilled when I heard the first #PixarSoul reels where Richard Ayoade voicing the main Terry. I had no idea he was cast but I recognized his voice immediately. I've been a fan since "Garth Marenghi's Darkplace" which you need to see if you haven't.

#PixarSoul Of course I'm jealous I didn't get to light the gorgeous nightclub performance scene lit by Maria Powers. I think every lighter is. I should also mention our brilliant Directors of Photography, Matt Aspbury (Layout) and my Soul boss Ian Megibben (Lighting).

#PixarSoul It was beyond thrilling to hear the music of the score by Trent Reznor and Atticus Ross over the images of the Soul world. Would not have guessed one day I would share film credits with these guys back when I was headbanging to Pretty Hate Machine in 1989/90.

#PixarSoul I should mention the scenes of Terry in NYC earlier were lit by Jeremy Birn and Jennifer King. Jeremy also lit some of these shots on the Astral Plane, along with Jesse Hollander. Charu Clark lit the shots of LostSoul22 in the You Seminar.

#PixarSoul Tim Best and Keith Cormier lit much of the work inside Lost Soul22. Shout out to FX TD Carl Kaphan's work of the swirling lost soul dust inside 22. Lots of work.

#PixarSoul The shot of Joe putting the seed into 22's hand. A quick, simple shot that took a lot of custom work by the Characters team to get the volumes and line work of the hand looking just right. Lit by Keith Cormier.

#PixarSoul I neglected to mention earlier the tremendous work done by Jonathan Hoffman on Terry and the Jerrys. He did the entire Houdini setup for taking an animated line mesh and turning it into what you see on screen.

#PixarSoul And Chia-chi Hu's great work on setting up the compositing recipe for Terry and the Jerrys.

#PixarSoul The scene of Joe and 22 going through the earth portal was beautifully lit by Andy Lin. It was one of the first sequences lit featuring the mostly-final versions of the soul characters, so we really had to figure out how to get them working for that one.

#PixarSoul Maria Powers lit the final scene of SoulJoe and Jerry in the Great Beyond. The shot of the close-up of Joe's foot is another example of a simple shot that needed a lot of custom work (in Houdini) to get it looking good & focus enough voxel density in the right areas.

#PixarSoul Sungyeon Joh and Edward Chen did a lot of the little vignettes of the young souls seen through the credits.

#PixarSoul One of the new bits of tech we needed on this show was to take the volumetric shadowing signal and, instead of just attenuate the light, use it to saturate the base color. @levork did this work in PRMan. Here's a gif of Soul22 with and without this tech.

Julian @levork Fong already has a viral tweet thread about some of the shadow work he did:

https://twitter.com/levork/status/1342585635257741312?s=20

#PixarSoul Broken renders! In this shot, for some reason, the mesh that comes out of our shot conversion process that is used to generate volumes was not hidden like it was supposed to be, and the volumes were offset from the mesh. Why? Maybe @gnguyen can figure it out?

#PixarSoul This was a very important piece of art--a paintover by @DocterPete himself I believe on a render of a work in progress version of Soul22. This helped lead us to the finishing touches on the souls, including the ethereal helmet & @levork's shadow saturation work.

#PIxarSoul This paintover by Director of Photography Ian Megibben was instrumental in really finding the lighting language of the soul characters. The idea was that light would scatter through their volumes similar to how dispersion of light in water droplets creates a rainbow.

#PixarSoul When we lit a soul, we generally used one light. We had a warm-to-cool color ramp (warm side pointed at key light) as well as fake diffuse/specular response on top of the volumetric shading, both controlled independently to give more control over their light shaping.

#PixarSoul That warm rim between 1-2 oclock on Soul22's head was rendered in a separate pass from her regular beauty pass and added on top. The volume in the rim light pass had shadow material settings that would go completely to black, turning off her usual colored shadow.

Her usual colored *self-shadow*, I should say. Self-shadow, for a semi-transparent volume, is partially visible throughout the volume. If her main self-shadow attenuated to black in her main beauty layer, she would look dingy and dirty.

One of the many ways that lighting characters made of volumes is just *weird* and unintuitive to lighters used to lighting characters made of surfaces.

Before I abandon this thread, I also want to shout out to the other lighting leads on the team for #PixarSoul : Michael Sparber, Mitch Kopelman, Chia-chi Hu, @MaxBickley, and Mark VandeWettering, whose support and collaboration were invaluable.