Thread by @thebeauanthony

#TheLastOfUsPartII’s Breathing System Thread Part 1

One gift of working at Naughty Dog is that the sky is limit. With artists and programmers collaborating closely, magic can happen.

The Breathing System is an example of that magic.

1.1/25

There are two fundamental concepts to understand with this system: Murmuration and Heart Rate.

1.2/25

Murmuration is a looping sound on a character. It's the lowest priority sound possible, so any sound that triggers (from melee, anim, script or code) will stomp it. After the oneoff plays out, the loop will return where it left off, creating a seamlessness in the breath.

1.3/25

It's dubbed Murmuration because it's 'the sound under the sounds'. For Ellie & Abby it's immersive breathing, for Clickers it's clicking, shrieking or frenzy-ing, and for dogs it's lick-lipping/snorting/etc.

It's anything we need it to be, and that's the power behind it.

1.4/25

Heart Rate is a value expressed as a number: 0.0 to 1.0.

It is an encapsulation of what a character's Heart Rate would be in given context, with 1.0 being the most exerted and 0.0 being the most relaxed.

1.5/25

This 'heart beat' exists for every Player character, Buddy character, Infected, Dog, and Human NPC in the game.

1.6/25

To measure the Heart Rate, there are states that I assign numerical values to called Heart Rate States. These states can be based on AI awareness (like Unaware), enemy attack type (like Bloater Charge), or anything needing a looping solution.

1.7/25

Programming was able to split out as many states as I needed to get the job done.

For example, Clickers have states that include Frenzy and Unaware, which have a target Heart Rate of 1.0 and 0.7 respectively.

1.8/25

This gives me the ability to be able to play a different looping sound based on all sorts of game context.

1.9/25

In this thread, I want to exclusively talk about Ellie, one of the player characters.

Ellie has Murmuration enabled, so she's always playing a sound loop.

She has several Heart Rate States that are split by tension and whether she's sprinting or not.

1.10/25

As an example, if Ellie is sprinting in ambient tension, her Heart Rate state is 'ambient-high'. When you let go of sprint, (NOT sprinting) it becomes 'ambient-low'.

1.11/25

With this, I'm able to determine how long it takes the Heart Rate number to go up/down from different states, letting me choose how long I want to hear Ellie's exhausted breaths peter out or how long it takes Ellie to sound worn-out after sprinting for a period of time.

1.12/25

The possibilities become endless. I am able to do things like:

1. Play different sounds based on how long you've been sprinting, in any tension.
2. Play various stages of exhausted breathing after you've been sprinting for certain amount of time.

1.13/25

If Ellie starts sprinting when the enemy is aware of her, you'll hear Ellie's sprint breaths become increasingly feverish and fatigued.

These are her Stages.

Every tension has 3 stages of assets, varying in intensity.

1.14/25

When you let go of sprint, you'll hear the stages of her exhausted breath variations as it blends into her baseline breaths.

This is one instance of Heart Rate that happens seamlessly throughout systemic gameplay.

1.15/25

In total, there are hundreds of breathing assets (single sound files of an exhale into an inhale) transitioning based on variables such as the tension, player health, enemy awareness, Heart Rate value, Vertigo scale (For Abby) and others.

1.16/25

The system is designed to ensure the assets never interrupt each other and always play out before game logic decides to pick a different type of breathing.

This implementation feels natural and makes you truly feel like you're controlling a living, breathing entity.

1.17/25

Heart Rate is even scripted based on story moments. I'm able to spike a character's Heart Rate to hear an audible 'come-down' in the breathing from a heightened state, like a jump scare.

1.18/25

In this video, I scripted a spike in Ellie's Heart Rate when she progresses through this squeeze-through to showcase her fear.

This blends beautifully with the gameplay dialog which triggers after Ellie grabs the pipe!

1.19/25

We also override Murmuration with a custom loop for various story beats, gameplay moments, or cinematic moments.

This puts you in their shoes and helps you empathize with their story by communicating how they're feeling.

Breath can be an excellent conveyor of emotion!

1.20/25

One note-worthy feature is the Open/Closed Mouth Stealth Breathing, Mike Hourihan's (@m1keadelic) brilliant idea! If the enemy is within a certain distance and in the player's view frustum, Ellie will transition to closed-mouth breathing.

1.21/25

This creates a tension that a player might not pick up on, but affects them nonetheless. Sound has an interesting tendency to bypass the brain and head straight to the heart.

1.22/25

Abby's vertigo was also integrated into this system. The vertigo game data that affects the camera talks to the audio middleware to play different breathing if Abby is looking over a ledge. The volume & intensity is affected by how much or little you're looking over!

1.23/25

Injured breaths are integrated into the system as well.

A flag is set when there's an arrow embedded in the character or if player's health is low. That enables my scripts to potentially play an injured variant integrated into the the base systemic breathing variants.

1.24/25

All of the breathing for Player and Buddy characters were recorded with this system in mind, giving us the most optimal source to be integrated into the system.

Big thanks to all the main cast for entertaining our needs!

1.25/25

Breathing Thread #2 can be found here:

https://twitter.com/thebeauanthony/status/1284544453709934592