Thread by @marknca

here we go! Monday Night Live with Peter DeSantis is about to kick off at @awscloud #reinvent 2022!

a 🧵👇 (/cc @AWSEvents)

@awscloud @AWSEvents I love how much fun Peter has with this keynote!

#reinvent

Peter reminds everyone that this keynote is all about "how" @awscloud does things. lots of behind the scenes info in this one..

#reinvent

big focus on performance tonight.

Peter reminds us it's not all about what you can see

"Great performance is the result of innovation from the ground up and investing over time"

#reinvent

starting with a dive into @awscloud Nitro...

more on Nitro at aws.amazon.com/ec2/nitro/

#reinvent

here’s a great talk from Abby Fuller on the technical details of AWS Nitro:

#reinvent

Peter announces a new version of Nitro (v5), with big gains behind the scenes. continued innovation behind the scenes from @awscloud here

#reinvent

NEW INSTANCE TYPE:

C7gn

uses the new Nitro v5 chip under the hood. lots of benefits. tip of the iceberg here

#reinvent

NEW CPU:

Graviton3E

custom tuned Graviton3 silicon. lots of benefits for the targeted use cases

#reinvent

NEW INSTANCE TYPE (coming soon):

HPC7g

…combines the Graviton3E + Nitro v5 for some big boosts for HPC applications

#reinvent

not sure I’m digging the graphic style for these slides...

#reinvent

typography with Amazon Ember is on point though. nice and clean. super clear. great typeface

#reinvent

back to the tech, here’s a great summary of the new instance types just announce from @jeffbarr: aws.amazon.com/blogs/aws/new-…

#reinvent

btw, more on the Graviton processor at aws.amazon.com/ec2/graviton/

#reinvent

Peter has moved on to how the @awscloud Global Infrastructure is designed and built

specifically talking about SRD (scalable reliable datagram)

more on that at: aws.amazon.com/blogs/hpc/in-t…

#reinvent

more on the @awscloud global infrastructure at aws.amazon.com/about-aws/glob…

#reinvent

Peter is giving a great high level overview of TCP as a protocol

tl:dr it works great, but focuses on “getting there” vs. performance

at @awscloud scale, they need something more efficient and focused on hyper scale issues

#reinvent

key benefits of SRD on the @awscloud network:

- multi-pathing
- retries in microseconds
- runs on nitro (dedicated hardware)

#reinvent

remember, @awscloud runs at a completely different scale. very few companies run at this size. it’s a completely different set of problems, requiring completely different solutions

#reinvent

SRD is helping deliver better performance for Amazon EBS, especially for “tail latency” …average latency doesn’t matter if you just need the data!

#reinvent

teal line is SRD performance. big improvement overall, but massive in the edge cases

…btw, all new EBS io2 volumes will get this performance boost in the new year (stay tuned, they are just saying “coming soon")

#reinvent

NEW LAUNCH:

ENA Express, brings SRD to ENA…available today!

more details on ENA (basic, not express): docs.aws.amazon.com/AWSEC2/latest/…

#reinvent

here’s the SRD boost for ENA with ENA Express (needs to be enabled on your ENAs)

#reinvent

lots of improvements backed by the move to SRD in the @awscloud network

#reinvent

“A veritable hoedown of machine learning”, Peter DeSantis

🤦🤦🤦

oh Peter….

#reinvent

this graphic redeems part of the hoedown comment…

would have been better if he drove it out on stage 😉

#reinvent

gist of this section: 🧠 machine learning models are getting MASSIVE

they need more performance (processing, storage, network, etc.) in order to train properly. @awscloud innovations in performance engineering like SRD are helping drive these advances

#reinvent

here are the details on the @awscloud Trn1 instance type that Peter referenced: aws.amazon.com/ec2/instance-t…

#reinvent

while they make the point, graphs without units and numbers are tough for me to look at

that said, numbers and more specifics would actual make these slides weaker and muddle the point Peter is making

#reinvent

“Just as a refresher. Stochastic rounding is probabilistic rounding”, Peter

#reinvent

why is that important 👆? it helps train the more faster with less loss. @awscloud accomplishes this with hardware acceleration in the Trainium silicon

more on that chip at aws.amazon.com/machine-learni…

#reinvent

also, here’s a mathematical primer on stochastic rounding: nhigham.com/2020/07/07/wha…

you know, because MATH! 🤣

#reinvent

the tl:dr of this section is that @awscloud has been focusing on a lot of deeply interesting (to me at least!) performance engineering problems so that we can build & train ml 🧠 models faster and more accurately

#reinvent

how else would we get fun/weird avatar pics in different styles?

🤣🤣🤣

#reinvent

Peter is currently giving a great explanation of how to scale out a workload across multiple processors. I can’t explain it in 280 character chunks 🤦

basically: many hands make light work?

#reinvent

you’re going to have to watch this segment for yourself. it starts at 35m into this keynote with this slide 👇

#reinvent

Peter brings this back home using a Rings of Power reference. btw, the show is very much worth watching

amazon.com/Lord-Rings-Pow…

#reinvent

lots of work to average out the results, but the “ring of rings” works much faster than “one ring”

…as long as you don’t give 9 to the race of man, 3 to the Elves, 7 for the Dwarves, and one to Sauron 🤦

#reinvent

new instance type coming soon to help speed things up even more:

Trn1n
- faster networking
- faster training

#reinvent

Jock Clear up now talking about the technical complexity of F1 racing

more details on @awscloud + F1 at aws.amazon.com/sports/f1/

#reinvent

three key aspects of making zoom-y cars zoom faster...

(somewhere @mosescj58 just felt a chill 🤣)

#reinvent

with @awscloud, Jock and the team built a virtual ground speed sensor

it’s an AI/ML tool that provides more accurate data faster…and more importantly with less weight

…would love to see more technical details on that one

#reinvent

btw, @QuinnyPig is also live tweeting this keynote. honestly, probably with a lot more—and funnier—jokes

check out his thread at

https://twitter.com/QuinnyPig/status/1597430775330537472?s=20&t=reKJgcbkR9LjU2gLMZeRXg

#reinvent

Jock continues to lay out the challenges that F1 teams face outside of the race and the onslaught of challenges within a race

lots of engineering challenges and trade offs…

#reinvent

I’m not a car person (see the previous “zoom-y” comment), but I do love engineering challenges. lots of crazy ones here

Jock is constantly calling out the sheer number of decisions the team needs to make. they need data & analysis to make those decisions

#reinvent

they are launching an app with a bunch of race/interview content

…was hoping for an engineering data app like the real time MLB analysis: aws.amazon.com/blogs/machine-…

..it’s not that, but I’m sure F1 fans will enjoy it

#reinvent

Peter is back up now. new topic: #serverless computing

#reinvent

“Behind me you can see a youthful @Werner announcing @awscloud Lambda”, Peter

😬🤣

#reinvent

Lambda has grown a LOT since it’s launch in 2014

thanks again @timallenwagner for setting us on this path!

#reinvent

Peter is diving into the technical details and some of the performance engineering that goes into @awscloud Lambda in order to reach this scale...

#reinvent

if you are interested in not only #serverless compute, but the whole concept. check out serverlessland.com

#reinvent

#reinvent

Peter using a castle 🏰 analogy for virtual machines. key point: small number of access points. built for defence

#reinvent

“Mostly secure isn’t good enough”, Peter

#reinvent

#reinvent

“At @awscloud, we always use an isolated virtual machine for customer compute workloads”, Peter >> thus Nitro!

#reinvent

back to Lambda…

think of it as a cache. a big pool of compute

#reinvent

each slot was specific to a customer function. new function? new virtual machine

if a VM is up for the customer function, great. if not, initialize a new instance (empty & waiting) to handle it >> a/k/a cold starts

…it works, but there’s room for improvement

#reinvent

more graphs

#reinvent

so cold starts are a cache miss. that happens, caching isn’t perfect

#reinvent

obvious solution: bigger cache on the @awscloud backend

that’s more expensive and less efficient

enter Firecracker

#reinvent

more details on Firecracker: aws.amazon.com/blogs/aws/fire…

#reinvent

moving to Firecrack provides more efficient use of the existing hardware. it also spins up faster, allowing things to sit in the cache longer!

#reinvent

results are Firecracker, bigger arrow, less bump

numbers would be helpful here. Peter mentioned one but I’m not sure I heard it right

assumption: things got better with the move to Firecracker

#reinvent

even with these improvements, you’ll see still cache misses (a/k/a cold starts)

need to make them faster. where is that time going?

for Java devs? a lot of it is in initialization

#reinvent

solution is to save the initialized start. think opening up your MacBook’s lid and working vs. turning it on from completely off

#reinvent

this feature is called AWS Lambda SnapStart and it’s available today!

bye bye cold starts…at least most of them

#reinvent

places where SnapStart _won’t_ work/help #1:

- some cryptography use cases where uniqueness is required
- can adjust with code changes

#reinvent #serverless

interesting division of snapshot-ing here. separating out the uniqueness to improve performance even further

#reinvent

last piece of the puzzle, predictive snapshot loading

get the pieces needed to start the function up and running as quickly as possible…before it’s even asked for

#reinvent

pumped to see the benefits of @awscloud Lambda SnapStat. love that I really don’t have to do anything to get those benefits

#reinvent #serverless

Peter signs off with some great session recommendations for performance engineering

#reinvent

great keynote from Peter DeSantis.hoping we’re going to see some @AmazonScience or @awscloud papers on these innovations soon

lots of great advancements & engineering work to deliver more features & functionality to builders

…more tomorrow as #reinvent kicks into high gear!