here we go! Monday Night Live with Peter DeSantis is about to kick off at @awscloud #reinvent 2022!
a 🧵👇 (/cc @AWSEvents)
@awscloud @AWSEvents I love how much fun Peter has with this keynote!
#reinvent
Peter reminds everyone that this keynote is all about "how" @awscloud does things. lots of behind the scenes info in this one..
#reinvent
big focus on performance tonight.
Peter reminds us it's not all about what you can see
"Great performance is the result of innovation from the ground up and investing over time"
#reinvent
starting with a dive into @awscloud Nitro...
more on Nitro at aws.amazon.com/ec2/nitro/
#reinvent
here’s a great talk from Abby Fuller on the technical details of AWS Nitro:
#reinvent
Peter announces a new version of Nitro (v5), with big gains behind the scenes. continued innovation behind the scenes from @awscloud here
#reinvent
NEW INSTANCE TYPE:
C7gn
uses the new Nitro v5 chip under the hood. lots of benefits. tip of the iceberg here
#reinvent
NEW CPU:
Graviton3E
custom tuned Graviton3 silicon. lots of benefits for the targeted use cases
#reinvent
NEW INSTANCE TYPE (coming soon):
HPC7g
…combines the Graviton3E + Nitro v5 for some big boosts for HPC applications
#reinvent
not sure I’m digging the graphic style for these slides...
#reinvent
typography with Amazon Ember is on point though. nice and clean. super clear. great typeface
#reinvent
back to the tech, here’s a great summary of the new instance types just announce from @jeffbarr: aws.amazon.com/blogs/aws/new-…
#reinvent
btw, more on the Graviton processor at aws.amazon.com/ec2/graviton/
#reinvent
Peter has moved on to how the @awscloud Global Infrastructure is designed and built
specifically talking about SRD (scalable reliable datagram)
more on that at: aws.amazon.com/blogs/hpc/in-t…
#reinvent
more on the @awscloud global infrastructure at aws.amazon.com/about-aws/glob…
#reinvent
Peter is giving a great high level overview of TCP as a protocol
tl:dr it works great, but focuses on “getting there” vs. performance
at @awscloud scale, they need something more efficient and focused on hyper scale issues
#reinvent
key benefits of SRD on the @awscloud network:
- multi-pathing
- retries in microseconds
- runs on nitro (dedicated hardware)
#reinvent
remember, @awscloud runs at a completely different scale. very few companies run at this size. it’s a completely different set of problems, requiring completely different solutions
#reinvent
SRD is helping deliver better performance for Amazon EBS, especially for “tail latency” …average latency doesn’t matter if you just need the data!
#reinvent
teal line is SRD performance. big improvement overall, but massive in the edge cases
…btw, all new EBS io2 volumes will get this performance boost in the new year (stay tuned, they are just saying “coming soon")
#reinvent
NEW LAUNCH:
ENA Express, brings SRD to ENA…available today!
more details on ENA (basic, not express): docs.aws.amazon.com/AWSEC2/latest/…
#reinvent
here’s the SRD boost for ENA with ENA Express (needs to be enabled on your ENAs)
#reinvent
“A veritable hoedown of machine learning”, Peter DeSantis
🤦🤦🤦
oh Peter….
#reinvent
this graphic redeems part of the hoedown comment…
would have been better if he drove it out on stage 😉
#reinvent
gist of this section: 🧠 machine learning models are getting MASSIVE
they need more performance (processing, storage, network, etc.) in order to train properly. @awscloud innovations in performance engineering like SRD are helping drive these advances
#reinvent
here are the details on the @awscloud Trn1 instance type that Peter referenced: aws.amazon.com/ec2/instance-t…
#reinvent
while they make the point, graphs without units and numbers are tough for me to look at
that said, numbers and more specifics would actual make these slides weaker and muddle the point Peter is making
#reinvent
“Just as a refresher. Stochastic rounding is probabilistic rounding”, Peter
#reinvent
why is that important 👆? it helps train the more faster with less loss. @awscloud accomplishes this with hardware acceleration in the Trainium silicon
more on that chip at aws.amazon.com/machine-learni…
#reinvent
also, here’s a mathematical primer on stochastic rounding: nhigham.com/2020/07/07/wha…
you know, because MATH! 🤣
#reinvent
the tl:dr of this section is that @awscloud has been focusing on a lot of deeply interesting (to me at least!) performance engineering problems so that we can build & train ml 🧠 models faster and more accurately
#reinvent
how else would we get fun/weird avatar pics in different styles?
🤣🤣🤣
#reinvent
Peter is currently giving a great explanation of how to scale out a workload across multiple processors. I can’t explain it in 280 character chunks 🤦
basically: many hands make light work?
#reinvent
you’re going to have to watch this segment for yourself. it starts at 35m into this keynote with this slide 👇
#reinvent
Peter brings this back home using a Rings of Power reference. btw, the show is very much worth watching
amazon.com/Lord-Rings-Pow…
#reinvent
lots of work to average out the results, but the “ring of rings” works much faster than “one ring”
…as long as you don’t give 9 to the race of man, 3 to the Elves, 7 for the Dwarves, and one to Sauron 🤦
#reinvent
new instance type coming soon to help speed things up even more:
Trn1n
- faster networking
- faster training
#reinvent
Jock Clear up now talking about the technical complexity of F1 racing
more details on @awscloud + F1 at aws.amazon.com/sports/f1/
#reinvent
three key aspects of making zoom-y cars zoom faster...
(somewhere @mosescj58 just felt a chill 🤣)
#reinvent
with @awscloud, Jock and the team built a virtual ground speed sensor
it’s an AI/ML tool that provides more accurate data faster…and more importantly with less weight
…would love to see more technical details on that one
#reinvent
btw, @QuinnyPig is also live tweeting this keynote. honestly, probably with a lot more—and funnier—jokes
check out his thread at
#reinvent
Jock continues to lay out the challenges that F1 teams face outside of the race and the onslaught of challenges within a race
lots of engineering challenges and trade offs…
#reinvent
I’m not a car person (see the previous “zoom-y” comment), but I do love engineering challenges. lots of crazy ones here
Jock is constantly calling out the sheer number of decisions the team needs to make. they need data & analysis to make those decisions
#reinvent
they are launching an app with a bunch of race/interview content
…was hoping for an engineering data app like the real time MLB analysis: aws.amazon.com/blogs/machine-…
..it’s not that, but I’m sure F1 fans will enjoy it
#reinvent
Peter is back up now. new topic: #serverless computing
#reinvent
Lambda has grown a LOT since it’s launch in 2014
thanks again @timallenwagner for setting us on this path!
#reinvent
Peter is diving into the technical details and some of the performance engineering that goes into @awscloud Lambda in order to reach this scale...
#reinvent
if you are interested in not only #serverless compute, but the whole concept. check out serverlessland.com
#reinvent
Peter using a castle 🏰 analogy for virtual machines. key point: small number of access points. built for defence
#reinvent
“Mostly secure isn’t good enough”, Peter
#reinvent
“At @awscloud, we always use an isolated virtual machine for customer compute workloads”, Peter >> thus Nitro!
#reinvent
back to Lambda…
think of it as a cache. a big pool of compute
#reinvent
each slot was specific to a customer function. new function? new virtual machine
if a VM is up for the customer function, great. if not, initialize a new instance (empty & waiting) to handle it >> a/k/a cold starts
…it works, but there’s room for improvement
#reinvent
more graphs
#reinvent
so cold starts are a cache miss. that happens, caching isn’t perfect
#reinvent
obvious solution: bigger cache on the @awscloud backend
that’s more expensive and less efficient
enter Firecracker
#reinvent
more details on Firecracker: aws.amazon.com/blogs/aws/fire…
#reinvent
moving to Firecrack provides more efficient use of the existing hardware. it also spins up faster, allowing things to sit in the cache longer!
#reinvent
results are Firecracker, bigger arrow, less bump
numbers would be helpful here. Peter mentioned one but I’m not sure I heard it right
assumption: things got better with the move to Firecracker
#reinvent
even with these improvements, you’ll see still cache misses (a/k/a cold starts)
need to make them faster. where is that time going?
for Java devs? a lot of it is in initialization
#reinvent
solution is to save the initialized start. think opening up your MacBook’s lid and working vs. turning it on from completely off
#reinvent
this feature is called AWS Lambda SnapStart and it’s available today!
bye bye cold starts…at least most of them
#reinvent
places where SnapStart _won’t_ work/help #1:
- some cryptography use cases where uniqueness is required
- can adjust with code changes
#reinvent #serverless
interesting division of snapshot-ing here. separating out the uniqueness to improve performance even further
#reinvent
last piece of the puzzle, predictive snapshot loading
get the pieces needed to start the function up and running as quickly as possible…before it’s even asked for
#reinvent
pumped to see the benefits of @awscloud Lambda SnapStat. love that I really don’t have to do anything to get those benefits
#reinvent #serverless
Peter signs off with some great session recommendations for performance engineering
#reinvent
great keynote from Peter DeSantis.hoping we’re going to see some @AmazonScience or @awscloud papers on these innovations soon
lots of great advancements & engineering work to deliver more features & functionality to builders
…more tomorrow as #reinvent kicks into high gear!
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.