HPC Architecture Ronin, currently at @nvidia. Ramblings are my own.
He/Him
May 31, 2022 • 25 tweets • 5 min read
As promised: Let's clarify what GPU system "Peak" performance means, a🧵
Let's start with three definitions:
1) Maximum Architectural Peak Rate is the number of FP64 Function Units (FUs) that can run in parallel multiplied by the maximum clock rate the chip is capable of. Note that this number need not be achievable under typical/any circumstances. (see:
1/ How should we talk about DP FLOPS for A100? A thread. #HPC#AI#Nvidia2/ Nvidia announced A100 this morning and you can read about it from any number of outlets, e.g. nextplatform.com/2020/05/14/nvi…