Thread by @PyTorch

ICYMI: PyTorch 1.10 was released last Thursday. Here are some highlights of the release.

Stay tuned for tweet threads in the next couple weeks delving deeper into these cool new features!

1/8

CUDA Graphs are now in beta, and allow you to capture (and replay!) static CUDA workloads without needing to relaunch kernels, leading to massive overhead reductions! Our integration allows for seamless interop between CUDA graphs and the rest of your model.

2/9

FX, an easy to use Python platform for writing Python-to-Python transforms of PyTorch programs, is now in stable. FX makes it easy to programmatically do things like fusing convolution w/ batch norm. Stay tuned for some FX examples of cool things that users have built!

3/9

nn.Module parametrization (moving from Beta to Stable) allows you to implement reparametrizations in an user-extensible manner. For example, you can apply spectral normalization or enforce that the parameter is orthogonal! See

https://twitter.com/rasbt/status/1451542449130811399

for an example.

4/9

We have also turned conjugation on complex tensors from an O(N) copy into a constant time operation (just like transpose)! This allows us to fuse conjugation with other PyTorch operators, like matmuls, for as much as 50% increase in speed & 30% increase in memory savings!

5/9

A new LLVM-based JIT compiler is now available for CPUs that can fuse together sequences of PyTorch ops to improve performance. While we’ve had this ability for some time on GPUs, this release brings this capability to CPUs. For certain cases this can bring massive speedups!
6/9

NNAPI (which was moved from prototype to beta) allows PyTorch on Android to leverage specialized hardware such as GPUs or other specialized chips to accelerate neural networks. Since the prototype release, we’ve been hard at work adding more op coverage and other goodies.

7/9

One pain point for using torch.jit.script is that it has often required type annotations in order for compilation to be successful. Now, we’ve enabled profile directed typing for torch.jit.script by leveraging existing tools like MonkeyType, accelerating the process.

8/9

Some additional links:
CUDA Graphs: pytorch.org/docs/master/no…
FX: pytorch.org/docs/master/fx…
CPU Fuser: colab.research.google.com/drive/1xaH-L0X…
NNAPI: pytorch.org/tutorials/prot…
Conjugate View: pytorch-dev-podcast.simplecast.com/episodes/conju…

9/9

Here is an example benchmark showing the improved PyTorch performance on a HPC benchmark: github.com/dionhaefner/py…