Discover and read the best of Twitter Threads about #PerceiverIO
Most recents (1)
Rewatched @Tesla's AI day recently, and when @karpathy introduced the Transformer used in AutoPilot, it immediately reminded me of @DeepMind's #PerceiverIO which I recently contributed @huggingface. Wonder whether Tesla's approach was inspired by it... 
... or whether they were already using this (long) before the paper's introduction. Especially the sentence "you initialize a raster the size of the output space that you'd like and tile it with position encodings "=> this is exactly what Perceiver IO does as well! @drew_jaegle
This idea is brilliant actually: the features of the 8 camera's serve as keys (K) and values (V), while the individual pixels of the output (vector) space (bird's eye view) provide queries (Q) for multi-head attention (tiled with sin/cos position embeddings).
