At Apple working on new deep learning methods and fundamental ML research. Previously built neural networks for FaceID.
Jul 30, 2022 • 9 tweets • 8 min read
I’m a bit late, but check out @Apple research papers presented at #ICML2022 here: machinelearning.apple.com/research?year=…
Includes work on transformer pre-training, dynamic pooling, private federated statistics, and generative models.
1/narxiv.org/abs/2207.07611 provides a simple method for transformer pre-training based on predicting the positions of a set of orderless tokens. Often requires fewer pre-training iterations to achieve a good model.
2/n
Apr 3, 2021 • 4 tweets • 2 min read
Look! Realistic scene generation with a freely moving camera! Compare the effect of constrained vs free viewing. This is an effort we envisioned several years ago when thinking about the power of neural nets to learn expressive priors about the world. 1\4
If we can generate realistic scenes from a prior, we can start to understand what it means to ‘see’ within the context of a scene, by inversion (analysis-by-synthesis). We wanted to do this with realistic scenes to ensure that GSN really understand the 3D world. 2\4
