Thread by @jsusskin

I’m a bit late, but check out @Apple research papers presented at #ICML2022 here: machinelearning.apple.com/research?year=…
Includes work on transformer pre-training, dynamic pooling, private federated statistics, and generative models.
1/n

arxiv.org/abs/2207.07611 provides a simple method for transformer pre-training based on predicting the positions of a set of orderless tokens. Often requires fewer pre-training iterations to achieve a good model.
2/n

By @zhaisf @navdeepjaitly10 @jramapuram @danbusbridge Tatiana Likhomanenko @josephycheng @waltertalbott @chen_huang23 @hanlingoh @jsusskin.
3/n

A plug-in dynamic pooling approach was introduced in arxiv.org/abs/2207.01844 for CNNs and transformers that improves performance-cost trade-offs. By @chen_huang23, @waltertalbott @navdeepjaitly10 @jsusskin.
4/n

@HilalAsi2 @vitalyFM @_kunal_talwar_ offer new algorithms for low variance, private mean estimation in arxiv.org/abs/2205.02466.
5/n

Private histogram frequency estimators are introduced in arxiv.org/abs/2203.00194. By @vitalyFM @minilek @HuyLeNguyen @_kunal_talwar_.
6/n

Xavier Suau, Luca Zappella, and Nick Apostoloff introduced arxiv.org/abs/2110.02802 for guiding language generation with expert units derived from latent transformer features, helping to study and mitigate bias in generated text.
7/n

Finally, colleagues in AI/ML along with #UCSD authors introduced a new method for controlling style and content for handwritten text in arxiv.org/abs/2110.02891.
8/n

I hope this is helpful for those who may have missed the #ICML2022 presentations or wanted to se a more comprehensive list. Enjoy!
n/n