We release the full technical report & code for our OGB-LSC entry, in advance of our KDD Cup presentations! 🎉
arxiv.org/abs/2107.09422
See thread 🧵 for our insights gathered while deploying large-scale GNNs!
with @PeterWBattaglia @davidmbudden @andreeadeac22 @SibonLi et al.
arxiv.org/abs/2107.09422
See thread 🧵 for our insights gathered while deploying large-scale GNNs!
with @PeterWBattaglia @davidmbudden @andreeadeac22 @SibonLi et al.
For large-scale transductive node classification (MAG240M), we found it beneficial to treat subsampled patches bidirectionally, and go deeper than their diameter. Further, self-supervised learning becomes important at this scale. BGRL allowed training 10x longer w/o overfitting.
For large-scale quantum chemical computations (PCQM4M), going deeper (32-50 GNN layers) yields monotonic and consistent gains in performance. To recover such gains, careful regularisation is required (we used Noisy Nodes). RDKit conformers provided a slight but significant boost.
Our findings are backed by extensive single-model ablations on the LSC validation data, and we hope they will be of use for large-scale GNN practitioners in the future. We thank the OGB-LSC team (@weihua916 @rusty1s @jure @ericdongyx et al.) for enabling this fantastic contest!
• • •
Missing some Tweet in this thread? You can try to
force a refresh
