Graph neural networks are driving lots of progress in machine learning by extending deep learning approaches to complex graph data and applications.
Let’s take a look at a few methods ↓
1) A Graph Convolutional Network, or GCN, is an approach for semi-supervised learning on graph-structured data. It’s based on an efficient variant of CNNs which operates directly on graphs and is useful for semi-supervised node classification.
2) Diffusion-convolutional neural networks (DCNN) introduce a diffusion-convolution operation to extend CNNs to graph data. This enables learning of diffusion-based representations. It's used as an effective basis for node classification.
3) Graph Attention Network (GAT) is a graph neural network that leverages masked self-attentional layers. Hidden representations of nodes are computed by attending to neighbours using self-attention. It achieves SOTA results on node classification.
4) GraphSAGE is a general inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate useful node embeddings for previously unseen data.
5) The Graph Transformer is a generalization of transformer neural networks for arbitrary graphs. The architecture introduces new properties that leverage the graph connectivity inductive bias to perform well on problems where graph topology is important.
And finally but not least... here is an extended list of graph neural networks and their associated papers, benchmark datasets, trends, and open source codes.
It proposes architectural changes that suppress aliasing and forces the model to implement more natural hierarchical refinement which improves its ability to generate video and animation.
In the cinemagraph below, we can see that in StyleGAN2 the texture (e.g., wrinkles and hairs) appears to stick to the screen coordinates. In comparison, StyleGAN3 (right) transforms details coherently:
2/8
The following example shows the same issue with StyleGAN2: textural details appear fixed. As for alias-free StyleGAN3, smooth transformations with the rest of the screen can be seen.
In a new paper from @wightmanr et al. a traditional ResNet-50 is re-trained using a modern training protocol. It achieves a very competitive 80.4% top-1 accuracy on ImageNet without using extra data or distillation.
The paper catalogues the exact training settings to provide a robust baseline for future experiments:
It also records training costs and inference times on ImageNet classification between other architectures trained with the proposed ResNet-50 optimized training procedure:
🚨 Newsletter Issue #3. Featuring a new state-of-the-art on ImageNet, a trillion-parameter language model, 10 applications of transformers you didn’t know about, and much more! Read on below:
⏪ Papers with Code: Year in Review. We’re ending the year by taking a look back at the top trending papers, libraries and benchmarks for 2020. Read on below!