Our new topological, diffusion-bridge based graph generation framework, HOG-Diff [], leverages higher-order information as a guide to progressively generate plausible graphs thru a coarse-to-fine curriculum. #TDL #TopologicalDeepLearning #GenAI #ML #AI 👇 arxiv.org/abs/2502.04308 Relations beyond pairwise interactions are essential in networks. Yet, their role in semantic graph generative models remains unclear, as mentioned in our #ICML2024 paper []. We address this by integrating higher-order information in a principled manner.👇arxiv.org/abs/2402.08871

Our latest work [] explores sudden generalization in neural nets, aka #grokking. We identify & propose solutions to two key issues hindering grokking: (i) floating point errors in Softmax and (ii) aligned gradients naively scaling logits post-overfitting.👇 arxiv.org/abs/2501.04697 First, we observe that grokking is prevented by absorption errors in the Softmax, in the absence of regularization. This problem, which we refer as the Softmax Collapse (SC), causes vanishing gradients and puts an end to learning, sometimes resulting in complete overfitting.👇

Graphs & geometric deep learning are cool, but what's next? Our latest work leverages topology to model higher order relationships beyond graphs, covering domains like meshes, sets & their mixtures naturally. Learn more: arxiv.org/abs/2206.00606 #TopologicalDeepLearning #TDL So what's a #HigherOrderRelationship? Essentially, it's a relation between relations. Graphs only consider node-to-node relations (via so called edges). Yet, relations between edges or edges & faces, encode interesting patterns, leading to a full hierarchy of relationships.