Discover and read the best of Twitter Threads about #stationaryrepresentation
Most recents (1)
🧩Three intriguing concepts in the world of #DeepLearning: #BackwardCompatibility, #StationaryRepresentation, and #NeuralCollapse. Their connection may hold the key to better deep learning models with possibly novel ways to use them
(1/7)
(1/7)
🤝Backward Compatibility enables matching internal feature representations from different neural networks. Stationary Representation maintains feature spatial configurations fixed during learning. How does Neural Collapse fit in?
(2/7)
(2/7)
💥Neural Collapse is when feature representations converge towards a simpler structure during training. This fascinating phenomenon is strongly related to compatibility and stationary representation.
(3/7)
(3/7)
