Thread by @DataScienceHarp

Skip connections are one of the most important innovations in the history of deep learning.

You need to understand the problems it solves and how it helps us build deep networks.

Let me break it down for you in this thread 👇🏽🧵

#deeplearning #machinelearning

Skip connections are a common feature in modern CNN architectures. They create an alternative path for the gradient to flow through, which can help the model learn faster.

In a neural network, the gradient measures how much a change in one part of the network affects the output. We use the gradient to update the network during training to recognize data patterns better.

But in a deep network with many layers, the gradient can become very small as it flows backwards through the network. If the gradient is too small, the network won't learn effectively.

Skip connections help to prevent this problem by allowing the gradient to flow through multiple paths rather than just one. This can keep the gradient from becoming too small and speed up the training process.

There are two main ways to implement skip connections: addition and concatenation. In residual networks, skip connections are added. In densely connected networks, skip connections are concatenated.

Skip connections are a useful tool for improving deep learning model performance.

If you want more deep learning fundamentals in your feed for free, then follow me @DataScienceHarp