5 activation functions you should know!
🧵
1/9
🧵
1/9
Yesterday we touched on activation functions, you should read that thread as well:
2/9
https://twitter.com/levikul09/status/1664915552379482112?s=20
2/9
Activation Functions are just like any other mathematical function.
It has three elements/steps:
- Input (X-axis)
- Calculation
- Output (Y-axis)
Different activation functions do different math. Let's discuss 5 🔽
3/9
It has three elements/steps:
- Input (X-axis)
- Calculation
- Output (Y-axis)
Different activation functions do different math. Let's discuss 5 🔽
3/9
1️⃣ ReLU
ReLU is widely used due to its simplicity and effectiveness.
It returns the input value if it is positive and zero otherwise.
Usually, ReLU is the default activation function.
4/9
ReLU is widely used due to its simplicity and effectiveness.
It returns the input value if it is positive and zero otherwise.
Usually, ReLU is the default activation function.
4/9
2️⃣ Sigmoid
The sigmoid is a smooth S-shaped curve that maps the input to a value between 0 and 1.
Sigmoid can be used for learning complex decision functions since it introduces non-linearity.
It is mainly used for binary classification.
5/9
The sigmoid is a smooth S-shaped curve that maps the input to a value between 0 and 1.
Sigmoid can be used for learning complex decision functions since it introduces non-linearity.
It is mainly used for binary classification.
5/9
3️⃣ Tanh
Tanh is similar to the Sigmoid function but maps the input to a value between -1 and 1.
6/9
Tanh is similar to the Sigmoid function but maps the input to a value between -1 and 1.
6/9
4️⃣ Leaky ReLU
Leaky ReLU is a variation of the ReLU function.
It introduces a small slope for negative inputs, preventing neurons from becoming completely inactive (zero).
7/9
Leaky ReLU is a variation of the ReLU function.
It introduces a small slope for negative inputs, preventing neurons from becoming completely inactive (zero).
7/9
5️⃣ Softmax
Softmax is primarily used in the output layer for multi-class classification problems.
It transforms the raw outputs of the neural network into a vector of probabilities.
Softmax ensures that the sum of the output probabilities is equal to 1.
8/9
Softmax is primarily used in the output layer for multi-class classification problems.
It transforms the raw outputs of the neural network into a vector of probabilities.
Softmax ensures that the sum of the output probabilities is equal to 1.
8/9
That's it for today.
I hope you've found this thread helpful.
Like/Retweet the first tweet below for support and follow @levikul09 for more Data Science threads.
Thanks 😉
9/9
I hope you've found this thread helpful.
Like/Retweet the first tweet below for support and follow @levikul09 for more Data Science threads.
Thanks 😉
9/9
https://twitter.com/levikul09/status/1665280560481312771
• • •
Missing some Tweet in this thread? You can try to
force a refresh
