Thread by @rohanpaul_ai

1/ "Software is eating the world. Machine learning is eating software. Transformers are eating machine learning."

Let's understand what these Transformers are all about

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataAnalytics

2/ #Transformers architecture follows Encoder and Decoder structure.

The encoder receives input sequence and creates intermediate representation by applying embedding and attention mechanism.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI

3/ Then, this intermediate representation or hidden state will pass through the decoder, and the decoder starts generating an output sequence.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics

4/ As opposed to directional models, which read text input sequentially (left-to-right or right-to-left), the Transformer encoder reads the entire sequence of words at once

Therefore it is considered bidirectional, though it may be said that its non-directional

#DataScience

5/ This characteristic allows the Transforme model to learn the context of a word based on all of its surroundings (left and right of the word).

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics #Python

6/ In the case of Machine Translation, like English to French translation, the Encoder will process English sentences as input, apply the attention mechanism, and encode it in the intermediate representation.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python

7/ This encoded input sequence then passes to the Decoder, generating the corresponding french sentence.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics #programming #ArtificialIntelligence #Data

8/
There are three types of transformers models
1. Encoder Only
2. Decode Only
3. Encoder-Decoder

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics #programming #ArtificialIntelligence #Data #Stats

9/ 👉 Encoder Only

BERT leverages the Encoder architecture of the transformer. BERT takes text sequence as input. The BERT Encoder produces BERT embedding, which can be used to perform downstream tasks like Text classification or Named Entity Recognition.

#DataScience

10/ 👉 Decoder Only
GPT models leverage the decoder architecture of the transformer. Given the input sequence as prompt, GPT starts generating the response. Therefore, GPT models are best suitable for text or sequence generation.

#DataScience #MachineLearning #DeepLearning

11/
👉 Encoder-Decoder
T5 is a model which uses both Encoder and Decoder architecture. It treats each task as text to text or sequence to sequence.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics

12/ E.g. with T5 model, In text classification, the Encoder takes text as input, and the Decoder generates text labels instead of classifying them

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics

13/ For daily tips and techniques on #DeepLearning, #ComputerVision and #MachineLearning follow me @rohanpaul_ai

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics #programming #ArtificialIntelligence

@threadreaderapp unroll