Encoding text in numbers is a very important part of NLP as the better this can be done, the better are the possible results!
Word embedding works but they don't have the full context of the sentence.
This is where BERT comes in
But what is BERT?
1/9🧵
Word embedding works but they don't have the full context of the sentence.
This is where BERT comes in
But what is BERT?
1/9🧵
When we do word embedding, both sentences
• They are running a test
• They are running a company
Will have very similar embeddings but the meaning of both sentences are very different. Without this context, the model using this encoding will be blind to the context
2/9🧵
• They are running a test
• They are running a company
Will have very similar embeddings but the meaning of both sentences are very different. Without this context, the model using this encoding will be blind to the context
2/9🧵
This is where Bidirectional Encoder Representations from Transformers (BERT) comes in play!
It is a Transformer-based network created in 2018 and
takes into account the context of the word's occurrence. For the previous example, it gives very different embeddings.
3/9🧵
It is a Transformer-based network created in 2018 and
takes into account the context of the word's occurrence. For the previous example, it gives very different embeddings.
3/9🧵
Understanding context is key for search engines and that's why Google's search engine is powered by BERT models in multiple languages!
blog.google/products/searc…
4/9🧵
blog.google/products/searc…
4/9🧵
BERT models are usually measured against some common benchmarks tasks like
• GLUE
• SQuAD
• SWAG
These tasks involve:
• Q&A
• Is a sentence grammatically correct?
• is sentence A a follow up of B?
• Is sentence A similar to B?
• Sentiment Analysis
• ...
5/9🧵
• GLUE
• SQuAD
• SWAG
These tasks involve:
• Q&A
• Is a sentence grammatically correct?
• is sentence A a follow up of B?
• Is sentence A similar to B?
• Sentiment Analysis
• ...
5/9🧵
BERT models are usually very big but there are versions available to run on-device or on the browser!
A very nice use case is on the Recorder App.
It will create a smart scroll for the transcription of the recorded audio!
ai.googleblog.com/2020/11/naviga…
6/9🧵
A very nice use case is on the Recorder App.
It will create a smart scroll for the transcription of the recorded audio!
ai.googleblog.com/2020/11/naviga…
6/9🧵
Another nice example is Question Answering.
This sample can help you get started on a mobile phone with TFLite:
tensorflow.org/lite/examples/…
7/9🧵
This sample can help you get started on a mobile phone with TFLite:
tensorflow.org/lite/examples/…
7/9🧵
You can find multiple BERT (and BERT-like) models on #TFHub in this collection:
tfhub.dev/google/collect…
Do you want to know more about BERT models?
leave your questions on comments
9/9🧵
tfhub.dev/google/collect…
Do you want to know more about BERT models?
leave your questions on comments
9/9🧵
• • •
Missing some Tweet in this thread? You can try to
force a refresh
