Sometimes you need to build a Machine Learning model that cannot be expressed with the Sequential API
For these moments, when you need a more complex model, with multiple inputs and outputs or with residual connections, that's when you need the Functional API!
[2.46 min]
1/8🧵
The Functional API is more flexible than the Sequential API.
The easiest way to understand is to visualize the same model created using the Sequential and Functional API
2/8🧵
You can think of the Functional API as a way to create a Directed Acyclic Graph (DAG) of layers while the Sequential API can only create a stack of layers.
Functional is also known as Symbolic or Declarative API
3/8🧵
Thinking with the Graph creation in mind, given the layers A and B are "vertices", when you call:
B(A)
You're are creating an edge between them like: A ➡️ B
4/9🧵
Benefits of the Functional API:
• Plotting the model and model.summary work as expected to give a good visualization
• Debugging will happen during model definition since layers can be "type-checked"
5/9🧵
The Functional API has the limitation of enable only directed acyclic graphs, if you need something like a dynamic network or recursive networks, than you won't be able to build with this API
6/9🧵
This blog post by @random_forests has an even better explanation of the Functional API
One very interesting task on the NLP fields is text generation.
There are very advanced techniques and a lot of research on it and even business based solely on it!
But how does it work?
[7.47min]
[I guarantee it's a much better read then doom scrolling!!!]
1/11🧵
Let's think: what a model would have to do to generate text?
The rationale is, as humans we form sentences by trying to create a sequence of words that makes sense.
The less random this sequence looks like, the better the output text is and closer to human like.
2/11🧵
Here is where ML can help.
A model should learn how to combine the words the best way possible.
The simplest way to teach this is: given a sentence, hide the last word and let the model try to guess it.
The loss function measures how good the model's guess is.
Sometimes you need to create your own model for your specific data corpus (eg: legal, science, medical texts)
To create your own model, AutoML Natural Language can help you:
2/4🧵
If you want to build everything from scratch, then you'll need:
• a language embedding (like BERT, ELMO, USE) and #TFHub have all you need
• a dataset and this github.com/juand-r/entity… can help you find one
Encoding text in numbers is a very important part of NLP as the better this can be done, the better are the possible results!
Word embedding works but they don't have the full context of the sentence.
This is where BERT comes in
But what is BERT?
1/9🧵
When we do word embedding, both sentences
• They are running a test
• They are running a company
Will have very similar embeddings but the meaning of both sentences are very different. Without this context, the model using this encoding will be blind to the context
2/9🧵
This is where Bidirectional Encoder Representations from Transformers (BERT) comes in play!
It is a Transformer-based network created in 2018 and
takes into account the context of the word's occurrence. For the previous example, it gives very different embeddings.