New code walkthrough on keras.io: speech recognition with Transformer. Very readable and concise demonstration of how to build and train a speech recognition model on the LJSpeech dataset.
keras.io/examples/audio…
keras.io/examples/audio…
This example was implemented by @NandanApoorv. Let's take a look at the model architecture.
It starts by defining two embedding layers: a positional embedding for text tokens, and an embedding for speech features, that uses 1D convolutions with strides for downsampling.
It starts by defining two embedding layers: a positional embedding for text tokens, and an embedding for speech features, that uses 1D convolutions with strides for downsampling.
Then it defines a Transformer encoder, which is your usual Transformer block, as well as a Transformer decoder, which is also your usual Transformer block, but with causal attention to prevent later timesteps to influence the decoding of earlier timesteps.
Finally, it combines the encoder and decoder in a Model subclass, where training logic is packaged in the `train_step()` method (this enables training via `fit()`, which gives you callbacks and distribution support for free). Also note the `generate()` method for inference!
• • •
Missing some Tweet in this thread? You can try to
force a refresh
