New Keras feature: the TextVectorization layer. It takes as input strings and takes care of text standardization, tokenization, and vocabulary indexing.
This enables you to create models that process raw strings.
End-to-end text classification example: colab.research.google.com/drive/1RvCnR7h…
This enables you to create models that process raw strings.
End-to-end text classification example: colab.research.google.com/drive/1RvCnR7h…
Key features:
- Supports sparse outputs (int sequences), to be fed into an Embedding layer
- Supports dense outputs (binary, tf-idf, count)
- Built-in ngram generation
Full credits to Mark Omernick for the code example and doing much of the work on this project.
- Supports sparse outputs (int sequences), to be fed into an Embedding layer
- Supports dense outputs (binary, tf-idf, count)
- Built-in ngram generation
Full credits to Mark Omernick for the code example and doing much of the work on this project.
Such a layer makes your text-processing model end-to-end: ingests strings, outputs classes/etc. You can deploy your model without worrying about the external preprocessing pipeline.
