FELIX, a fast and flexible text-editing system that models large structural changes and achieves a 90x speed-up compared to seq2seq approaches whilst achieving impressive results on four monolingual generation tasks.
Abs aclweb.org/anthology/2020…
github.com/google-researc…
Abs aclweb.org/anthology/2020…
github.com/google-researc…
https://twitter.com/GoogleAI/status/1390046279468830721
Compared to traditional seq2seq methods, FELIX has the following three key advantages:
Sample efficiency: Training a high precision text generation model typically requires large amounts of high-quality supervised data.
Sample efficiency: Training a high precision text generation model typically requires large amounts of high-quality supervised data.
FELIX uses three techniques to minimize the amount of required data:
(1) fine-tuning pre-trained checkpoints,
(2) a tagging model that learns a small number of edit operations, and
(3) a text insertion task that is very similar to the pre-training task.
(1) fine-tuning pre-trained checkpoints,
(2) a tagging model that learns a small number of edit operations, and
(3) a text insertion task that is very similar to the pre-training task.
Fast inference time: FELIX is fully non-autoregressive, avoiding slow inference times caused by an autoregressive decoder.
Flexible text editing: FELIX strikes a balance between the complexity of learned edit operations and flexibility in the transformations it models
Flexible text editing: FELIX strikes a balance between the complexity of learned edit operations and flexibility in the transformations it models
• • •
Missing some Tweet in this thread? You can try to
force a refresh
