As promised, we have made the Text-To-Text Transfer Transformer (T5) models much easier to fine-tune for new tasks, and we just released a Colab notebook where you can try it yourself on a free TPU!
👇
tiny.cc/t5-colab
(1/3)
👇
tiny.cc/t5-colab
(1/3)
In the Colab, we fine-tune T5 to answer questions *without context*! This forces T5 to "look up knowledge" it obtained during pre-training, and it does surprisingly well.
For example, we can ask "How many legs does a ladybird have?” and it knows! Do you?
hint:🐞
(2/3)
For example, we can ask "How many legs does a ladybird have?” and it knows! Do you?
hint:🐞
(2/3)
Looking forward to seeing what fun, creative (I'm looking at you #MadeWithMagenta crowd), and/or important tasks you fine-tune T5 on.
Let us know!
(3/3)
Let us know!
(3/3)
PS - for those of you at #NeurIPS2019:
@colinraffel, @katherine1ee, and I will be demoing and answering questions about T5 at the @GoogleAI booth on Wednesday at 3:05.
See you there!
(4/3)
@colinraffel, @katherine1ee, and I will be demoing and answering questions about T5 at the @GoogleAI booth on Wednesday at 3:05.
See you there!
(4/3)
PPS - big thanks to @sharan0909 for making the API changes and Noam Shazeer for coming up with the Trivia task!
(5/3)
(5/3)
3/4 
1. 
