Excited to announce that our work, “Computational Language Acquisition with Theory of Mind”, has been accepted to #ICLR2023! We equip language-learning agents with Theory of Mind, which improves their performance on an image referential game. arxiv.org/abs/2303.01502 (1/5)
Many language acquisition theories involve children’s ability to ascribe mental states to others. We model this as an internal module trained to predict listener behavior. The speaker generates many candidates and uses an internal listener to rerank and choose between them. (2/5) 
We find significant gains in referential game accuracy, suggesting that equipping language-learning agents with ToM can improve referential game performance independently of general captioning ability. We also observe improvements in the fluency and precision of captions. (3/5)
We also consider the role of environmental pressures by selecting more semantically or visually similar distractors. We find that speakers trained on harder distractors learn utterances that are longer, more fluent, and closer to the ground-truth utterances. (4/5)
Code and data are available at github.com/neulab/ToM-Lan…. Feel free to reach out or visit us at the ICLR poster session in Kigali! With @_Hao_Zhu @_emliu @ybisk @gneubig (5/5)
• • •
Missing some Tweet in this thread? You can try to
force a refresh
