Thread by @hadyelsahar

Marco Baroni is starting the first invited talk at #rep4nlp
"Language is representations by itself. " #ACL2019nlp @sigrep_acl

@sigrep_acl Marco is talking about his previous work about the emergence of language communication between agents.

References:

"MULTI-AGENT COOPERATION
AND THE EMERGENCE OF (NATURAL) LANGUAGE"
arxiv.org/pdf/1612.07182…

"How agents see things"
aclweb.org/anthology/D18-…

#ACL2019nlp

@sigrep_acl Efficient encoding of input information:

"We explore providing some information to the sender and receiver agents and look at the emerging language if it develops to ignore the redundant parts of the input"

Kharitonov et al. 2019
arxiv.org/abs/1905.13687

#ACLnlp2019 #rep4nlp

@sigrep_acl The Straight-through Gumbelsoftmax was used to backpropagate through the discrete sampling process of language generation between agents.

The discreteness of language (lower values of Gumbel softmax temp) is useful to overcome noise "maybe this is why language is discrete in the first place?"

Very interesting point of view to see language!

Human language doesn't only efficiently encode language but also in how it encodes it.

Most frequent words are short while less frequent ones are usually longer

#REP4nlp #ACL2019nlp

Agents have similar patterns as the human language on how to efficiently encode messages forms in between them (assigning short forms to frequent meanings)

@RahmaChaabouni et al.
arxiv.org/abs/1905.12561

#REP4nlp #ACL2019nlp

However, the longer messages are easier to discriminate for the receiver agent so the encoder is not pushed to stick to that

This yields to the "anti-efficient encoding".

Penalizing agents with brevity penalty fixes that but still doesn't reach the efficiency of human languages

@RahmaChaabouni However, the longer messages are easier to discriminate for the receiver agent so the encoder is not pushed to stick to that. This yields to the "anti-efficient encoding".

Penalizing agents with brevity penalty fixes that but still doesn't reach the efficiency of human languages

@RahmaChaabouni Conclusions see pic :)

End of talk.

#ACL2019nlp
#REP4nlp

@RahmaChaabouni Conclusions see pic :)

End of talk.

#ACL2019nlp
#REP4nlp

Full summary of Marco baroni's invited talk at #rep4nlp #acl2019nlp about "emerging (natural) language between agents" 👆👆👆

Thanks Marco for the great talk!!

@threadreaderapp unroll