Interesting analysis by @mhmazur. Human work is driven by clear goals and is informed by task-specific context. A model that is optimized for generating plausible-sounding text, ignoring goals and context, virtually never produces any useful answer (unless by random chance).
https://twitter.com/mrpatto/status/1362844477157560320
Reminder: language serves a variety of purposes -- transmit information, act on the world to achieve specific goals, serve as a social lubricant, etc. Language cannot be modeled as a statistical distribution independent of these purposes.
This is akin to modeling the appearance of animals as a statistical distribution while ignoring the environment in which they live. You could use such a model to generate plausible-looking animals, but don't expect them to be able to survive in the wild (environmental fitness)
Animals evolved to fit their environment -- everything about them (us) is a product of environmental constraints. Likewise language is a construct evolved to fit a specific set of functions, and you cannot model it independently from this context.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
