Thread by @996roma

this paper from @shaohua0116 on guiding RL agents with program counterparts of natural language instructions was one of my favourites at #ICLR2020. here is why i think it's exciting and quite different from existing work. 1/6

https://twitter.com/shaohua0116/status/1207872533191520256

there's a large literature of #NLProc work on semantic parsing (converting language->executable meaning representations) for a variety of tasks. this is helpful e.g., for database operations, for goals/rewards for planners, to ground to predefined robotic actions etc. 2/6

apart from select works, a lot of the time, the programs are treated as static---their executions are pre-defined, are usually used once at some beginning/end-point (e.g. to produce a goal state for some RL algorithm/planner) and do not extend over time or with interactions. 3/6

this work uses programs that are more _functional_, involving control flows, conditionals, as well as nested subtasks. agents therefore perceive the environment and interact with it following the control flows---which is far more natural than previous RL/language setups. 4/6

this should allow better handling of temporal conditions and lifelong learning guided by program synthesis. moreover, lots of dependencies/signals in natural language that couldn't be captured by static goal-based programs, now have a better chance of being realised! 5/6

there's still a lot left to do on the language side here (e.g., this has rule-based program interpreters and doesn't do the language->program pipeline), but i'm pretty excited about this direction and future work that does! 6/6