CS PhD student @UofT | Research Intern @RealityLabs | I study interactions between human and AI | Prev: @GoogleAI @AdobeResearch | 🇹🇼🇺🇸 in 🇨🇦
Feb 27, 2023 • 13 tweets • 6 min read
#LLMs are powerful, but can they make existing GUIs interactable with language? Last summer at @GoogleAI, we found that LLMs can perform diverse language-based mobile UI tasks using few-shot prompting. Exciting implications for future interaction design! #chi2023 Thread 🧵
🧠 Key Takeaway: Using LLMs, designers/researchers can quickly implement and test *various* language-based UI interactions. In contrast, traditional ML pipelines require expensive data collection and model training for a *single* interaction capability.