Bryan Wang Profile picture
CS PhD student @UofT | Research Intern @RealityLabs | I study interactions between human and AI | Prev: @GoogleAI @AdobeResearch | 🇹🇼🇺🇸 in 🇨🇦
Feb 27, 2023 13 tweets 6 min read
#LLMs are powerful, but can they make existing GUIs interactable with language? Last summer at @GoogleAI, we found that LLMs can perform diverse language-based mobile UI tasks using few-shot prompting. Exciting implications for future interaction design! #chi2023 Thread 🧵 Image 🧠 Key Takeaway: Using LLMs, designers/researchers can quickly implement and test *various* language-based UI interactions. In contrast, traditional ML pipelines require expensive data collection and model training for a *single* interaction capability.

Learn more about it👇