Discover and read the best of Twitter Threads about #LLM4Mobile
Most recents (1)
#LLMs are powerful, but can they make existing GUIs interactable with language? Last summer at @GoogleAI, we found that LLMs can perform diverse language-based mobile UI tasks using few-shot prompting. Exciting implications for future interaction design! #chi2023 Thread 🧵 
🧠 Key Takeaway: Using LLMs, designers/researchers can quickly implement and test *various* language-based UI interactions. In contrast, traditional ML pipelines require expensive data collection and model training for a *single* interaction capability.
Learn more about it👇
Learn more about it👇
To adapt LLMs to mobile UIs, we designed prompting techniques and an algorithm to convert the view hierarchy data in Android to the HTML syntax, which is well-represented in LLMs’ training data.
