Can AI agents design better memory mechanisms for themselves?
Introducing Learning to Continually Learn via Meta-learning Memory Designs. A meta agent automatically designs memory mechanisms, including what info to store, how to retrieve it, and how to update it, enabling agentic systems to continually learn across diverse domains. Led by @yimingxiong_ with @shengranhu 🧵👇 1/ Researchers have devoted considerable manual effort to designing memory mechanisms to improve continual learning in agents. But the history of machine learning shows that handcrafted AI components will be replaced by learned, more effective ones.
We introduce ALMA (Automated meta-Learning of Memory designs for Agentic systems), where a meta agent searches in a Darwin-complete search space (code) with an open-ended algorithm, growing an archive of ever-better memory designs.🧠🔬📈✨ 2/

After a conversation with @joelbot3000 & @kenneth0stanley we concluded there’s an important AI safety point deserving broader discussion: In short, any mandatory “nutrition label” for foundation models needs to go well beyond just disclosures on training data... 🧵1/ Digital assistants will help & befriend us, but we should know if they have ulterior motives (eg to sell us products, influence us politically, or maximize engagement). A mandated "nutrition label for AI" should cover all the relevant ingredients. 2/

Introducing Video PreTraining (VPT): it learns complex behaviors by watching (pretraining on) vast amounts of online videos. On Minecraft, it produces the first AI capable of crafting diamond tools, which takes humans over 20 minutes (24,000 actions) 🧵👇openai.com/blog/vpt Challenge: online videos are unlabeled (missing actions), so we first train an inverse dynamics model (IDM) on a small amount of contractor data, which predicts which action must have been taken between video frames. We then label online videos with the IDM and behavior clone.