Pan Lu Profile picture
Postdoc @Stanford | PhD @CS_UCLA @uclanlp | Amazon/Bloomberg/Qualcomm Fellows | Ex @Tsinghua_Uni @Microsoft @allen_ai | ML/NLP: Math Reasoning, AI4Science, LLMs
Jul 8 8 tweets 4 min read
#TextGrad now features multimodal reasoning!

🔬 ScienceQA (multimodal scientific reasoning)
- Error rate drops by 20%, achieving the highest zero-shot performance we know of.

📊 MathVista (multimodal math reasoning)
- Boosting the score from 63.8% to 66.1% on GPT-4o!

Explore more:
💻 Code:
📄 Doc:
🌐 Project: 🧵github.com/zou-group/text…
github.com/zou-group/text…
textgrad.comImage 🧵1/n Here's an example from ScienceQA: . TextGrad reviews the multimodal question and provides evaluations to “propagate” the initially incorrect answer to the correct one. scienceqa.github.io
Image
Jun 12, 2023 7 tweets 4 min read
🎉Exciting news: LLaMA-Adapter is now fully unlocked! 🧵6

1⃣ As a general-purpose #multimodal foundation model, it integrates various inputs like images, audio, text, video, and 3D point clouds, while providing image, text-based, and detection outputs. It uniquely accepts the… twitter.com/i/web/status/1… Image 🧵1/6 Experience the magic of LLaMA-Adapter! Transforming real-world inputs like text, images, videos, audio, and 3D point clouds into engaging text. The reality you know, reimagined through AI.
🖼️📽️🔉🌐➕📝 ➡️➡️🦙➡️➡️ 📝 ImageImageImageImage
Mar 29, 2023 5 tweets 3 min read
🔥Excited to release LLaMA-Adapter! With only 1.2M learnable parameters and 52K instruction data, LLaMA-Adapter turns a #LLaMA into an instruction-following model within ONE hour, delivering high-quality responses!

🚀Paper: arxiv.org/abs/2303.16199
🚀Code: github.com/ZrrSkywalker/L… We adopt learnable adaption prompts and prepend them to the input text tokens at higher transformer layers. A zero-init attention mechanism with zero gating adaptively injects the new instructional cues into LLaMA, while effectively preserving its pre-trained knowledge.
Nov 2, 2022 7 tweets 6 min read
🚨Struggling to select examples for GPT-3? Try our PromptPG, the first work that applies RL to select in-context examples for GPT-3! PromptPG achieves a gain of 5.31% on TabMWP, a new dataset of tabular math word problems! Check out data and codes:👇
promptpg.github.io 🧵1/7 Resources are all available now!
📄Paper: arxiv.org/abs/2209.14610
🔖Project page: promptpg.github.io
💻Code: github.com/lupantech/Prom…
🧐Explore: promptpg.github.io/explore.html
🪜Leaderboard: promptpg.github.io/leaderboard.ht…
🔗Data: github.com/lupantech/Prom… 🧵2/7