Discover and read the best of Twitter Threads about #ScienceQA
Most recents (1)
🔥Excited to release LLaMA-Adapter! With only 1.2M learnable parameters and 52K instruction data, LLaMA-Adapter turns a #LLaMA into an instruction-following model within ONE hour, delivering high-quality responses!
🚀Paper: arxiv.org/abs/2303.16199
🚀Code: github.com/ZrrSkywalker/L…
🚀Paper: arxiv.org/abs/2303.16199
🚀Code: github.com/ZrrSkywalker/L…
We adopt learnable adaption prompts and prepend them to the input text tokens at higher transformer layers. A zero-init attention mechanism with zero gating adaptively injects the new instructional cues into LLaMA, while effectively preserving its pre-trained knowledge.
With efficient training, LLaMA-Adapter generates high-quality responses, comparable to Alpaca with fully fine-tuned 7B parameters.
