I just learned about an exciting development of (by @jeremyphoward), they just released an open source system that can train a 70b large language model on a regular desktop computer with two or more standard gaming GPUs (RTX 3090 or 4090).
The system combines FSDP (Fully Sharded Data Parallel) and QLoRA (Quantized Low-Rank Adaptation) techniques, allowing users to train large models on consumer-grade hardware.
It's the result of a collaboration between , Tim Dettmers (U Washington), and Hugging Face.
@jeremyphoward The goal is to make useful AI accessible to everyone, enabling people to create their own personalized models and have control over their AI systems. This project is a key foundation for achieving that goal.
3/n