The AI development platform - From idea to AI, Lightning fast ⚡️. Creators of AI Studio, PyTorch Lightning... Get help: https://t.co/a69wnEARV9
Jan 19, 2023 • 4 tweets • 1 min read
Train a 20-billion parameter GPT model for text prediction on 3 GPU nodes with Lightning. 🤯
The entire training process is contained in a simple script that you can scan, read, and understand in just a few seconds.✅
🧵(1/4)
You can customize any part of this process: the dataset, model, hyperparameters, and training strategy.
Also, you can easily swap out which 🛠️hardware you’re using. With a single flag, you can choose to run on 3 nodes (as in this example), or higher to fit your model.