Lightning AI ⚡️ Profile picture
The AI development platform - From idea to AI, Lightning fast ⚡️. Creators of AI Studio, PyTorch Lightning... Get help: https://t.co/a69wnEARV9
Jan 19, 2023 4 tweets 1 min read
Train a 20-billion parameter GPT model for text prediction on 3 GPU nodes with Lightning. 🤯

The entire training process is contained in a simple script that you can scan, read, and understand in just a few seconds.✅

🧵(1/4) Image You can customize any part of this process: the dataset, model, hyperparameters, and training strategy.

Also, you can easily swap out which 🛠️hardware you’re using. With a single flag, you can choose to run on 3 nodes (as in this example), or higher to fit your model.

🧵(2/4)