Wanna build the next ChatGPT?
Here's a 6 month learning plan for AI large language models (LLMs) from scratch.
(thread)
Here's a 6 month learning plan for AI large language models (LLMs) from scratch.
(thread)
1. Familiarize yourself with basic machine learning concepts and Python programming.
Books:
1.1. Introduction to Machine Learning with Python by Andreas Müller and Sarah Guido
1.2. Python Machine Learning by @rasbt
Books:
1.1. Introduction to Machine Learning with Python by Andreas Müller and Sarah Guido
1.2. Python Machine Learning by @rasbt
2. Learn about natural language processing (NLP) and its applications.
Books:
2.1. Speech and Language Processing by Daniel Jurafsky and James H. Martin
2.2. Natural Language Processing with Python by Steven Bird, Ewan Klein, and Edward Loper
Books:
2.1. Speech and Language Processing by Daniel Jurafsky and James H. Martin
2.2. Natural Language Processing with Python by Steven Bird, Ewan Klein, and Edward Loper
3. Learn about the specific architecture and inner workings of language models, such as the transformer architecture used in GPT3.
Paper:
Attention Is All You Need by Vaswani et al.
proceedings.neurips.cc/paper/2017/fil…
Paper:
Attention Is All You Need by Vaswani et al.
proceedings.neurips.cc/paper/2017/fil…
4. Train your own small-scale language model using an open-source library such as TensorFlow or PyTorch.
Study the @TensorFlow and @PyTorch documentations, as well as tutorials and courses on websites like @kaggle and @coursera.
Study the @TensorFlow and @PyTorch documentations, as well as tutorials and courses on websites like @kaggle and @coursera.
5. Learn about fine-tuning pre-trained language models and apply this knowledge to a specific task of your choice.
The @huggingface documentation on fine-tuning will be very helpful.
huggingface.co/docs/transform…
The @huggingface documentation on fine-tuning will be very helpful.
huggingface.co/docs/transform…
6. Explore the latest research in language models and their applications.
The papers from @NeurIPSConf (nips.cc) and @TheICLR (iclr.cc) are a good starting point.
The papers from @NeurIPSConf (nips.cc) and @TheICLR (iclr.cc) are a good starting point.
7. The best part about this is that everything is basically free. Happy learning and good luck!
If you enjoyed this thread, you'll probably love the next ones! Like, retweet and follow me for more.
#machinelearning #ai #gpt3 #chatgpt #llms #nlp #python #tensorflow @OpenAI
If you enjoyed this thread, you'll probably love the next ones! Like, retweet and follow me for more.
#machinelearning #ai #gpt3 #chatgpt #llms #nlp #python #tensorflow @OpenAI
• • •
Missing some Tweet in this thread? You can try to
force a refresh
