Thread by @RLanceMartin

GPT-3.5 and LLaMA2 fine-tuning guides 🪄

Considering LLM fine-tuning? Here's two new CoLab guides for fine-tuning GPT-3.5 & LLaMA2 on your data using LangSmith for dataset management and eval. We also share our lessons learned in a blog post here:

blog.langchain.dev/using-langsmit…

... 1/ When to fine-tune? Fine-tuning is not advised for teaching an LLM new knowledge (see references from @OpenAI and others in our blog post). It's best for tasks (e.g., extraction) focused on "form, not facts":
anyscale.com/blog/fine-tuni…

... 2/ With this in mind, we fine-tuned LLaMA-7b-chat & GPT-3.5-turbo for knowledge graph triple extraction (see details in blog post and CoLab). Notebooks here:
LLaMA CoLab:
GPT-3.5-turbo CoLab:

colab.research.google.com/drive/1tpywvzw…
colab.research.google.com/drive/1YCyDHPS…

https://twitter.com/RLanceMartin/status/1691880034058064365?s=20

... 3/ We used LangSmith for managing / cleaning the train & test set and for eval, using a GPT4 grader. All code is shared in CoLabs. Results comparing few-shot GPT4, GPT3.5 vs fine-tuning are shown below, with grades from 0 (worst) to 100% (best).

... 4/ Lesson 1: always consider approach like few-shot prompting or RAG before fine-tuning. Few-shot prompting of GPT4 scored better than any fine-tuning (w/ a small 1.5k instruction dataset / 7b base-model).

tidepool.so/2023/08/17/why…

https://twitter.com/karpathy/status/1655994367033884672

... 5/ Lesson 2: but, we find that fine-tuning a small (7b) base model can outperform a larger generalist (GPT-3.5) w/ few shot prompting, a result also shown recently by @anyscalecompute and others.
anyscale.com/blog/fine-tuni…

... 6/ Lesson 3: dataset collection, cleaning is often the most challenging part. We iterated through several public datasets. LangSmith automatically logs project generations w/ a queryable interface to select for and fix poor quality examples for fine-tuning.

... 7/ Lesson 4: eval is challenging. We used LangSmith to run eval and inspect generations. We found base models w/o fine-tuning were verbose / chatty, and in one case hallucinated Homer Simpson as the subject (vs fine-tuned LLMs extracted triples much closer to label format):

Overall, fine-tuning is a powerful tool but should be considered vs prompt eng / RAG. LangSmith can help w/ fine-tuning pain points (data capture / cleaning / eval) and works well w/ fine-tuning recipes (e.g., via @huggingface / @maximelabonne, @OpenAI).
mlabonne.github.io/blog/posts/Fin…