Akshay 🚀 Profile picture
Feb 15 10 tweets 3 min read Read on X
Let's fine-tune DeepSeek-R1 (distilled Llama) 100% locally:
Before we begin, here’s what we’ll be doing:

We’ll fine-tune our private and locally running DeepSeek-R1 (a distilled Llama variant).

Tech stack:

- @UnslothAI for efficient fine-tuning.
- @Ollama to run it locally.

Let’s go! 🚀
1️⃣ Load the model

We begin by loading the Distilled Llama-8B model and the tokenizer for DeepSeek-R1 using Unsloth: Image
2️⃣ Define LoRA Config

We must use efficient techniques like LoRA to avoid fine-tuning the entire model's weights.

In this code, we utilize Unsloth's PEFT by specifying:

- The model
- LoRA low-rank (r)
- Modules for fine-tuning
- A few more parameters Image
3️⃣ Prepare dataset

Next, we use the Alpaca dataset to prepare a conversation dataset.

The conversation_extension parameter defines the number of user messages in a single conversation. Image
4️⃣ Define Trainer

Here, we create a Trainer object by specifying the training config like learning rate, model, tokenizer, and more.

Check this out👇 Image
5️⃣ Train

With that done, we initiate training. We notice a decreasing loss, which means the model is fine-tuning well.

Check this code and output👇 Image
6️⃣ Export to Ollama

Finally, we export the model to Ollama as follows.

Done! Image
We have fine-tuned DeepSeek (distilled Llama).

Now we can interact with it like any other model running on Ollama using:

- The CLI
- Ollama's Python package
- Ollama's LlamaIndex integration, etc. Image
That's a wrap!

And, if you enjoyed this breakdown:

Find me → @akshay_pachaar ✔️

Everyday, I share insights and tutorials around AI and Machine Learning.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Akshay 🚀

Akshay 🚀 Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @akshay_pachaar

Jun 28
I have tested 100+ MCP servers in the last 3 months!

Here are 6 must-use MCP servers for all developers (open-source):
1️⃣ Graphiti MCP server

Agents forget everything after each task.

Graphiti MCP server lets Agents build and query temporally-aware knowledge graphs, which act as an Agent's memory!

Check this👇
2️⃣ Opik MCP server

This enables traceability into AI Agents and lets you monitor your LLM applications, by @Cometml.

Tools include:
- Creating projects
- Enable tracing
- Getting tracing stats

Check this demo👇
Read 9 tweets
Jun 25
Let's generate our own LLM fine-tuning dataset (100% local):
Before we begin, here's what we're doing today!

We'll cover:
- What is instruction fine-tuning?
- Why is it important for LLMs?

Finally, we'll create our own instruction fine-tuning dataset.

Let's dive in!
Once an LLM has been pre-trained, it simply continues the sentence as if it is one long text in a book or an article.

For instance, check this to understand how a pre-trained LLM behaves when prompted 👇 Image
Read 12 tweets
Jun 22
Let's build a real-time Voice RAG Agent, step-by-step:
Before we begin, here's a quick demo of what we're building

Tech stack:

- @Cartesia_AI for SOTA text-to-speech
- @AssemblyAI for speech-to-text
- @LlamaIndex to power RAG
- @livekit for orchestration

Let's go! 🚀
Here's an overview of what the app does:

1. Listens to real-time audio
2. Transcribes it via AssemblyAI
3. Uses your docs (via LlamaIndex) to craft an answer
4. Speaks that answer back with Cartesia

Now let's jump into code!
Read 11 tweets
Jun 21
Let's build an MCP-powered audio analysis toolkit:
Before we dive in, here's a demo of what we're building!

Tech stack:
- @AssemblyAI for transcription and audio analysis.
- Claude Desktop as the MCP host.
- @streamlit for the UI

Let's build it!
Here's our workflow:

- User's audio input is sent to AssemblyAI via a local MCP server.
- AssemblyAI transcribes it while providing the summary, speaker labels, sentiment, and topics.
- Post-transcription, the user can also chat with audio.

Let's implement this!
Read 11 tweets
Jun 19
AI agents can finally talk to your frontend!

The AG-UI Protocol bridges the critical gap between AI agents and frontend apps, making human-agent collaboration seamless.

MCP: Agents to tools
A2A: Agents to agents
AG-UI: Agents to users

100% open-source.
Here's the official GitHub repo for @CopilotKit's AG-UI:

(don't forget to star 🌟)github.com/ag-ui-protocol…
Here's a really good illustration of how it works!

Key features:

🤝 Works with LangGraph, LlamaIndex, Agno, CrewAI & AG2
🎯 Event-based protocol with 16 standard event types
💬 Real-time agentic chat with streaming
🧑‍💻 Human-in-the-loop collaboration
💬 ChatUI & Generative UI
Read 4 tweets
Jun 16
Top 4 open-source LLM finetuning libraries!

From single-GPU “click-to-tune” notebooks to trillion-param clusters, these four libraries cover every LLM finetuning scenario.

Understand which one to use, & when...👇 Image
1️⃣ Unsloth

Unsloth makes fine-tuning easy and fast, turning a mid-range GPU into a powerhouse with a simple Colab or Kaggle notebook.

Perfect for hackers and small teams using 12–24 GB GPUs needing quick LoRA experiments without DeepSpeed configs or clusters

Check this out👇
github.com/unslothai/unsl…
2️⃣ Axolotl

Axolotl keeps your entire pipeline in one YAML file—write once, reuse from data prep to serving.

Perfect for teams that crave reproducibility and want to toggle advanced recipes by flipping a YAML switch.

Check this out👇
github.com/axolotl-ai-clo…
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(