Akshay 🚀 Profile picture
Feb 15 10 tweets 3 min read Read on X
Let's fine-tune DeepSeek-R1 (distilled Llama) 100% locally:
Before we begin, here’s what we’ll be doing:

We’ll fine-tune our private and locally running DeepSeek-R1 (a distilled Llama variant).

Tech stack:

- @UnslothAI for efficient fine-tuning.
- @Ollama to run it locally.

Let’s go! 🚀
1️⃣ Load the model

We begin by loading the Distilled Llama-8B model and the tokenizer for DeepSeek-R1 using Unsloth: Image
2️⃣ Define LoRA Config

We must use efficient techniques like LoRA to avoid fine-tuning the entire model's weights.

In this code, we utilize Unsloth's PEFT by specifying:

- The model
- LoRA low-rank (r)
- Modules for fine-tuning
- A few more parameters Image
3️⃣ Prepare dataset

Next, we use the Alpaca dataset to prepare a conversation dataset.

The conversation_extension parameter defines the number of user messages in a single conversation. Image
4️⃣ Define Trainer

Here, we create a Trainer object by specifying the training config like learning rate, model, tokenizer, and more.

Check this out👇 Image
5️⃣ Train

With that done, we initiate training. We notice a decreasing loss, which means the model is fine-tuning well.

Check this code and output👇 Image
6️⃣ Export to Ollama

Finally, we export the model to Ollama as follows.

Done! Image
We have fine-tuned DeepSeek (distilled Llama).

Now we can interact with it like any other model running on Ollama using:

- The CLI
- Ollama's Python package
- Ollama's LlamaIndex integration, etc. Image
That's a wrap!

And, if you enjoyed this breakdown:

Find me → @akshay_pachaar ✔️

Everyday, I share insights and tutorials around AI and Machine Learning.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Akshay 🚀

Akshay 🚀 Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @akshay_pachaar

Mar 13
Model Context Protocol (MCP), clearly explained:
MCP is like a USB-C port for your AI applications.

Just as USB-C offers a standardized way to connect devices to various accessories, MCP standardizes how your AI apps connect to different data sources and tools.

Let's dive in! 🚀
At its core, MCP follows a client-server architecture where a host application can connect to multiple servers.

Key components include:

- Host
- Client
- Server

Here's an overview before we dig deep 👇
Read 11 tweets
Mar 12
Let's build a RAG app using Google DeepMind's Gemma 3 (100% local):
Google just dropped a multilingual and multimodal open-source LLM.

Today, we're building a RAG app powered by @GoogleDeepMind's Gemma3.

Tech stack:

- @Llama_Index for orchestration
- @Ollama to locally serve Gemma 3
- @Streamlit for the UI

Let's go! 🚀
The architecture diagram presented below illustrates some of the key components & how they interact with each other!

It will be followed by detailed descriptions & code for each component: Image
Read 11 tweets
Mar 10
Let's build a Multimodal RAG app over complex webpages using DeepSeek's Janus-Pro (running locally):
The video depicts a multimodal RAG built with a SOTA tech stack.

We'll use:
- ColiVara's SOTA document understanding and retrieval to index webpages.
- @firecrawl_dev for reliable scrapping.
- @huggingface transformers to locally run DeepSeek Janus

Let's build it!
Here's an overview of our app:

1-2) Generate a PDF of web page screenshots with Firecrawl.
3) Index it on ColiVara for SOTA retrieval.
4-5) Query the ColiVara client to retrieve context.
6-7) Use DeepSeek Janus Pro as the LLM to generate a response.

Now let's jump into code!
Read 12 tweets
Mar 8
Object oriented programming in Python, clearly explained:
We break it down to 6 important concepts:

- Object 🚘
- Class 🏗️
- Inheritance 🧬
- Encapsulation 🔐
- Abstraction 🎭
- Polymorphism 🌀

Let's take them one-by-one... 🚀 Image
1️⃣ Object 🚘

Just look around, everything you see can be treated as an object.

For instance a Car, Dog, your Laptop are all objects.

An Object can be defined using 2 things:

- Properties: that describe an object
- Behaviour: the functions that an object can perform

...👇
Read 11 tweets
Mar 7
Bayes' Theorem clearly explained:
Bayes' Theorem is a cornerstone of probability theory!

It calculates the probability of an event, given that another event has occurred.

It's like updating your guess with fresh information!

Before we delve into the details, let's take a quick look at its formula: Image
Imagine you're trying to guess if it will rain today. ☔️ You start with a general belief based on the weather forecast (say, a 40% chance of rain).

This is your 'prior' probability: Image
Read 8 tweets
Mar 6
Let's build a corrective RAG (CRAG) agentic workflow, step-by-step (100% local):
Before we dive in, here's a quick demo of our agentic workflow!

Tech stack:

- @Llama_Index workflows for orchestration
- @Linkup_platform for deep web search
- @Cometml's Opik to trace and monitor
- @Qdrant_engine to self-host vectorDB

Let's go! 🚀
Here's an overview of what the app does:

— First search the docs with user query
— Evaluate if the retrieved context is relevant using LLM
— Only keep the relevant context
— Do web search if needed
— Aggregate the context & generate response

Now let's jump into code!
Read 13 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(