Thread by @akshay_pachaar

Let's build a RAG app using MetaAI's Llama-3 (100% local):

Before we begin, take a look at what we're about to create!

Here's what you'll learn:

- @Ollama for locally serving a LLM (Llama-3)
- @Llama_Index for orchestration
- @Streamlit for building the UI
- @LightningAI for development & hosting

Let's go! 🚀

The architecture diagram presented below illustrates some of the key components & how they interact with each other!

It will be followed by detailed descriptions & code for each component:

1️⃣ & 2️⃣ : Loading the knowledge base

A knowledge base is a collection of relevant and up-to-date information that serves as a foundation for RAG. In our case it's the docs stored in a directory.

Here's how you can load it as document objects in LlamaIndex:

3️⃣ The embedding model

Embedding is a meaningful representation of text in form of numbers.

The embedding model is responsible for creating embeddings for the document chunks & user queries.

We are using @SnowflakeDB's `arctic-embed-m`, one of the best model in it's class.

4️⃣ Indexing & storing

Embeddings created by embedding model are stored in a vector store that offers fast retrieval and similarity search by creating an index over our data.

By default, LlamaIndex provides a in-memory vector store that’s great for quick experimentation.

5️⃣ Creating a prompt template

A custom prompt template is use to refine the response from LLM & include the context as well:

6️⃣ & 7️⃣ Setting up a query engine

The query engine takes a query string & use it to fetch relevant context and then sends them both as a prompt to the LLM to generate a final natural language response.

Here's how you set it up:

8️⃣ The Chat interface

We create a UI using Streamlit to provide a chat interface for our RAG application.

The code for this & all we discussed so far is shared in the next tweet!

Check this out👇

I used @LightningAI⚡️ Studio for developing this application!

You will find all the code & everything you need to run it! ✨

Clone a FREE studio now & take it for a spin...👇
lightning.ai/lightning-ai/s…

If you interested in:

- Python 🐍
- Machine Learning 🤖
- AI Engineering ⚙️

Find me → @akshay_pachaar ✔️
Everyday, I share tutorials on above topics!

Cheers! 🥂