Let's build a RAG app using MetaAI's Llama-3 (100% local):
Before we begin, take a look at what we're about to create!
Here's what you'll learn:
- @Ollama for locally serving a LLM (Llama-3)
- @Llama_Index for orchestration
- @Streamlit for building the UI
- @LightningAI for development & hosting
Let's go! 🚀
Here's what you'll learn:
- @Ollama for locally serving a LLM (Llama-3)
- @Llama_Index for orchestration
- @Streamlit for building the UI
- @LightningAI for development & hosting
Let's go! 🚀
The architecture diagram presented below illustrates some of the key components & how they interact with each other!
It will be followed by detailed descriptions & code for each component:
It will be followed by detailed descriptions & code for each component:
1️⃣ & 2️⃣ : Loading the knowledge base
A knowledge base is a collection of relevant and up-to-date information that serves as a foundation for RAG. In our case it's the docs stored in a directory.
Here's how you can load it as document objects in LlamaIndex:
A knowledge base is a collection of relevant and up-to-date information that serves as a foundation for RAG. In our case it's the docs stored in a directory.
Here's how you can load it as document objects in LlamaIndex:
3️⃣ The embedding model
Embedding is a meaningful representation of text in form of numbers.
The embedding model is responsible for creating embeddings for the document chunks & user queries.
We are using @SnowflakeDB's `arctic-embed-m`, one of the best model in it's class.
Embedding is a meaningful representation of text in form of numbers.
The embedding model is responsible for creating embeddings for the document chunks & user queries.
We are using @SnowflakeDB's `arctic-embed-m`, one of the best model in it's class.
4️⃣ Indexing & storing
Embeddings created by embedding model are stored in a vector store that offers fast retrieval and similarity search by creating an index over our data.
By default, LlamaIndex provides a in-memory vector store that’s great for quick experimentation.
Embeddings created by embedding model are stored in a vector store that offers fast retrieval and similarity search by creating an index over our data.
By default, LlamaIndex provides a in-memory vector store that’s great for quick experimentation.
5️⃣ Creating a prompt template
A custom prompt template is use to refine the response from LLM & include the context as well:
A custom prompt template is use to refine the response from LLM & include the context as well:
6️⃣ & 7️⃣ Setting up a query engine
The query engine takes a query string & use it to fetch relevant context and then sends them both as a prompt to the LLM to generate a final natural language response.
Here's how you set it up:
The query engine takes a query string & use it to fetch relevant context and then sends them both as a prompt to the LLM to generate a final natural language response.
Here's how you set it up:
8️⃣ The Chat interface
We create a UI using Streamlit to provide a chat interface for our RAG application.
The code for this & all we discussed so far is shared in the next tweet!
Check this out👇
We create a UI using Streamlit to provide a chat interface for our RAG application.
The code for this & all we discussed so far is shared in the next tweet!
Check this out👇
I used @LightningAI⚡️ Studio for developing this application!
You will find all the code & everything you need to run it! ✨
Clone a FREE studio now & take it for a spin...👇
lightning.ai/lightning-ai/s…
You will find all the code & everything you need to run it! ✨
Clone a FREE studio now & take it for a spin...👇
lightning.ai/lightning-ai/s…
If you interested in:
- Python 🐍
- Machine Learning 🤖
- AI Engineering ⚙️
Find me → @akshay_pachaar ✔️
Everyday, I share tutorials on above topics!
Cheers! 🥂
- Python 🐍
- Machine Learning 🤖
- AI Engineering ⚙️
Find me → @akshay_pachaar ✔️
Everyday, I share tutorials on above topics!
Cheers! 🥂
• • •
Missing some Tweet in this thread? You can try to
force a refresh
