Let's build a context engineering workflow, step by step:
Today, we'll build a multi-agent research assistant using context engineering principles.
Tech stack:
- @tensorlake to get RAG-ready data from complex docs
- @zep_ai for memory
- @firecrawl_dev for web search
- @milvusio for vector DB
- @crewAIInc for orchestration
Let's go!
First, what is context engineering (CE)?
LLMs aren't mind readers. They can only work with what you give them.
Prompt engineering primarily focuses on “magic words” with an expectation of getting a better response.
CE involves creating dynamic systems that offer:
- The right info
- The right tools
- In the right format
This ensures the LLM can effectively complete the task.
I have been fine-tuning LLMs for over two years now!
Here are the top 5 LLM fine-tuning techniques, explained visually:
Traditional fine‑tuning is impractical for LLMs (billions of params; 100s GB).
Since this kind of computing isn't accessible to everyone, parameter-efficient finetuning (PEFT) is extensively used.
Today, we’ll cover the top 5 PEFT techniques, step by step.
Some background!
LLM weights are matrices of numbers adjusted during finetuning.
Most PEFT techniques involve finding a lower-rank adaptation of these matrices, which is a smaller-dimensional matrix that can still represent the information stored in the original.