Thread by @jerryjliu0

There are a ton of techniques for LLM-based QA over your data, each for diff use cases.
- Top-k semantic search
- Summarization
- Temporal recency weighting
- Document comparisons
- etc.

We propose a simple “router” that can unify every technique under a *single* interface 👇🧵

A “router” concept is at the core of an LLM agent/tool abstraction - given a set of tools, pick the best tool for the task.

Having a “router” (or even multiple) can be especially powerful for enhancing LLM retrieval + QA systems!

We added this abstraction into @gpt_index.

We’ve defined a “RouterQueryEngine” that can take as input a set of underlying query engines.

Each query engine could be defined over a specific view of your data + use case.

E.g. For top-k vector search, for summarization, for doc comparisons.

Each query engine can be treated as a “tool” with a text description attached to it.

You can think of the “RouterQueryEngine” as a zero-shot engine that can reason over different query engines.

During query-time, it can choose the right candidate query engine to execute.

Tutorials/Notebooks:

Ask q’s over specific docs, but also compare/contrast documents 🍎🍊

gpt-index.readthedocs.io/en/stable/guid…

Ask both summarization and semantic search q’s 💡

github.com/jerryjliu/llam…

We’ve written about this before.

Composing an index (🌲 tree index, vector index) is one way of defining a “router” to the rest of your data.

But we believe the RouterQueryEngine provides a more general abstraction that should be easier to use.

https://twitter.com/jerryjliu0/status/1647007095081009152?s=20

Last but not least, we put a tutorial on @GoogleColab so it’s easy to access - come check them out!

Unified query interface: colab.research.google.com/drive/1KH8XtRi…

Joint QA / Summarization: colab.research.google.com/drive/1Asq_obA…