Thread by @LangChainAI

To say today's release (🦜🔗v0.0.124) is MASSIVE would be an understatement

This merges in a massive backlog of PRs including:

📄 8 *brand new* document loaders
🪝 2 new retrieval methods
🪘 1 new embedding class
🛟 2 LLM improvements

🧵

💪 BigQuery Document Loader

Thanks to lu-cashmoney you can now load in data that you have in BigQuery

With awesome walkthrough of extra functionality like adding to metadata and specifying which columns are fetched

Docs: python.langchain.com/en/latest/modu…

📓 Notion Document Loader

Thanks to @sbusso you can now load data directly from @NotionHQ's APIs

Includes awesome instructions for getting started

Docs: python.langchain.com/en/latest/modu…

🟩 WhatsApp Document Loader

Thanks to Moshe Malka you can now easily load in data from your WhatsApp chats

Docs: python.langchain.com/en/latest/modu…

🦆 DuckDB Document Loader

Thanks to @HauckTrent, you can now easily load in data from the popular @duckdb database

Again, contains awesome instructions for only loading certain columns and adding source to metadata

Docs: python.langchain.com/en/latest/modu…

🐼 Pandas DataFrame Loader

@deliprao makes his return!

Thanks to Delip, you can now load in data from a pandas dataframe

Docs: python.langchain.com/en/latest/modu…

🗺️ Sitemap Document Loader

Helpful for loading pages from a whole website, this will load a sitemap from a given URL, and then scrape and load all the pages in the sitemap, returning each page as a document.

🙏 @tim_asp

Docs: python.langchain.com/en/latest/modu…

🪗 Load multiple URLs concurrently

This isn't a new loader, but thanks to @tim_asp you can load multiple webpages concurrently - speeding up web loading greatly

Docs: python.langchain.com/en/latest/modu…

🟦 Azure blob storage file and container loaders

For everyone with data in Azure - have no fear! Thanks to @eavanvalkenburg you can now easily load it

Container: python.langchain.com/en/latest/modu…

File: python.langchain.com/en/latest/modu…

🟥 Redis Retriever

Peter Shi added a new method to the Redis VectorStore to get all documents above some similarity

We took the opportunity to write a custom Redis Retriever to show how to use this new functionality

PR: github.com/hwchase17/lang…

Docs: python.langchain.com/en/latest/modu…

🦙 Llama Index Retriever

Lots of people have been asking for a better @gpt_index integration - thanks to @jerryjliu0 we now have one!

You can easily use your Llama Index as a retriever

PR: github.com/hwchase17/lang…

🛤️ Better token tracking

In a big quality of life improvement, iocuydi added support for more detailed token tracking

Notebook: python.langchain.com/en/latest/modu…

🎏 Support for streaming in @AnthropicAI LLM

You can now stream responses if you are using the Anthropic LLM wrapper!

Docs: python.langchain.com/en/latest/modu…

🪘 Jina Embeddings

There's a new embedding provider!

You can now use Jina to create embeddings from your documents!

Docs: python.langchain.com/en/latest/modu…

There's a lot of other, smaller quality of life improvements - we're going to shout those out in a separate thread

What a weekend 😅