We asked, you answered — our State of AI Agents Report is here! 🤖✨

We surveyed 1300+ industry professionals, from developers to business leaders, on how they're using AI agents today — and the results are in.

What are the top use cases for agents? The biggest challenges when building agents? And who's finding success after deploying their agents to production?

Read the full report ➡️ langchain.com/stateofaiagents

Here's 5 key insights in the thread below 🧵👇 1⃣ Agent adoption is a coin toss, but nearly everyone has plans for it.

About 50% of respondents have agents in production, with mid-sized companies leading the charge. That number is poised to grow, with 78% planning to implement AI agents soon.

⛴️ WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models

WebVoyager is a new kind of web-browsing agent, developed by Hongliang He, @wyu_nd, et. al.

Powered by large multi-modal models, like GPT-4V, it uses browser screenshots to conduct research, analyze images, and perform other tasks.

Older text-based web-browsing agents often fail to handle interactive web elements. Naive vision-based methods can struggle to use tools effectively.

WebVoyager uses “Set-of-mark” prompting to overlay the DOM with labeled bounding boxes and provide better guidance for the agent.

Check out the tutorial on how to build WebVoyager here: 2/ To jump straight to the code, check out the links below.

Python Code: 
WebVoyager Paper: 
Set-of-Mark Paper: github.com/langchain-ai/l…
arxiv.org/abs/2401.13919
arxiv.org/abs/2310.11441

⚙️ Agents are the “killer” LLM app, but building and evaluating agents is hard.

A huge part of agents is tool use, but there aren't enough open-source tool use benchmarks out there.

Today, we are excited to release four new test environments for benchmarking LLMs’ ability to effectively use tools.

📖

🧵 Below are some of our preliminary resultsblog.langchain.dev/benchmarking-a… 2/ Task 1: Typewriter (1 tool)

Agent has 1 tool (a typewriter). It has to type the provided word.

🔗langchain-ai.github.io/langchain-benc…

⭐️ Prompt Trends + Highlights ⭐️

We recently launched the LangChain Hub to support prompt sharing + workshopping.

We collected hundreds of prompts across many use-cases.

Here, we distill major themes and highlight interesting examples.

Blog:
blog.langchain.dev/the-prompt-lan…
Reasoning 🧠

Simple instructions ("think step by step") can improve many reasoning tasks.

Great thread from @_jasonwei w/ trade-offs:

Recent @GoogleDeepMind work (img below) shows accuracy across many such instructions:

https://x.com/_jasonwei/status/1712551642275655770?s=20

arxiv.org/abs/2309.03409

🏓Introduction LangServe

The best way to deploy your LangChains

📤Input/Output schema
📃/docs endpoint
🔠invoke/batch/stream endpoints
🎏/stream_log endpoint for streaming intermediate steps
🛠️LangSmith Integration

Used to power ChatLangChain and WebLangChain

Blog post and 🧵



Github Repo for the package:

We cover a lot of the motivation and features in a blog post here:

We'll pull out a lot of the most important points into a thread heregithub.com/langchain-ai/l…
blog.langchain.dev/introducing-la…

🚀Re-launching Chat LangChain

To help navigate the many features of 🦜🔗, we asked the amazing @mollycantillon to revamp the Chat LangChain chatbot.

Read about how she used LCEL, indexed our docs, deployed with FastAPI, ran evals, and more:

Highlights👇blog.langchain.dev/building-chat-… Ingestion

At a high level, the ingestion pipeline looks like this:
- Use document loaders to scrape the Python docs and API reference
- Chunk
- Using Indexing API to sync latest docs <> vecstore
- Use Github Actions to run ingestion daily

Weekend Reads

our favorites from this week

🧵 Thorough (and fun and well-written) overview of GenAI space by David Kypuros, @bobbyjohnstx, and Jason Nagin at @RedHat

demos, overview of key players, code–it’s got it all!

medium.com/@davidkypuros/…

🎡 Introducing LangChain Hub 🦜🔗

A place to publish, discover, and try out prompts

We’re particularly excited about a centralized hub’s promise to enable:
-Encoding of expertise
-Discoverability of prompts for a variety of models
-Inspectability
-Cross-team collaboration

🧵 Check it out here:

Read more about the motivation and future direction in our blog post here:

What are some of the motivations for the hub?

👇smith.langchain.com/hub
blog.langchain.dev/langchain-prom…

theres been a lot of excitement around fine-tuning recently, both in open source and with OpenAI's API

Here’s a list of some of our favorite resources, use-cases, and experiments on the topic over the last ~week

🧵 🗣️ Fine-tuning in your voice

No one wants their apps to feel generic or bot-like! Some resources on how to make them feel more like us!

Blog:

Webinar: blog.langchain.dev/chat-loaders-f…

🦜🛠️ Monitoring in LangSmith 📈

Launching today! Easily track analytics on your project over time

👍 feedback
💸 usage (chains, agents, LLMs, tokens)
⏲️ latency
🚨 errors
💬 time to first token

👇

👍/👎 Feedback Charts

Capturing feedback is incredibly important to get a sense of how your application is doing

You can now track this feedback over time, allowing you to have confidence that your users having the best possible interactions with your application

🎙️💬 Fine-tune with LangChain's ChatLoaders 🚀

1/ Want to make ChatGPT respond "in your own voice"? This week, we’ve added ChatLoaders to LangChain, making it easier to fine-tune models to your unique writing style! 2/ ChatLoaders make it easy to load your conversational data from popular platforms as chat messages. Use them for:
- Chat bots that “get” your unique speaking style
- Chatting reliably in a target language
-Customer communication in your brand's voice

New in 🦜🔗 Python:

🌌 @ainetwork_ai agent toolkit
🐻‍❄️ @DataPolars data loader
🚿 @AzureML online endpoint deployment
🪐 @epsilla_inc vector store

a 🧵: 🌌 @ainetwork_ai agent toolkit

Enable an agent to to transfer AINetwork tokens, read and write values, create apps, and more using the AINetworkToolkit by GH user klae01!

Docs: python.langchain.com/docs/integrati…

📰you can now subscribe to our blog!

in the last 2 weeks alone, we've had awesome posts about leveraging typescript, fine-tuning apps, RAG, Q&A over csv's, agents, and more (🧵)

if you don't want miss any future ones...

blog.langchain.dev @MultiON_AI bringing the power of agents to the web

blog.langchain.dev/multion-x-lang…

Happy Monday 🌞 Here's what we've added to 🦜🔗 over the weekend:

✅ Pydantic v2 compatibility
🔢 Ernie model embeddings
🚿 Streaming support for text-generation-webui
👆 @SharePoint document loader ✅ Pydantic v2 compatibility

https://twitter.com/LangChainAI/status/1693337392835645627?s=20

🗃️ Embeddings Cache 🗃️

This is one we've received a lot of requests for — caching embeddings to prevent redundant computation.

With the new ``CacheBackedEmbeddings`` you can wrap any existing embedding model, combine it with a storage mechanism, and voila!

Let's take a look: Store embeddings in memory

The InMemoryStore is an easy, setup-free storage mechanism to use for testing and prototyping.

We just set our embedding model (OpenAIEmbeddings), set our cache store (InMemoryStore), and watch as our embedding time goes from 160 ms -> 2 ms ⚡️

The latest and greatest in 🦜🔗:

🧠 @neural_internet Bittensor LLM
⚡ DashVector store
🤖 @zep_ai vector store
♻️ @elastic search self-querying retriever

a 🧵: 🧠 @neural_internet Bittensor LLM

Run models on the Bittensor protocol using this new interface from Neural Internet.

s/o to GH user Kunj-2206 for the integration!

Docs: https://t.co/yMAXfahpc1python.langchain.com/docs/integrati…

🛟Introducing Fallbacks in 🦜🔗

It’s not uncommon to encounter issues with LLM API's. In production, you need to gracefully handle such issues.

We’ve introduced Fallbacks to the LangChain Expression Language (LCEL) to help with just that.

Available in 🦜🔗 Python and JS! a 🧵: 🙅Handling API Errors

A request to an LLM API can fail for a variety of reasons - the API could be down, you could have hit rate limits, any number of things. Here’s how we can handle this with fallbacks:

The latest in 🦜🔗:

We've got 4 new model integrations for you!

⏹️ @anyscalecompute chat models
🔢 BAAI general embedding models
🦙 Ollama LLMs
🌌 Nebula LLM by @symbldotai

a 🧵: ⏹️ @anyscalecompute chat models

Anyscale Endpoints is a fast and scalable API to integrate OSS LLMs into your app.

With the new chat model integration by GH user oshuasundance-swca, you can now use it when running models like llama-2!

Docs: https://t.co/rrep0nnKqtpython.langchain.com/docs/integrati…

New day, new 🦜🔗 release!

Comes with:

🔍 ScaNN vector store
🗞️ Newspaper document loader
🛜 RSS document loader
🤹 EdenAICo LLM

a 🧵: 🔍 ScaNN vector store

ScaNN (Scalable Nearest Neighbors) is a method for efficient vector similarity search at scale developed at Google Research.

Now implemented as a vector store in 🦜🔗 thanks to GH user arron2003!

Docs: https://t.co/lEFbgzV5wZpython.langchain.com/docs/integrati…

The latest in 🦜🔗:

🧙Amazon Sagemaker Experiments callbacks
🪷 Huawei OBS document loader
🔀 Concurrent document loader
🗣️ Azure ML Online Endpoint chat model support
🎶 Run OpenAI Whisper locally

A 🧵: 🧙Amazon Sagemaker Experiments (SME)

SME lets you organize, track, compare and evaluate ML experiments and model versions.

Use the new callbacks to track and log prompts and LLM hyperparameters into SME.

s/o to GH user tesfagabir, mohjaz, sz640!

Docs: https://t.co/kBi2Zk7HZrpython.langchain.com/docs/integrati…

New in 🦜🔗:

🔎 @meilisearch vector store
🐙 Expanded @github toolkit
🌸 Expanded support for models run on Petals
🎁 @Dropbox document loader

a 🧵: 🔎 @meilisearch vector store

Meilisearch is a lightning fast open source search engine that also supports vector storage, now available through 🦜🔗!

Docs: https://t.co/F7MWPcPv2Vpython.langchain.com/docs/integrati…