Share this page!

Harrison Chase Profile picture
Twitter logo
Jan 6 • 5 tweets • 4 min read
🗣️ One pain point we heard with @LangChainAI was trying to get agents to work in a conversational setting

With a lot of help from @brunotorious, we now have an example of how to achieve exactly that

Lots of examples/resources below (including 2 agents talking to eachother!)

🧵
Conversation Agent example: langchain.readthedocs.io/en/latest/modu…

Google Collab Notebook with this example (as well other related agent improvements) colab.research.google.com/drive/1UsCLcPy…

Tweet thread all the new agent improvments
Additional awesome stuff from @brunotorious:

An example of using this to have two agents talk to each other: github.com/bborn/langchai…

I believe these new conversational agents (or a variant of them) are being used in howdoi.ai already
Getting agents to work in a conversational setting is hard, and the example we have makes progress towards that but isn't perfect

Given the difficulty, this was also a big group effort, with a thread in the discord (discord.gg/6adMQxSpJS) getting VERY long
So shoutout to not only @brunotorious but also @andrewgleave (tool direct returning) @nfcampos @Yoooongtae (example for tool priority) and everyone else!

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Harrison Chase

Harrison Chase Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @hwchase17

Harrison Chase Profile picture
Dec 26, 2022
⚠️Evaluating language models can be tricky⚠️

One new idea of how to do this is to have language models evaluate themselves

@LangChainAI has now added some examples for doing so. Mostly focused on question/answering, but expanding soon (hopefully with community help!)

Thread 🧵
If you are using language models for text generation, you've probably realized that traditional metrics like accuracy and f1 score don't work that well

The common way to evaluate text generation tasks is manually (with human experts), which is slow and not scalable
New research, like the below by @EthanJPerez, @karinanguyen_ et all at @AnthropicAI, propose ways to generate evaluations with LMs themselves

The capabilities in @LangChainAI are not nearly as detailed as those examples, but are a step in that direction
Read 12 tweets
Harrison Chase Profile picture
Dec 8, 2022
✨Data Augmented Generation✨

langchain.readthedocs.io/en/latest/expl…

🌻Summarization over a specific text
❓Question Answering over private data
🖨️WebGPT

All these use cases involve chains combining LLMs with external sources of data.

🧵Here's a break down of some methods for how to do so:
Some background:

⭐️LLMs are great at generating text generically, but they do that based on the data they were trained on

It is very common to want to generate text based on other sources of data besides what the model was trained on.
Common use cases include:

🌻Summarization
❓Question Answering
ℹ️Question Answering with Sources

As of @LangChainAI version 0.0.32, there are now wrappers for all of these use cases, with multiple different underlying types of chains.

Notebook links below 👇
Read 13 tweets
Harrison Chase Profile picture
Dec 6, 2022
A 🧵on results of a little investigation I did over the past week

❓How does text-davinci-003 do on agent-like tasks

TLDR: Displays superior understanding and ability to take multi-step actions towards original goal

h/t to @OfirPress and @momusbah for help/feedback with this
✨text-davinci-003 came out a week ago

Lots of people (like @blennon_ below) provided some great analysis of how it compared to `text-davinci-002`

This is my contribution to that, focused on agent-like tasks

What are agent-like tasks?

I focus mainly on exploring situations where the LLM is used as agent that has access to some tools, and needs to answer a question using the tools it has available. For example:

@OfirPress Self Ask with Seach
@ShunyuYao12 ReAct with Wikipedia
Read 21 tweets
Harrison Chase Profile picture
Dec 5, 2022
Some really cool stuff added by @subby_tech (👏👏👏) to @LangChainAI over the weekend:

⚕️APIChain

A general framework for interacting with an API in natural language

🧵See below for a more in depth explanation + examples
At a high level, the flow is:

1⃣ Format a prompt with API docs + a question
2⃣ Have an LLM generate API query to run to get an answer
3⃣ Run said API query
4⃣ Have LLM interpret API response and answer original question in natural language
Note that the LLM is doing in context learning (via the API docs) to figure out how to call the API

For popular APIs, the LLM may(?) be able to generate the correct API call without that context... but this methodology allows it to work on smaller, newer, or private APIs
Read 10 tweets
Harrison Chase Profile picture
Dec 5, 2022
With lots of "unofficial" ChatGPT APIs popping up (most based on @danielgross's code), there's been a lot of asks to hook this into LangChain.

Here's how to do so:
1. Step up an unofficial ChatGPT API

Here's one example from @taranjeetio: github.com/taranjeet/chat…

Here's another example from @kylejohnmorris: github.com/kylejmorris/ch…
2. Write a CustomLLM @LangChainAI wrapper

Here's an example wrapper for @taranjeetio's API implementation: gist.github.com/hwchase17/af22…

Although since most implementations are based on @danielgross's implementation, this wrapper should work for most
Read 4 tweets
Harrison Chase Profile picture
Dec 1, 2022
A new chain we introduced in @LangChainAI (with a lot of help from @johnjnay 👏👏):

❓Question Answering with Sources❓

github.com/hwchase17/lang…

This takes in a question and a list of documents, and uses those documents to answer that question, citing its sources
There are several cool things about this chain

1⃣ It's a general chain that can be applied to lots of problems

2⃣ It runs over each document individually, and then combines the answers, meaning it can work on longer documents

3⃣ Citing sources is an extremely important UX!
One potential use case (from @johnjnay at @CodeXStanford )

LLM understanding of legal reasoning / legal language.

Given that legal documents are long and complicated, they are developing approaches for LLMs to recursively analyze them in sequential chains of LLM interactions.
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

Send Email!

:(