Thread by @RLanceMartin

Extraction 📚➡️🗒️

Getting structured LLM output is hard! Part 3 of our initiative to improve @LangChainAI docs covers this w/ functions and parsers (see @GoogleColab ntbk). Thanks to @fpingham for improving the docs on this:

https://t.co/bMjFmCSZM3python.langchain.com/docs/use_cases…

1/ Getting LLMs to produce structured (e.g., JSON) output is challenge, often requiring tedious prompt eng:

https://twitter.com/goodside/status/1657396491676164096?s=20

2/ Functions (e.g., using OpenAI models) have been a great way to tackle this problem, as shown by the work of @jxnlco and others. LLM calls a function and returns output that follows a specified schema.
wandb.ai/jxnlco/functio…

3/ @LangChainAI support function calling for extraction, requiring only specification of the desired output schema (e.g., JSON, Pydantic class, etc).

4/ We can look under the hood using LangSmith trace, seeing that the prompt instructs the LLM (in this case OpenAI) to call "information_extraction" (function defined) here:

Trace:
https://t.co/gf6DW3r8RM https://t.co/JAvzqpEh8sgithub.com/langchain-ai/l…
smith.langchain.com/public/72bc320…

5/ The docs provide some examples for using functions, which can shine in cases where we don't specifically a priori all the fields we want to extract. Whereas a parser requires enumeration of each attribute, functions work w/ fields such as "give me extra information":

6/ The docs also cover parsers, which are useful esp for LLMs that don't yet support function calling. We can use a LangSmith trace to see that parsers use few shot prompting under the hood:
https://t.co/ZMLQpdquTRsmith.langchain.com/public/8e3aa85…

7/ It's worth noting that more LLMs are getting support for function calling (e.g., @AnthropicAI) :


Llama2 has been fine-tuned to support it as well:
https://t.co/ISAY1MsnF8

https://twitter.com/hwchase17/status/1685667799174393857?s=20

https://twitter.com/jxnlco/status/1687492227998662656?s=20

8/ For more in-depth, see past webinars on parsing, extraction, and function calling w/ @GregKamradt, @jerwelborn, @veryboldbagel, @fpingham, @jxnlco


https://t.co/uRbFgMnxW1

9/ And for more on the community initiative to improve the docs, see Part 2 on summarization:

https://twitter.com/RLanceMartin/status/1687497639112364032?s=20