Alright, the news is out. @jlowin and I are back at it again.

Super excited to launch ControlFlow.

ControlFlow is an enterprise-grade orchestration framework for multi-agent workflows, built on Prefect. Check it out at .github.com/PrefectHQ/Cont…

Most real use cases of Large Language Models are doing the boring stuff really well.

So let's talk about data deduplication: one of the most valuable, boring things we've been grinding on for a decade.

Important thing here is the idea, but @AskMarvinAI makes this ez-pz. (1/6) We want to think carefully about the data we want to extract from this text.

We really want a mapping or understanding that SF is the same as San Francisco, and the Big Apple is the same as New York City, etc.

So let's create a data model to represent a city. (2/6)

Sorry if you had weekend plans...

@AskMarvinAI 1.5 drops today

Three *huge* parts of this release:

1. ✅ @pydantic V2 and V1 compatibile
2. 🪄 @AnthropicAI Claude function calling support.

and 🥁🥁🥁

3. We're putting the engineering back in prompt engineer with prompt_fn If you don't follow @AskMarvinAI, we've been called the FastAPI of LLMs. Thousands of engineers choose Marvin because the LLM is "ambient".

You write pure python, we abstract away the rest.

But honestly it can be a little too magic?

We're fixing that.

https://twitter.com/DrJimFan/status/1657782710344249344

Sorry for the delay!

Let's talk about text-to-ORM.

In the before times we had to speak machine. So we built DSLs like SQL so it felt more like Latin and less like Greek, and ORMs so it felt more like English and less like Latin.

Now that machines speak actual English... (1/7)

https://twitter.com/AAAzzam/status/1680912656696213511

We're tempted to write text-to-SQL

But why? An ORM is struct-to-SQL, and already encapsulates all the guardrails/relational logic already

text-to-SQL is hard from a comprehension and syntactical perspective.

text-to-ORM is hard too, but easier if posed as text-to-struct (2/7)

I built a smart router for @FastAPI using @AskMarvinAI's 🏷️ai_classifier. It 🧠 deduces the best route based on an user's query:

/q=Why did my usage spike this morning?

redirects to /logs or, you know, whatever

Best of all it's bulletproof, cheap, and blazing fast 🧵👇 Why is it fast, bulletproof, and cheap?

ai_classifier enumerates the options and exploits logit_bias and max_tokens=1 to force your LLM to choose the index corresponding to the best option.

Your model may vary, but it's about 200ms and obviously cheap to get one response token

You can use @OpenAI `functions` to author and transpile code. This lets @OpenAI functions author its own `functions` for future calls.

How? We'll take a page from category theory!

We'll use @pydantic to model a function and get @OpenAI to write it. We'll give @OpenAI an objective 'write the nth fibonacci number' and pass it a 'write_function' `function`, with a @pydantic representing it's code, name, etc.

The result? Code, baby, code!

[Of course, untrusted, untested code! But let's dream for a second, folks.]

You can do constrained sampling in ChatGPT and micromanage its output. The catch? You only get one token. But you can do a lot with that one token.

You can make it a classifier, a logic gate, or have it choose tools deductively. Here's how 🧵

From my talk this week @trychroma You can make ChatGPT a logic gate. Using tiktoken you can find that it represents true as `1904` and false as `3934`. If you use logit_bias to dictate that it can only response with 1904 and 3934, and set max_tokens to 1, you guarantee that it'll always output true or false.