Krista Opsahl-Ong Profile picture
Mar 8 9 tweets 3 min read Read on X
Got a pipeline with **multiple prompts**, like a DSPy program? What's the right way to jointly optimize these prompts?

Introducing MIPRO, a Multi-prompt Instruction Proposal Optimizer.

We integrated MIPRO into DSPy. It can deliver +11% gains over existing DSPy optimizers!

MIPRO takes your Language Program (in DSPy), some training inputs (possibly without labels), and a metric to maximize (for accuracy, you’ll just need labels for final output).

Then it'll propose & validate instructions + examples for all prompts in your system! Image
What makes MIPRO special?

MIPRO is itself a multi-stage Language Program!

It implements an automated data scientist. First, it looks at your data to find interesting trends that can improve instruction quality! We’ve seen this find some pretty intuitive patterns!
Then, MIPRO runs your program on several inputs to collect examples of each module.

It'll keep the “good” traces (inputs/outputs) of each module, where the final output passes the metric’s validation. E.g., the answers are correct or the output fits the required criteria.
After that, MIPRO leverages the data summary & the few-shot examples it created to propose many plausible instructions — for each module in your DSPy program!

Then, MIPRO uses the proposed instructions & examples (for each module!) as candidates for joint optimization.
We treat the instructions/examples of each module as hyperparameters of the program and use a Bayesian Optimizer (TPE) to efficiently search this space — while pruning bad combinations quickly.

TPE helps our optimizer select better combinations over time.
Note: If you’d been using DSPy's BayesianSignatureOptimizer before this announcement, then you’ve been using early versions of MIPRO!

It’s been public for the past 2.5 months — and so many people have already used it successfully! E.g. @CShorten30 here:

We built this easy- and free-to-use notebook showing MIPRO on a retrieval-augmented DSPy program.

Everything in there is cached already, so you don’t even need an API key to try it:…
It's been amazing collaborating with @michaelryan207 @lateinteraction on this work! And big thank you to @StanfordAILab and @stanfordnlp for making all of this possible 🙌

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Krista Opsahl-Ong

Krista Opsahl-Ong Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!


0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy


3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!