Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
jason Profile picture
@jxnlco
Feb 8 • 4 tweets • 1 min read • Read on X
Guaranteed structure output with Ollama and Pydantic.
Check out the blog post to learn more about @pydantic and @ollama

jxnl.github.io/instructor/blo…
@pydantic @ollama I also just learned @ollama is @UWaterloo ece?!
instructor also supports partial streaming of json so:

1. don't wait for the full payload
2. no stress about json.loads

what do you think @ollama

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with jason

jason Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @jxnlco

jason Profile picture
Dec 31, 2023
if you know python, you don't need any LLMops.

trust , i've training models and deployed production applications that serve at >350M requests a day.

just need `pip install` and some good naming conventions

1. jinja - prompting frameworks
2. numpy - vector search
3. sqlite - evals, one row per exp
4. boto3 - data management, s3 and some folder structure
???
5. google sheets ;) - experiment tracking w/ a link to the artifacts saved in S3/GCS.

Disagree?
I've been training models in @PyTorch , and deploying them via @FastAPI since the library came out!

we did large image classification tasks where the folder structure reflected class labels and had a config.json in each directory.

our early a/b tests exported to google sheets and we served similar item recommendations via numpy brute force 3M skus with 40 dimension per vector (umap over resnet and matrix factorization machines)

newsroom.stitchfix.com/blog/your-shop…
@PyTorch @FastAPI I have nothing to sell you and sometimes i tweet about applied machine learning with @arnaudai (ex google brain) so give us a follow!
Read 4 tweets
jason Profile picture
Oct 12, 2023
For those who couldn't attend my 'Keynote' at @aiDotEngineer

> Pydantic is all you need

Making LLMs, software 3.0, backwards compatible with software 1.0



other links in the thread!youtube.com/live/veShHxQYP…
Presentation Slides:

tome.app/fivesixseven/p…
Repo: github.com/jxnl/instructor
Read 7 tweets
jason Profile picture
Jul 18, 2023
0/ Any real AI engineer knows that streaming REALLY improves the UX.

Today, I'm landing a change that defines a reliable way to stream out multiple @pydantic objects from @OpenAI .

Take a look, by the end, you'll know how to do streaming extraction and why it matters. Image
1/ Streaming is critical when building applications where the UI is generated by the AI.

Notice in the screenshot that the first item was returned in 560ms but the last one in almost 2000ms! a 4x difference in time to first content

How do we do this? Image
2/ By using `pip install openai_function_call` we can do the following:

1) Use MultiTask to dynamically make a schema based on a `cls` that we defined
2) Set stream=True to unlock the latency win
3) `from_streaming_response` allows us to parse the completion into generator[cls]. Image
Read 7 tweets
jason Profile picture
Jun 29, 2023
Why prompt engineer @openai with strings?

Don't even make it string, or a dag, make it a pipeline.

Single level of abstraction:

Tool and Prompt and Context and Technique? its the same thing, it a description of what I want.

The code is the prompt. None of this shit
"{}{{}} {}}".format{"{}{}"

PR in the next tweet.
@OpenAI github.com/jxnl/openai_fu…
with this, we can import templates as just additional messages.

you can colocate 'prompt' with your data structures
you can colocate 'prompt' with your computation

In a way that readable and shows you everything in a few lines of code
Read 6 tweets
jason Profile picture
Jun 22, 2023
If you've followed me from the last @LangChainAI webinar I wanted to share the repo that contains the code examples. Contributions of other ideas / evals / or examples are totally welcome. If you want to help you check the issues!



🧵github.com/jxnl/openai_fu…
I'm not trying to make a framework!

Its my exploration on abstractions and design patterns, something that can be leveraged (and has been) in both @LangChainAI & @llama_index

It's also a public notebook of sorts to capture some ideas I have in my mind.
Consider it... light research with design patterns, ergonomcs, and abstractions.
Read 5 tweets
jason Profile picture
Jun 21, 2023
1/ Defending from SQL injections w/ @OpenAI

I tried to add another layer of protection for sql writing systems by not just returning the string, but the template, and parameters separately.

So lets use @pydantic again to build a small example of going above SQL strings!
2/ So again, lets begin with the schema:

1) we define literals and identifiers as enum
2) we define what a parameter is so we can template
3) lastly we ask for the string template to use

as a nice to have we try to mark it safe which could be used to warn, or hook into UI.
Its a pretty simple change when you configure by code, in fact, its the same kind of change I'd ask my intern to do if they arn't doing it already.

I'd literally just show them this blog post: realpython.com/prevent-python…
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

Send Email!

:(