Ryan Samii Profile picture
Dec 1, 2022 11 tweets 4 min read Read on X
Prompting @OpenAI's ChatGPT with a series of simple to complex legal questions focused on contract language.

Some questions are handled exceptionally well; others highlight the shortcomings of applying AI to legal. Featuring interesting examples in this thread... (1/x)
First, gotta love that ChatGPT comes out of the gate with every lawyer's favorite way to start an answer: "It depends..."

Here, I ask the AI a contract interpretation question. It does well to punt on the q due to lack of sufficient info. (2/x) Image
Next, I try to constrain the AI's ability to punt by adding: "No other section of the NDA impairs this definition."

As a result, a more concrete response is returned by the AI. (3/x) Image
How about the AI's view on "standard" language? It returns a somewhat generic response, but tbh it's the type of response that I'd also expect from a lawyer on this same question. (4/x) Image
Ok, this next one is super interesting. I ask for a specific clause to be re-worked to be more favorable to a particular side of the contract.

The AI is headed in the right direction, but totally butchers the execution... (5/x)
The key words "OTHER THAN" are missing in the re-worked clause (should say "other than as a result...").

Explanation totally messes up the analysis. It functionally incentivizes breach of the agreement and leaves the "discloser" in a worse position than where it started. (6/x) Image
Perhaps the AI would've performed better with the context of the full contract.

But seeing the answer within this limited scope is a bit troubling... The AI seems to hit the right chords, but misses the key concept. Two words make all the difference. (7/x)
Ok, now testing the AI's ability to get into the weeds. I slip a few words into exception (A) that are absolutely NOT standard – "...or vaguely known..."

I then ask the AI if the block of language is standard. It doesn't catch the problematic words. (8/x) Image
But then when I provide a bit of a helping hand in my prompt and limit the language just to the problematic bit, the AI nails it!

Prompt engineering is key. (9/x) Image
@scottastevenson I know you've been on the frontlines of leveraging GPT for legal in all sorts of interesting ways.

Are you also finding that prompt engineering is important to avoid mistakes in analysis, recommendations, etc.?

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Ryan Samii

Ryan Samii Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @ryanrsamii

Jun 28, 2023
Navigating the world of pre-seed/seed funding is complex. But, every now and then, Twitter can be a goldmine of guidance.

For the past ~9 months, I've passively bookmarked the best links/tweets introducing pre-seed funds, knowing I'd revisit it someday.

That day is today 🧵👇🏽
1/15 – A curated list by @outsetcap of funds that lead pre-seed rounds (deploying $500k+ checks). Thank you to @RohdeAli and @kanjun for putting this together!

outsetcapital.com/writing/posts/…
2/15 – A tweet from @Julian introduced me to VCSheet, which is a super helpful resource.

Follow the link below for breakdowns like:
• funds that lead pre-seed
• most prolific pre-seed funds
• funds by geography
• funds by industry

vcsheet.com
Read 19 tweets
Nov 8, 2022
🎯 So important. Best way to find sticky, actionable legal tech insights? Try to bridge:

(1) legal tech content / outlooks that you encounter in articles, podcasts, etc.
with
(2) actual convos that you have with lawyers & stakeholders.
If your experience ends up being similar to mine, you'll encounter gaps. Sometimes legal tech-ers miss certain considerations that are really important to users. Other times they focus on considerations that barely move the needle.
I'll have a deeper 🧵 on this soon (based on my real-time, #buildinpublic experiences with @StandardDraft).

For now, want to share this great @ArtificialLawya piece that captures the spirit → artificiallawyer.com/2022/09/26/tow…
Read 4 tweets
Nov 7, 2022
About a year ago, I decided to take a DEEP dive into the world of #legaltech.

I’ve since spent 1000+ hours consuming legal tech content – articles, tweets, podcasts, video walkthroughs, you name it.

🧵of 15+ sites and newsletters that I’ve found particularly helpful!
First, a shoutout to the many incredible legal tech commentators & content creators out there.

I’ve benefited tremendously from their insights while building and iterating upon @StandardDraft.

Presenting the legal tech websites and newsletters in no particular order…
1 – @ArtificialLawya

Artificial Lawyer is the go-to source for legal tech news. Come for the comprehensive news & updates; stay for the on-point think pieces by Richard Tromans (the founder).

• Link → artificiallawyer.com
• Email news alerts available
Read 21 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(