We just rolled out “thought summaries” in the Gemini API, now you can see what the model is thinking and make use of that info!
A thread with the details and request for feedback 🧵
A thread with the details and request for feedback 🧵
To start, you can enable this with 2 lines of code (only 1 if you are already using thinking budgets).
Link to docs: ai.google.dev/gemini-api/doc…
Link to docs: ai.google.dev/gemini-api/doc…
Behind the scenes, the model is still reasoning with full thoughts, and then we have a summarization model which translates from full thoughts to summary, while preserving as much detail as possible.
The summarization technique will likely change over time and we are working to provide developers with more control (choose how you want to summarize, etc).
Summaries are experimental at the moment and free to enable.
Summaries are experimental at the moment and free to enable.
Current model pricing doesn’t change, the thought tokens in “usage metadata” refer to the full thoughts (which is what you pay for) and the summaries are just free (and experimental).
Overall, excited to get this out into the hands of devs, we have a lot of exploration to do here still and are in feedback + iteration mode, so pls share thoughts on what we can do to make this more useful.
And yes, I know people want full thoughts : )
And yes, I know people want full thoughts : )
• • •
Missing some Tweet in this thread? You can try to
force a refresh
