You can now connect your Model Context Protocol servers to Agents:
We’re also working on MCP support for the OpenAI API and ChatGPT desktop app—we’ll share some more news in the coming months.openai.github.io/openai-agents-…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
We're launching new tools to help developers build reliable and powerful AI agents. 🤖🔧
Timestamps:
01:54 Web search
02:41 File search
03:22 Computer use
04:07 Responses API
10:17 Agents SDK
Our new API primitive: the Responses API. Combining the simplicity of Chat Completions with the tool-use of Assistants, this new foundation provides more flexibility in building agents. Web search, file search, or computer use are a couple lines of code!
o‑series models excel at handling ambiguous, multi‑step tasks in domains such as math, engineering, legal, and finance—“the planners.” 🧠
Use o-series models to process unstructured data, find a needle in a haystack, improve code, or handle other complex tasks. For example, o1’s vision capabilities can analyze detailed architectural drawings. In this image, o1 recognized that “PT” wood posts were pressure-treated.
We’ve put together a reference implementation for building and orchestrating agentic patterns using the Realtime API. You can use this repo to prototype a voice app using multi-agent flows in less than 20 minutes!
Building with the Realtime API can be complex because of the low-latency, synchronous nature of voice interactions. This repo includes best practices we’ve learned for managing this complexity, like:
- Orchestrating agent handoffs (inspired by Swarm)
- Background escalation to o1 for advanced decision making
- Improving model instruction following by defining a state machine in the prompt
- Demos of applying these patterns to customer service and front desk use cases
It also includes a meta-prompt to make it fast and easy to define new agents with a range of personalities, and uses the newer, simpler WebRTC interface.
We announced the Realtime API last week at DevDay SF. It's been amazing to see its adoption—here are some of the coolest examples we’ve seen so far. Let us know if we missed any in this thread!
Today at DevDay SF, we’re launching a bunch of new capabilities to the OpenAI platform:
🗣️ Introducing the Realtime API—build speech-to-speech experiences into your applications. Like ChatGPT’s Advanced Voice, but for your own app. Rolling out in beta for developers on paid tiers. openai.com/index/introduc…
🗃️ Prompt Caching is now available. Our models can reuse recently seen input tokens, letting you add even more cached context into our models at a 50% discount and with no effect on latency. openai.com/index/api-prom…
Introducing a series of updates to the Assistants API 🧵
With the new file search tool, you can quickly integrate knowledge retrieval, now allowing up to 10,000 files per assistant. It works with our new vector store objects for automated file parsing, chunking, and embedding.
New token controls allow you to set maximum input and output tokens per run to manage costs. You can also choose how many recent messages to use for context truncation.
Support for tool choice lets you specify whether to use file search, code interpreter, or a particular function in a given run—increasing precision in your assistant’s operations.