Responses API How Does It Work? Explaining with a Real Link

Why We Must Understand Now Responses API

Responses API is the new interface form that OpenAI will be promoted in 2025, merging the previously fragmented Chat Completions, Function Calling, and Tool Use into a unified request-response protocol. This means that if you are still using old APIs to piece together conversation history and manually manage tool call states, it will become increasingly difficult to maintain in the second half of the year—both the new SDK and Agent SDK only support Responses API by default, and the old version is already counting down.

But this doesn't mean you only need to simply replace the endpoint URL. I've seen too many teams experience issues like "conversational context loss," "tool call order disorder," and "timeout responses swallowed" after migration. Essentially, Responses API has taken part of "explicit state management" away from developers. If you don't understand its data flow, it's easy to fall into traps.

A Real User Journey: From Asking to Full Response

To illustrate better, let's use a real-life scenario to understand the entire process: a user asks, "Help me check yesterday's server logs and then summarize the error types."

Step 1: Construct the request and send the user context

You just need to provide an "input" string and a list of available tools (if available). Responses API will automatically place this input into an internal state context, so you don't need to manually concatenate multiple rounds of message arrays. For example:

POST /v1/responses
{
  "model": "gpt-4o",
  "input": "帮我查一下昨天的服务器日志，然后汇总错误类型",
  "tools": [{
    "type": "function",
    "name": "query_logs",
    "description": "查询指定日期的服务器日志",
    "parameters": { ... }
  }]
}

这里的关键区别：旧 API 需要你把“我”的历史消息都放在 messages 数组里，而 Responses API 只关心当前轮的输入，以及你是否开启 ]previous_response_id[ 来延续上下文。

第二步：模型决定调用工具

模型会返回一个 ]response[ 对象，其中 ]output[ 字段不再是简单的“assistant 消息”，而是一个数组，可能包含多个 ]function_call[ 和 ]content[ 块。例如：

{
  "id": "resp_abc123",
  "output": [
    { "type": "function_call", "name": "query_logs", "arguments": "{\"date\":\"2025-02-10\"}" },
    { "type": "function_call", "name": "query_logs", "arguments": "{\"date\":\"2025-02-11\"}" }
  ]
}

请注意，这里一口气发起了两个工具调用（因为“昨天”可能跨日）。你需要并行执行这两个调用，然后把结果作为下一次请求的一部分返回。容易出错的地方是：很多开发者会像旧 API 那样只处理第一个 function_call，导致日志查询不完整。

第三步：把工具结果送回上下文

这是最容易失败的一步。你需要把工具的结果拼接成 ]status: completed[ 的 tool_call，然后通过 ]previous_response_id[ 关联到上一个响应，再次发出请求：

POST /v1/responses
{
  "model": "gpt-4o",
  "previous_response_id": "resp_abc123",
  "input": [
    { "type": "function_call_output", "call_id": "call_001", "output": "[...日志行...]" },
    { "type": "function_call_output", "call_id": "call_002", "output": "[...日志行...]" }
  ]
}

If you forget to set previous_response_id, the model will think it's a completely new conversation and have no idea what's happening ahead. This is the root cause of the return "lost context" error.

Step 4: Model Generates Final Answer

After obtaining the tool results, the model synthesizes all information and returns an output of type: "message", which is the summed error type.

The Most Prone Areas for Failure and Misunderstanding

Misconception 1: Thinking Responses API does not need to manage context

This is the biggest misunderstanding. Although the API retains a contextual state internally, you still need to explicitly pass previous_response_id. If you don't maintain this ID during multi-turn conversations, the model is always "first meeting."

Misconception 2: When calling parallel tools, only the first one is handled

As shown in step two, the output array may contain multiple function_call. You must respond to all calls; Otherwise, the model will get stuck or return an incomplete answer.

Failure scenario: Timeout causes response loss

The default timeout for Responses API is 30 seconds. If your tool runs for more than 30 seconds, the API will return a timeout error, and the results of the tool called will not be retained. At this point, you need to retry the entire request and let the model know that "the previous step timed out."

Backup plan: Fall back to Chat Completions + manual state management

If your tool's call logic is particularly complex (multi-turn nesting) or has very high latency requirements (<500ms), then manually managing message arrays using the Chat Completions API may be a more stable choice. Responses API is designed mainly to simplify regular agent scenarios and is not suitable for all scenarios.

If you want to land now, what should you do first?

1. Read the official Responses API OpenAI documentation (February 2025 edition), focusing on the previous_response_id and output structures.
2. In your development environment, migrate the most commonly used single-wheel tool call (such as "Check Weather") from Chat Completions to Responses API, and complete the entire process.
3. Record latency and token consumption before and after migration. Responses API It's easier to predict token usage because input no longer contains historical messages (but tokens are consumed within the context).
4. For multi-tool parallel scenarios, write a simple loop to ensure all function_call are collected and returned.
5. Set error retry logic: When timeout or a "context lost" error occurs, retry up to 2 times and consider a rollback plan.

Where to Go Next: Continue Systematic Learning

The above is just the basic usage of Responses API. If you want to turn this chain into production-level agent services—such as those involving long conversation memory, multi-tool orchestration, conditional branching, and error recovery—you need a more systematic methodology and real-world cases.