Agent Engineering Recovery Workflow Example vs Responses API recovery order: AI How should the programming team choose?

Applicable objects

This article is intended for engineers and technical leads on the AI programming team. You are building or maintaining an Agent system and have encountered a "recovery workflow" design problem: When the Agent call fails, the context is lost, or the tool execution times out, which method should be used to allow the system to automatically recover? You may have heard of the "Recovery Workflow Example" (a retry + state rollback template) recommended by the Agent Engineering community, and have also come into contact with the "recovery order" mechanism that comes with OpenAI Responses API. But which one to choose in what scenario? This needs to be judged based on the team's technology stack, system complexity, and demand for recovery accuracy.

Compare dimensions

1. Recovery capability range: customized vs. built-in platform

Agent Engineering Recovery Workflow Example is usually a custom-implemented recovery template that allows you to define a "recovery chain": for example, retry three times, roll back to the previous checkpoint if it fails, and degrade to read-only mode if it fails. This flexibility means you can precisely control the recovery logic at every step and even write conditional branches to handle different error types.

Responses API recovery order is the internal recovery mechanism provided by OpenAI, which is implemented at the API call layer. When a request fails, the API will automatically try alternative configurations according to preset priorities (such as falling back to a smaller model, lowering temperature, shortening max_tokens). But you can't interfere with this order, you can only change the priority by adjusting the "recovery order" array in the API parameters.

2. Applicable scenarios: when to use custom recovery and when to use API level recovery

Scenario 1: Recovery of multiple tool calls within Agent

Imagine an Agent that needs to call "Search->Code Generation->Test" sequentially. If code generation fails, you may want to roll back the side effects of the search (such as freeing memory) and try again. At this time, the Responses API recovery order is powerless because it only handles the failure of a single API call and cannot sense the state dependencies between multiple tools. You need to customize the Recovery Workflow Example and introduce a state machine to manage the compensation operations at each step.

How to do it: In the Agent's loop, define an "on_failure" callback for each tool call. When code generation fails, the search rollback function (such as clearing the cache) is first executed, and then it is decided whether to replace it with an alternate code model based on the error type.

Scenario 2: Rapid degradation of API calls under high load

If your team is building a chatbot using Responses API, get an expensive gpt-4o model. When the API returns 429 (throttle) or 500 (server error), you want to automatically switch to the smaller, cheaper gpt-3.5-turbo model and reduce the response length to save costs. At this time, it is most direct to use Responses API recovery order: set "recovery_order": [{"model": "gpt-3.5-turbo", "max_tokens": 512}, {"model": "gpt-3.5-turbo", "max_tokens": 256}] in the request. The API automatically tries in sequence every time it fails, without you having to write retry logic.

3. The boundary where failure is most likely

Failure Point 1: Custom Recovery Workflow Status Expansion

When many teams first start using the Recovery Workflow Example, they like to map each error into a recovery step. But soon the state diagram turned into a mess. An engineer shared on Reddit: His Agent has 12 states and 30 transitions, resulting in less than 30% test coverage. Finally, in a production accident, the recovery workflow entered an infinite loop, and the logs were all "recovery->rollback->retry->recovery".

The key is: don't design recovery for every error. Prioritize errors and define recovery paths only for "recoverable" errors (such as network timeout and temporary current limiting). For "irrecoverable" ones (such as invalid input format, insufficient permissions), they should be thrown directly to humans for processing.

Failure point 2: Responses API recovery order cannot handle business-level errors

The recovery order of Responses API only covers HTTP layer errors. If in your business logic, the API returns 200 but the JSON field is missing or the content is truncated, it will not trigger recovery. One time we used gpt-4-1106-preview to generate code, and the code block returned was truncated in half, but the API considered the request successful. In this case the recovery order will not initiate recovery, we need to check the integrity at the application layer and trigger a retry.

A real scenario (including failure and solution)

Last year, I helped a AI product team doing code review to refactor. They used Responses API to build a code analysis agent: receive diff, call API analysis, and return suggestions. The API often times out during peak periods, so they set up a recovery order: first retry the same configuration twice, then switch to gpt-3.5-turbo. The solution seemed reasonable, but after going online, I found that when gpt-3.5-turbo also times out, the API will return an error, and the recovery order will be executed without retrying. This error is not caught by the application layer, causing the user to see a blank page.

Later, we changed to a customized Recovery Workflow Example: Wrapping a layer of recovery logic outside the API execution layer - if all API recovery steps fail, a degraded response with "Analysis timeout, please try again later" is returned instead of letting the error penetrate to the UI. At the same time, the context of this failure is recorded in the log (including the diff summary) for subsequent offline re-running.

This case illustrates: Responses API recovery order is suitable as the first line of defense, but it cannot solve all problems. For complex scenarios that require graceful downgrade and log tracking, custom recovery workflows are a necessary supplement.

Executable practices: How should the team choose

1. List the core call types of your Agent: Is it a single API call or a multi-tool chain call? For single calls, consider Responses API recovery order, and for multiple calls, consider custom Recovery Workflow.
2. Grading errors: Which errors are "recoverable" (such as timeout, current limit)? Which ones are "unrecoverable" (e.g. invalid parameters, failed authentication)? Only design recovery paths for recoverable errors.
3. Start simple: First use Responses API recovery order as a fast layer, and then add custom recovery for the critical path. Don't design a complex recovery state machine on day one.
4. Monitor the recovery effect: Record the log of each recovery success or failure. If the recovery success rate is less than 80%, the recovery strategy needs to be adjusted (such as increasing the retry interval, replacing the backup model).
5. Prepare a backup plan: If neither recovery method can handle the error, ensure that the system has a "downgrade response" mechanism (such as returning cached results or prompting the user to try again later) rather than exposing the error to the end user.

Limitations

• Agent Engineering Recovery Workflow Example requires the team to have state machine design capabilities, and the maintenance cost is high.
• Responses API recovery order is only valid for OpenAI API. If you switch to other model providers (such as Anthropic), you need to implement it yourself.
• Neither method guarantees 100% recovery success. For failures where the root cause is a code bug or data error, recovery will only mask the problem and ultimately require manual intervention.

Summary

On the AI programming team, choosing a recovery workflow is not an "either/or" but a layered defense. Responses API recovery order is suitable for quickly handling transient failures at the API layer, while the custom Recovery Workflow Example is suitable for handling complex business-level recovery and state management. Based on your call chain length and business tolerance, decide at which layer to deploy recovery logic.

Next step

If you want to systematically master the design patterns of Agent engineering, including core skills such as recovery workflow, context management, tool calibration, etc., we recommend in-depth study of high-quality original paid articles and AI advanced programming courses.