Designing Reliable Error Handling for AI Agent Workflows

Why an AI agent needs more than a retry loop

A conventional request may call one service and return one response. An agent workflow can select a tool, generate arguments, call an external service, interpret the response, and decide whether another step is needed. A failure at each boundary has a different cause and a different recovery strategy.

For example, retrying a timeout may be reasonable. Retrying invalid JSON with the same prompt is usually not. Repeating a payment, email, or data mutation without an idempotency key can create a duplicate side effect. Reliability begins by separating these cases instead of treating every error as “ask the model again.”

Map the workflow before adding recovery

Write down the steps of the workflow and mark where data changes form or crosses a trust boundary. A useful minimum map is:

1. User request and authorization context
2. Model response and selected tool call
3. Tool argument validation
4. External tool execution
5. Tool result validation and redaction
6. Final response or side effect

This map gives each failure a location. It also shows where logs should carry a request identifier and where a human confirmation step may be required.

Validate model-generated arguments

Never pass model-generated arguments directly into a database, shell command, email provider, or third-party API. Parse them with a schema, enforce authorization separately, and reject values outside the tool’s allowed range. Validation is not a replacement for permission checks: a well-formed request can still be unauthorized.

const parsed = toolSchema.safeParse(modelToolCall.arguments);

if (!parsed.success) {
  return {
    status: 'needs_correction',
    message: 'The tool arguments did not match the required format.',
  };
}

if (!canRunTool(user, parsed.data)) {
  return { status: 'forbidden', message: 'This action is not allowed.' };
}

const result = await runTool(parsed.data);
return resultSchema.parse(result);

A structured validation result can be returned to the model for a limited correction attempt, but the system should still enforce a maximum number of attempts.

Classify failures before choosing a response

• Validation failure: repair or reject the input; do not blindly retry.
• Authentication or authorization failure: stop and report the boundary.
• Rate limit or temporary network failure: retry with backoff if safe.
• Provider or dependency failure: use a known fallback or stop clearly.
• Business-rule failure: explain what condition prevented the action.
• Unknown failure: record diagnostics and avoid a success-shaped response.

Use bounded, safe retries

A retry policy should specify the maximum attempts, eligible failure classes, delay strategy, and total time budget. Exponential backoff with jitter prevents many workers from retrying at exactly the same moment. Do not retry a non-idempotent side effect unless the downstream operation supports an idempotency key or the application can prove that the first attempt did not complete.

for (let attempt = 0; attempt < MAX_ATTEMPTS; attempt += 1) {
  try {
    return await callWithTimeout(tool, TIMEOUT_MS);
  } catch (error) {
    if (!isRetryable(error) || attempt === MAX_ATTEMPTS - 1) {
      throw error;
    }
    await sleep(backoffWithJitter(attempt));
  }
}

Make side effects observable and idempotent

Reading data and changing data are different risk levels. Before an agent sends a message, creates a record, changes access, or triggers a deployment, define the operation identifier and the confirmation policy. Store enough state to recognize a repeated request. If the action is high impact, show the user what will happen and ask for confirmation after the agent has prepared the action but before it executes it.

Design an honest fallback

A fallback should preserve the truth about the operation. “The request was accepted” is not an acceptable fallback if the downstream system never confirmed it. Prefer messages such as “We could not verify the result; check the activity log before trying again.” For recoverable tasks, offer a retry button that starts a new, traceable attempt.

Log for diagnosis without logging secrets

Record a workflow identifier, step name, latency, retry count, failure category, and whether a fallback or human review was used. Redact access tokens, personal data, and full prompts when they are not required for diagnosis. A log that contains secrets is a new security problem, not an observability improvement.

• Workflow and request identifiers
• Tool name and schema version
• Validation result and failure category
• Timeout, retry, and provider status information
• Final outcome: completed, rejected, pending, or needs review

A practical test matrix

Reliability should be tested with deterministic failures, not only successful demos. Include malformed tool arguments, empty results, rate limits, timeouts, expired credentials, duplicate delivery, partial downstream success, and a model response that chooses an unavailable tool. Verify that every case produces the intended user-visible status and leaves no untracked side effect.

FAQ

Should every model response be retried?

No. Retry only a classified, likely temporary failure. A malformed response should be rejected or corrected with additional context, while an authorization failure should stop the workflow.

How many retries should an agent have?

Start with a small explicit budget and measure latency, cost, and recovery rate. A read-only lookup can tolerate a different budget from a workflow that changes data.

How can I estimate prompt size before a run?

The LLM Token Estimator provides a rough, browser-only estimate. It is not a tokenizer for a specific provider, so production code should use the exact tokenizer or usage data for the selected model.

References

• OpenAI function-calling guidance
• JSON Schema fundamentals
• Google guidance on helpful, reliable content