Skip to main content

What this snippet demonstrates

LLM API calls fail — rate limits, outages, timeouts. AFK’s fallback_model_chain lets you define an ordered list of models to try when the primary model fails. This snippet shows how to configure fallback chains for resilience, cost optimization, and provider diversification.

Basic fallback chain

When gpt-5.5 fails (timeout, rate limit, outage):
  1. AFK retries with the primary model (controlled by retry policy)
  2. If retries exhaust, it falls through to gpt-5.5
  3. If that also fails, it tries gpt-5.5
  4. If all models fail, the llm_failure_policy determines the outcome

Cost-optimized fallback

Use expensive models only when needed:

Circuit breaker integration

AFK’s built-in circuit breaker works with fallback chains. When a model triggers too many failures, the breaker opens and the system skips straight to the next fallback:

Multi-agent with different model tiers

Use different model tiers for different specialists:

Inspecting which model was used

After a run, check the result metadata and usage aggregate:

Recommendations