# Why LLM APIs fail, and failover

> LLM APIs fail more often than most web APIs: providers time out, return 429 rate limits and 5xx overload errors, especially at busy times. Failover sends the same request to another provider or model when that happens, while errors caused by the request itself are returned straight away.

Source: https://digitspace.ai/learn/llm-failover · Last checked: 2026-10-30

## Errors, and what to do with them

| Error | Usual cause | Retry elsewhere? |
| --- | --- | --- |
| Timeout, connection reset | Provider overloaded or network trouble | Yes |
| 429 | Rate limit or quota reached on that provider | Yes |
| 500, 502, 503, 529 | Provider error or overload | Yes |
| 400, 401, 403, 404, 422 | The request, key or model name is wrong | No: fix the request |

## Doing failover well

- **Track health:** Count recent successes and failures per provider and model, and try healthy ones first.
- **Cap attempts:** Two or three tries is enough; more only makes a slow failure slower.
- **Fail fast on bad requests:** Another provider won't fix a malformed request.
- **Record every attempt:** So a provider that keeps failing is visible before it becomes an incident.

## Questions

### Isn't a simple retry loop enough?

Retrying the same provider during an outage usually fails again. Moving to a different provider or model is what keeps the request alive.

### Does failover change the answer?

Moving to another provider of the same model gives the same model's answer. Moving to another model, as "auto" may, can change wording but stays within your quality level.

## Related

- [Failover](https://digitspace.ai/features/failover)
- [How it works](https://digitspace.ai/how-it-works)
- [What is an AI gateway?](https://digitspace.ai/learn/what-is-an-ai-gateway)
- [SaaS products](https://digitspace.ai/solutions/saas-products)

Start free: https://digitspace.ai/signup · Docs: https://digitspace.ai/docs · Pricing: https://digitspace.ai/pricing