What Is an Uptime SLA?

What Is an Uptime SLA?

An uptime SLA (Service Level Agreement) is a contractual commitment to a minimum availability percentage for a service over a defined billing period. Common targets: 99.9% allows about 8.77 hours of downtime per year, 99.95% about 4.38 hours, 99.99% about 52 minutes. For an LLM application the SLA covers the full user-visible path — model inference, gateway, retrieval, tools, and response. A miss usually triggers service credits. The operational question is which combination of model fallback, retry policy, and multi-provider routing keeps the system above target when an upstream model provider has an outage.

Why It Matters in Production LLM and Agent Systems

LLM applications inherit availability from a long chain: your code, your gateway, the model provider, the retrieval index, every tool API the agent calls. The chain is only as available as its weakest link, and model providers have not historically delivered the four-nines availability that B2B customers expect. A single OpenAI or Anthropic regional incident can take a single-provider app offline for hours.

The pain is felt across roles. SREs page when error rates spike, then discover the upstream provider is down and there is no failover. Product owners face customer escalations for incidents the team cannot fix directly. Sales teams have to negotiate SLAs that depend on a third party they do not control. Compliance leads need an availability story that survives an enterprise security review.

In 2026 agent stacks, partial outages are worse than full outages. A planner step succeeds, the next tool times out, the agent retries with no backoff, and a single user request consumes 12 model calls and 90 seconds. From the user’s perspective the system is “up but broken.” Uptime SLAs that count only HTTP success miss this entirely; meaningful SLAs include trajectory completion within a latency budget.

How FutureAGI Handles Uptime SLA

FutureAGI’s approach combines two surfaces: the Agent Command Center (gateway) gives you the failover primitives, and traceAI gives you the visibility to know when the SLA is at risk.

Concretely: a chat application is contractually on a 99.95% uptime SLA. The Agent Command Center is configured with a primary route to GPT-4-class, a model-fallback route to Claude on a different provider, a retry-strategy of two retries with exponential backoff, and a routing-policy: least-latency that re-balances when one provider’s p99 spikes. When the primary provider has a regional outage, traffic auto-fails over without user-visible impact. On the observability side, traceAI-langchain records every span with provider id, route, retry count, and llm.token_count.prompt; a dashboard tracks availability by route, fallback rate, and trajectory completion within latency budget. A Slack alert fires when fallback rate crosses 5% — long before SLA is at risk. Unlike a single-provider stack that prays the provider’s status page stays green, this is observable, gated availability.

How to Measure or Detect It

Signals to track:

• Availability percent over rolling window: 30-day and 90-day rolling availability per route.
• Fallback rate: percent of requests that hit the secondary route — a leading indicator of upstream stress.
• Retry count distribution: rising tails are a sign of provider-side flapping.
• Trajectory-completion rate within latency budget: end-to-end success including tool calls.
• Error class breakdown: 5xx vs 429 vs timeout — each requires a different mitigation.
• llm.token_count.prompt trend during incidents: spikes during retries explain cost overruns.

# Example: a simple availability monitor on traces
def availability(traces, window):
    successes = sum(1 for t in traces if t.status == "ok")
    return successes / max(len(traces), 1)

Common Mistakes

• Counting only model errors. A 200 OK with a hallucinated answer is not an SLA hit, but it is a customer hit; pair availability with eval-fail-rate.
• Single-provider lock-in with no fallback. If your SLA is 99.95% and your model provider’s is 99.9%, you cannot meet your target without redundancy.
• No retry budget. Aggressive retries during a partial outage amplify the upstream load and extend the incident.
• Measuring at the gateway only. End-to-end SLA must include retrieval and tool latency, not just inference time.