Know about problems
before your users do
Set alert rules on error rates, latency, token usage, LLM failures, and custom eval metrics. Static thresholds or percentage-change detection. Warning and critical levels. Slack and email. Full resolution tracking.
Alerting built for
AI agent metrics
Monitor error counts, error rates for function calling, LLM API failure rates, span response time, LLM response time, token usage (daily and monthly), error-free session rates, service provider error rates, and any custom evaluation metric you define. Each metric has its own query engine optimized for speed.
View all metricsSet fixed thresholds (error rate > 5%) or let the system calculate baselines automatically. Percentage-change alerts compute the historical mean and standard deviation over a configurable window, then fire when the current value deviates beyond your configured percentage. Dual-level thresholds - warning and critical - so you can escalate proportionally.
Configure thresholdsRoute alerts to up to 5 email recipients and Slack channels simultaneously. Every alert trigger is logged with timestamps, severity level, and the exact metric values that fired. Mark alerts as resolved individually or in bulk, with full audit trail of who resolved what and when.
Set up notificationsPreview alert graphs before creating the monitor - see how your thresholds would have performed over the last 7 days. Mute monitors during maintenance windows without deleting configuration. Every monitor tracks its check frequency, last checked time, and full event log.
Explore alert lifecycle Stay ahead of
every issue
Evaluation metric regression
Alert when any custom eval metric - faithfulness, context adherence, relevance - drops below threshold or deviates from its 7-day baseline.
LLM response time spikes
Catch latency degradation from provider outages or model changes. Percentage-change alerts detect anomalies the moment response times deviate from the historical norm.
Token usage budget alerts
Monitor daily and monthly token spend per project. Set warning at 80% and critical at 100% of budget. Prevent runaway agents from burning through credits overnight.
Function calling error rates
Alert when tool-use and function calling errors spike. Filter by specific observation types to isolate which agent components are failing.
LLM API failure monitoring
Catch provider outages instantly. Monitor failure rates across all LLM API calls and get notified before your error budget is depleted.
Error-free session rate drops
Track the percentage of user sessions without errors. Alert when the error-free rate drops below your SLA target - a leading indicator of user experience degradation.
From alert to
resolution in three steps
Pick a metric and set thresholds
Choose from 11 built-in metrics or any custom evaluation. Set static thresholds or percentage-change detection with warning and critical levels. Preview the graph to see how your rule would have performed over the last 7 days.
Route to Slack and email
Add up to 5 email recipients and a Slack webhook. Attach custom notes for context. Set the check frequency - alerts evaluate every 5+ minutes against your configured time window.
Resolve with full context
Every alert log includes the metric value that triggered, the threshold it crossed, and the time window. Mark alerts resolved individually or in bulk. Full audit trail of who resolved what and when.
Powering teams from
prototype to production
From ambitious startups to global enterprises, teams trust Future AGI to ship AI agents confidently.