AI Guardrails vs Evals: What Each One Actually Catches

Your agent ran a database migration at 2am. It passed every eval in your suite, shipped clean, and then dropped a column that a downstream service still read from. The incident review asked one question: should we have measured that risk before release, or blocked the action live?

That review is an AI guardrails question, and the answer depends on the action, not on taste. Two controls were on the table, and they do different jobs. Evals measure whether behavior matches expectations, mostly before you ship and on samples afterward. Guardrails enforce a decision at runtime, on the one request in front of them. The migration passed the first and needed the second.

This piece maps the two controls to where they run and what each one misses. You get the four points in the request path where AI guardrails attach, the three places evals run, the failures neither catches alone, and a framework that ties the control you pick to the blast radius of the action. By the end you can look at any agent action and name the layer it needs.

One boundary up front, because we have written a lot about guardrails. Our ultimate guide to LLM guardrails is the build manual: rail-by-rail implementation, backend selection, latency budgets, and precision/recall tuning. Agent runtime guardrails goes deep on the tool-call layer specifically. This page is neither. It is the decision page, for the moment you are holding one agent action and have to say which control it needs and why.

TL;DR

• AI guardrails enforce a decision at runtime; evals measure quality before ship and on sampled traffic. Different jobs, not competitors.
• Guardrails attach at four points in the request path: input, retrieval, tool call, and output.
• Evals run offline on a golden set, in CI as a release gate, and online on sampled production traffic.
• Evals miss the rare live outlier because they sample; guardrails miss the slow quality trend because they judge one request at a time.
• Match the control to blast radius: read-only actions need evals, irreversible actions need a runtime guardrail plus human approval.

Two different jobs: measuring behavior vs enforcing it

Separate the two controls by the job each performs, because the textbook definitions blur them. An eval is a measurement. You take a behavior, compare it against an expectation, and get back a number: a score, a pass rate, a win rate against a baseline. Most evals run before you ship and on samples of traffic afterward. What they produce is knowledge about the trend, whether this release got better or worse, and where.

A guardrail is an enforcement. It sits in the live request path and makes one decision about one request: allow it, change it, or block it. It does not track the trend across a thousand requests. It checks whether this specific input, retrieval, tool call, or output is safe to let through right now. What it produces is an intervention, not a report.

That difference drives the rest. Because evals sample, they can tell you quality slipped four points this week but cannot stop the single harmful response that ships to a user on Tuesday. Because guardrails act per request, they can block that Tuesday response but cannot tell you the model has been drifting for a month. One watches the population, the other watches the individual, and a setup that treats them as interchangeable goes blind on one of the two. We covered the measurement side in depth in the definitive guide to AI agent evaluation; here the focus is how it pairs with enforcement.

Where Do AI Guardrails Run in the Request Path?

AI guardrails are not a single thing bolted to the model. They attach at four points along the request path, and each point catches a different class of problem. Naming them separately keeps you from assuming a single output filter covers everything. The naming comes from NVIDIA NeMo Guardrails, which defines five rail types: input, dialog, retrieval, execution, and output. The four below are the ones that carry a distinct enforcement decision; NeMo’s dialog rails shape prompting rather than allow or block, so they sit outside this comparison.

The input rail runs first, on the incoming prompt before the model sees it. It blocks or cleans obvious attacks and disallowed content: a prompt-injection string trying to override instructions, a request to exfiltrate a system prompt, raw PII that should never enter the model at all. Prompt injection has sat at LLM01 in the OWASP Top 10 for LLM Applications since the list was published, and this is the rail it lands on first. Catching it here is cheaper than catching it in the output.

The retrieval rail sits between your retriever and the model. In a RAG or agent setup, it filters what context is allowed to enter the prompt. If a document pulled from the vector store carries a stale secret or an injected instruction, the retrieval rail strips it before it reaches generation. It is the rail people skip, because retrieval feels internal and trusted, and it is exactly where indirect prompt injection lands.

The tool-call rail gates actions. When an agent decides to call a function, hit an API, or run a shell command, this rail checks the call against policy before it fires. A force-push to a main branch, a migration on a production table, a payment above a threshold: these are stopped or routed to approval here, not after the damage.

The output rail runs last, on the generated text before it reaches the user. It masks PII the model echoed back, enforces a required JSON schema, and blocks unsafe or off-policy language. This is the rail most people mean when they say guardrails, but it is only one of four.

Read left to right, the rails form a pipeline: clean the input, filter the retrieval, gate the action, check the output. A gap at any point is an unguarded path, which is why agent safety needs all four together rather than a lone output filter.

For the production build of each rail, see our ultimate guide to LLM guardrails. Our write-up on self-correcting agent loops shows how these checkpoints feed corrective retries.

Where evals run: offline, in CI, and on sampled traffic

Evals have their own three operational positions, and they line up with the software lifecycle rather than the request path. Miss any one and a class of regression goes unwatched.

Offline evals run during development against a golden set: a curated collection of inputs with known good outputs. You change a prompt, rerun the set, and see whether the score moved. This is your fast feedback loop, and it runs hundreds of times before anything ships.

CI evals run as a release gate. The same golden set, or a larger held-out set, executes on every pull request or before every deploy, and a drop below threshold fails the build. This is where an eval stops a regression from reaching production at all, and it is the cheapest place to catch one. Our agent eval harness walkthrough covers wiring this gate.

Online evals run on sampled production traffic. You cannot score every live request without cost and latency, so you sample a percentage, grade it asynchronously, and watch the trend. This is how you catch drift that only shows up against real inputs: a model that quietly degrades as user behavior shifts under it. None of these three intercept a single request in flight. They observe, score, and report, and that is the one thing guardrails cannot do.

What Do Guardrails Catch That Evals Miss?

Each control has a blind spot baked into how it works, and the blind spots point opposite ways. Swap one for the other and you open a gap, not a redundancy you can trim.

Evals sample, so they miss the rare live outlier. Say your online eval grades 2% of traffic. The arithmetic is unkind: a failure that happens once in ten thousand requests has roughly a 2% chance of ever landing in the graded slice, so you will usually see it first in a support ticket, not a dashboard. It still reached the user who sent it. The eval is not wrong. It is measuring the population, and a one-in-ten-thousand event is invisible at the population level until it is a headline.

That is the same gap, viewed from the other side, that we walk in why your agent passes evals and fails in production. That post is about the eval set going stale against a moving world. This one is about the eval set being structurally unable to see a single request, even when it is perfectly current.

Guardrails act per request, so they miss the trend. An output rail can block every response containing an unmasked SSN, and it will do that on request one and request one million. But it has no memory and no aggregate. If the model’s answer quality has been sliding for three weeks because an upstream retrieval index went stale, the guardrail sees nothing wrong, because each individual response is still well-formed and policy-compliant. Its job is to enforce rules; judging quality falls to the eval.

So the rare catastrophic request needs a guardrail, and the slow quality decay needs an eval, and neither instrument covers the other’s failure. This is also why post-incident work pulls from both signals, the blocked-request log and the eval trend line. Our roundup of AI agent error analysis tools walks through reading them side by side.

Do You Need Guardrails, Evals, or Both?

The practical question is never guardrails or evals in the abstract. It is which control this specific action needs, and the cleanest way to decide is by blast radius: how much damage the action can do if it goes wrong, and whether you can undo it.

Low blast radius covers read-only and draft actions. Summarizing a document, drafting text a human will review, answering a question with no side effects. If this goes wrong, the cost is a bad output someone catches and discards. Measure it with evals and skip the runtime block. A guardrail here just adds latency to an action that cannot hurt anything.

Medium blast radius covers writes to non-critical state. Inserting a row into a scratch table, updating a cache, posting to a low-stakes internal channel. A mistake is recoverable but annoying. Measure with evals and add an alert, so a spike in bad writes pages someone, while you avoid blocking every call in the path.

High blast radius covers the irreversible. Moving money, running a production migration, deploying, force-pushing to a main branch, sending a message a customer will see. You cannot take these back. Here you want all three layers: an eval so you know the trend, a hard runtime guardrail so the dangerous call is gated live, and human approval for the top tier. The migration from the opening incident lives here.

The rule: let blast radius pick the controls, not habit. Do not wrap a read-only summarizer in three rails, and never let a production migration through on evals alone. The reliability failures worth reviewing almost always trace back to a mismatch between the risk of an action and the weight of the control guarding it. This tiering is also what keeps AI guardrails affordable, because you spend the expensive controls only where an action can cause real harm.

Implementing an output guardrail

To make the output rail concrete, here is a small one you can read in full. It does two jobs a real output guardrail does: it masks PII the model echoed back, and it optionally enforces that the response is valid JSON. The important design choice is what it returns.

import re, json

EMAIL = re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+")
SSN = re.compile(r"\b\d{3}-\d{2}-\d{4}\b")

def guard_output(text, require_json=False):
    """Output rail: mask PII, optionally enforce JSON, return an enforcement decision."""
    cleaned = SSN.sub("[REDACTED_SSN]", EMAIL.sub("[REDACTED_EMAIL]", text))
    if require_json:
        try:
            json.loads(text)
        except json.JSONDecodeError:
            return False, "block:invalid_schema", cleaned
    leaked = text != cleaned
    return (not leaked), ("mask:pii" if leaked else "pass"), cleaned

if __name__ == "__main__":
    print(guard_output("contact bob@acme.io or 123-45-6789"))
    # -> (False, 'mask:pii', 'contact [REDACTED_EMAIL] or [REDACTED_SSN]')
    print(guard_output('{"ok": true}', require_json=True))   # -> (True, 'pass', '{"ok": true}')
    print(guard_output("not json", require_json=True))        # -> (False, 'block:invalid_schema', 'not json')

First, the function returns a decision, not just a boolean. The tuple is (allowed, action, payload): whether the response passes, what the guardrail did about it, and the cleaned text to send. A boolean alone would tell you pass or fail and throw away the two things you need next, what enforcement happened for your logs, and the safe payload to forward.

Second, masking and blocking are different actions, and the return value distinguishes them. When PII is found, the guardrail does not reject the response. It masks the offending spans and lets a cleaned version through, tagged mask:pii. When required JSON fails to parse, it cannot clean its way out, so it returns a hard block:invalid_schema. A guardrail that treated every violation as a block would throw away recoverable responses, and one that treated every violation as a mask would let malformed output ship.

Third, the payload comes back cleaned regardless. Even on a block you get the redacted text, so nothing downstream, including your logs, ever sees the raw SSN. It is a small detail that carries a real compliance consequence.

This is deliberately minimal. Production output rails add toxicity and jailbreak classifiers, groundedness checks against the retrieved context, and rate controls, and those heavier checks are exactly the ones you reserve for higher-risk paths per the tiering above. But the shape holds at every scale: inspect the output, decide allow, mask, or block, and return both the decision and a safe payload. An enforcement control that returns only true or false is not finished.

How Future AGI runs guardrails and evals together

This pairing, enforcement in the path and measurement across the trend, is how Future AGI is built to run. Protect is the enforcement half, and it covers all four rails rather than the output filter people usually mean. It ships 28 guardrail checks: 10 first-party, including pii-detector, injection-detector, secrets-detector, system-prompt-protection, and data-leakage-prevention, plus 18 provider-backed ones such as llama-guard, lakera-guard, presidio-pii, and bedrock-guardrails.

Two of those provider-backed checks are the tool-call rail: tool-permissions validates the function or tool the model wants to invoke, and mcp-security screens MCP traffic, so the migration from the opening incident is gated at the layer that fires it, not after. The retrieval rail is the same input-stage checks pointed at retrieved chunks, since the Protect SDK screens any string you hand it, model output, user message, or a document you just pulled from a vector store.

Each check takes a stage, pre, post, or both, defaulting to pre, and an action, block, warn, mask, or log, which is the same allow/mask/block trichotomy the code above returns. The post stage is worth dwelling on, because it is the honest answer to “how do you stop an attack you have never seen.” You cannot recognise a novel jailbreak on the way in. You can screen what the model produced on the way out, before it reaches the caller, and refuse to return it.

One default cuts the other way, so know it before you rely on it: Fail Open is On, which means a verdict that does not return inside the timeout lets the request through unchecked. That is the right default for availability and the wrong one for a high-blast-radius path. Turn it off there.

Enforcement also leaves a record. Each request log carries a Guardrail Triggered flag alongside latency, tokens, and cost, which is the blocked-request signal the post-incident section above pairs with the eval trend line. The Command Center guardrails docs walk through wiring it into an agent.

The measurement half runs over the same traffic. Online evals score a sampled slice of production spans on a schedule, which is the trend line a guardrail structurally cannot produce, and Error Feed reads a sample of those traces, decides unaided what went wrong, and groups traces with the same failure into one issue instead of a thousand alerts. Built-in checks such as context adherence grade groundedness; anything specific to your task you write as a custom eval with your own rule and threshold.

The reason to run both on one system is boring and practical: the blocked-request log and the eval trend line describe the same requests, so a post-incident review reads one timeline instead of joining two exports. The platform is open source on GitHub, so you can sign up and run it managed or deploy it inside your own environment when the request logs cannot leave.

Building a reliability stack, not choosing a side

Step back and the versus in the title falls apart. Measurement and enforcement are not two options competing for the same slot. They are two layers of one reliability stack, and the useful question is not which to adopt but which layer each action needs. Evals give you the trend line and the release gate. Guardrails give you the live intervention. A system with only evals ships confident and gets surprised by the outlier. A system with only guardrails blocks the outlier and never notices it has been getting worse for a month.

Return to the migration that dropped the column. It passed every eval because the suite measured the population, and this was a single high-blast-radius action. What it needed was a tool-call guardrail gating the migration and a human approval on an irreversible change. Evals would still have caught a slow decline in migration quality over time. The two questions were never competing. They were the trend and the moment, and a real reliability stack answers both.