What Is Programmatic AI Assessment?

What Is Programmatic AI Assessment?

Programmatic AI assessments are automated, code-defined evaluations of an AI system’s outputs. They run on a dataset or a live trace stream and return per-row scores without human review of each item. Common categories include correctness checks (ExactMatch, FuzzyMatch), safety evaluators (Toxicity, PII), retrieval quality metrics (Groundedness, ContextRelevance), schema validators (JSONValidation, SchemaCompliance), and agent-trajectory metrics (ToolSelectionAccuracy, TaskCompletion). Unlike manual review, programmatic assessments are reproducible, versioned, and fast enough to run on every commit, every prompt change, and every percent of production traffic.

Why It Matters in Production LLM and Agent Systems

Manual review does not scale past a few hundred examples. Production LLM systems generate millions of outputs per day; without programmatic assessments, the only feedback signal is user complaints, and most users do not complain — they leave. Programmatic assessments turn evaluation into infrastructure: every release gets the same checks, every regression is detected within minutes, and every compliance question can be answered with a query rather than a forensic excavation.

The pain when assessments are not programmatic is concrete. A team ships a prompt change on Friday and discovers Monday morning that JSON output is broken for 4% of users — a JSONValidation assessment in CI would have caught it before merge. A RAG team rolls a new chunking strategy and watches faithfulness silently degrade because their golden set runs once per quarter. A compliance lead is asked to attest that no PII leaks in production and has only sample notes from the last manual review three months ago.

In 2026 multi-step agent stacks the case is even stronger. A trajectory has dozens of decision points — tool selection, parameter extraction, plan revision, tool output parsing — each capable of failing silently. Step-level programmatic assessments tied to OpenTelemetry spans surface where in the trajectory the failure occurred, not just that the final answer was wrong. This is why FutureAGI treats programmatic assessment as the foundational layer of LLM operations.

How FutureAGI Runs Programmatic Assessments

FutureAGI’s approach is to give programmatic assessments three production-grade properties: reproducibility, breadth, and gated deploys.

Reproducibility. Every assessment is a class in fi.evals. The engineer attaches it to a versioned Dataset via Dataset.add_evaluation(). Results are stored against the dataset run id and the prompt commit; rerunning produces the same result, and diffs against prior runs surface regressions.

Breadth. The fi.evals package exposes 50+ built-in assessments across cloud-template evaluators (Groundedness, AnswerRelevancy, Toxicity, PII, ContentModeration), local metrics (ExactMatch, FuzzyMatch, EmbeddingSimilarity, JSONValidation, SchemaCompliance, ROUGEScore), trajectory metrics (ToolSelectionAccuracy, TaskCompletion, GoalProgress, StepEfficiency), and security detectors (SQLInjectionDetector, HardcodedSecretsDetector). Custom rubrics wrap as CustomEvaluation with a few lines of code.

Gated deploys. Each assessment returns a score; the engineer sets a metric-threshold and configures CI to fail the build if the threshold is missed. In production, the same assessments run against a sample of live traces via traceAI, and eval-fail-rate-by-cohort becomes a first-class dashboard signal.

A real workflow: a coding-agent team attaches JSONValidation, ToolSelectionAccuracy, and TaskCompletion to a Dataset of 500 representative tasks, gates merge on score-deltas vs. the prior commit, samples 5% of production traces with the same evaluators online, and pages an engineer when the live failure rate diverges from the offline rate. Unlike Ragas, which focuses on RAG-specific metrics, FutureAGI’s programmatic assessment surface covers eval, RAG, agent trajectories, security, and compliance in one stack.

How to Measure or Detect It

Programmatic assessments are measured through their own coverage and result patterns:

• Assessment-fail-rate-by-cohort: percentage of evaluated rows that miss threshold, sliced by user cohort, model, or prompt version.
• Score distribution shift: KS-distance between today’s assessment scores and last week’s; flags subtle regressions point estimates miss.
• Coverage rate: percentage of releases gated by a programmatic assessment; below 100% means parts of the system ship un-evaluated.
• Time-to-detect: minutes from a regression entering production to an assessment firing; aim for single-digit on critical paths.
• Human-vs-programmatic agreement: per-evaluator agreement rate against a small human-labeled cohort; below 85% means the evaluator is the wrong tool for that rubric.

from fi.evals import Groundedness, JSONValidation

groundedness = Groundedness()
schema = JSONValidation()
result = groundedness.evaluate(
    input="What is the order status?",
    output="Order #123 is shipped.",
    context="Order #123: status=shipped"
)
print(result.score, result.reason)

Common Mistakes

• Treating one assessment as the answer. A single score hides which failure mode fired; track per-evaluator distributions.
• Running assessments only offline. Production traffic carries shifts the golden set never sees; wire the same evaluators to live traces.
• Ignoring agreement-with-humans calibration. A programmatic evaluator that disagrees with reviewers on 30% of cases is a coin flip in disguise.
• Letting the judge model and the generator be the same model. Self-evaluation inflates scores; pin judge to a different model family.
• No threshold, no alert. An assessment that runs but never blocks a deploy or pages an engineer is a vanity metric.