What Is a Unary Predicate?

What Is a Unary Predicate?

A unary predicate is a function that takes a single argument and returns a boolean. In first-order logic, P(x) is a unary predicate where P is the relation and x is the term. In LLM evaluation, the analogous construct is a single-argument boolean evaluator: take one output and return true/false. Most of FutureAGI’s deterministic evaluators — IsJson, IsPolite, OneLine, ContainsCode, IsEmail, IsCompliant — are unary predicates over the response. Their value comes from composability: a CustomEvaluation rubric or AggregatedMetric panel chains many unary predicates into a richer judgment without the cost of a single judge-model call.

Why It Matters in Production LLM and Agent Systems

LLM evaluation suffers from too much reliance on subjective judge-model graders. Judge models are useful but slow, expensive, and inconsistent across runs. Unary predicates are the deterministic counterweight — they run in microseconds, are perfectly reproducible, and surface failure modes a judge model often misses (a malformed JSON response that “looks fine” to a 70%-accurate judge prompt).

The pain of not using unary predicates shows up cleanly in production. A team grading agent outputs only with a judge model finds the judge accepts responses with broken JSON because the prose preamble looks reasonable; downstream parsers crash. A RAG team trusts a judge to grade citation presence; the judge hallucinates that citations were there. Adding a unary ContainsValidLink predicate catches these in one line of code with no model call.

For 2026-era agent stacks, unary predicates are essential at every step. A planner step output should be parseable JSON (IsJson), contain a known tool name (ContainsCode or a custom predicate), and respect a max-step budget. Each is a unary predicate; together they form a deterministic guardrail layer that no judge model can replicate at the same cost. Composability matters: AggregatedMetric lets you combine ten unary predicates into a per-step pass/fail rule.

How FutureAGI Handles Unary Predicates

FutureAGI’s evaluator catalogue is built on the observation that most useful checks are unary predicates. The platform exposes them as first-class objects.

Built-in predicates. fi.evals.IsJson, IsPolite, IsConcise, OneLine, ContainsCode, IsEmail, IsCompliant, IsHelpful, IsHarmfulAdvice, NoApologies, and around 30 others are deterministic or near-deterministic unary predicates over the response field. Each returns a boolean (or 0/1) plus a reason.

Custom predicates. CustomEvaluation lets you wrap any callable returning a boolean into a named evaluator. Write a regex check, a JSON-schema validator, or a simple Python function — register it, version it, and run it over a Dataset like any other evaluator.

Composition. AggregatedMetric chains predicates with weights. A team requiring “structured output, no PII, polite tone” composes IsJson + PII (negated) + IsPolite with weights summing to 1.

Concretely: a customer-support agent team uses a panel of seven unary predicates plus two judge-model evaluators per response. The unary predicates run in milliseconds and catch ~70% of failures (malformed JSON, missing fields, PII leaks) — only the residual 30% need expensive judge calls. The result is an evaluation pipeline that is both fast and faithful. Compared to running a single judge over every response, the predicate-first approach cuts eval cost by ~5x and improves reproducibility.

How to Measure or Detect It

Unary predicates produce boolean signals; treat them as binary classifiers:

• Predicate failure rate: percentage of rows where the predicate returns false.
• fi.evals.IsJson: returns boolean; IsJson.evaluate(output).score == 1 means valid JSON.
• fi.evals.Equals or Contains: simple boolean string predicates.
• fi.evals.CustomEvaluation: wrap any one-argument boolean function.
• Per-cohort failure rate: split predicate outcomes by route, model, or user segment.

Minimal Python:

from fi.evals import ContainsJson, OneLine

predicates = [ContainsJson(), OneLine()]
for predicate in predicates:
    result = predicate.evaluate(output=model_response)
    print(predicate.__class__.__name__, result.score)

Aggregate predicate-pass rates over a cohort to surface deterministic failure modes.

Common Mistakes

• Using only judge-model evaluators. Wastes money and misses deterministic failures cheap unary predicates would catch.
• Writing predicates that secretly call an LLM. Defeats the deterministic-counterweight value; keep them deterministic where possible.
• Composing predicates with bad weights. A predicate that fails 90% of the time dominates the aggregated score — calibrate weights against per-predicate base rates.
• Skipping reasoning fields. A predicate that returns false without a reason is hard to debug; always emit a one-line explanation.
• Confusing unary and binary predicates. Equals(response, expected) is binary, not unary — use the right primitive.