What Is a Bias Metric?

What Is a Bias Metric?

A bias metric is a quantitative measure of disparate impact, stereotyped output, or unfair behaviour in an ML or LLM system. Common bias metrics include demographic-parity difference, equal-opportunity difference, calibration gap, refusal-rate ratio, and language-model-specific scores like StereoSet, CrowS-Pairs, and BBQ accuracy. In production LLM workflows, a bias metric is the number you threshold on, alert on, and gate releases against. FutureAGI exposes bias-specific evaluators (BiasDetection, NoGenderBias, NoRacialBias, NoAgeBias, Sexist) that return bias metrics on offline cohorts and live traces.

Why It Matters in Production LLM and Agent Systems

A bias metric is what turns “the model seems unfair” into “the model fails this cohort 4.2× more often.” Without the number, the conversation stays anecdotal — a screenshot of a bad output, a complaint from a single user — and the fix lands as ad-hoc prompt edits that may or may not generalise. With the number, you have a release gate, a regression alarm, and an audit artifact.

The pain is shared. Compliance leads need a metric for the disparate-impact section of an audit response. Product leads need it to compare model candidates on an even footing. Engineers need it to know whether a prompt change improved or regressed bias. SREs need it to know which alert to wire to which rotation.

In 2026-era stacks, bias metrics fragment by axis. A model can pass demographic-parity but fail equal-opportunity; pass StereoSet but fail BBQ; look unbiased on average but show 3× refusal asymmetry on a specific cohort. The teams that handle this well do not pick one bias metric — they pick a small set, segment by cohort, and dashboard each one. The EU AI Act’s high-risk classification effectively makes this mandatory for many regulated deployments, so bias metrics now sit inside both the legal and engineering stacks.

How FutureAGI Handles Bias Metrics

FutureAGI’s approach is to make bias metrics first-class evaluators that run on the same surface as quality and safety evaluators. The fi.evals package exposes BiasDetection (general bias score), NoGenderBias, NoRacialBias, NoAgeBias (axis-specific demographic-parity-style scores), and Sexist (stereotype detection). Each returns a 0–1 score with a reason string, can run offline against a Dataset or online against trace samples, and writes results back as a span_event for dashboard segmentation.

A concrete example: a credit-decisioning agent runs BiasDetection and NoGenderBias against an eval cohort built from real applications. The dashboard surfaces a 0.18 gap on NoGenderBias for a specific income band — the model is too cautious with female applicants in that band. The team isolates the failing trajectories, builds a regression test in the simulate-sdk’s Scenario library, retrains the prompt with explicit fairness instructions, and uses Agent Command Center’s traffic-mirroring to compare the new prompt against the old on live traffic before promotion. Unlike a static fairness audit run quarterly, FutureAGI keeps bias metrics live, cohort-segmented, and alertable. We have found that bias regressions land far more often through downstream prompt edits than through model swaps — which is why daily bias eval runs catch what release-gate eval misses.

How to Measure or Detect It

Pick the bias metrics that match your system and your audit obligations:

• BiasDetection evaluator: 0–1 general bias score with reason; the headline number.
• NoGenderBias, NoRacialBias, NoAgeBias: axis-specific demographic-parity-style evaluators.
• Sexist evaluator: stereotype-loaded-output detector.
• Demographic-parity difference: |P(positive | A) − P(positive | B)| across cohorts; classical fairness metric.
• Equal-opportunity difference: TPR gap across cohorts; the right metric when false negatives hurt one cohort.
• Refusal-rate ratio: refusal-rate(A) / refusal-rate(B); a stable LLM-specific bias signal sensitive to prompt rewrites.
• eval-fail-rate-by-cohort: dashboard signal segmented by every axis you care about.

A minimal NoGenderBias check on a generation:

from fi.evals import NoGenderBias

metric = NoGenderBias()
result = metric.evaluate(
    input="Describe a typical CEO",
    output="A CEO is usually a confident leader who...",
)
print(result.score, result.reason)

Common Mistakes

• Reporting one bias number. Different metrics measure different things; track demographic-parity and equal-opportunity separately.
• Picking the metric your model passes. It is tempting; it is also misleading. Pick metrics from the audit framework, not the optimisation result.
• Ignoring intersectional cohorts. Bias on (gender × language) often hides inside per-axis numbers.
• Letting the bias judge be the same family as the model. Self-evaluation under-reports; pin to a different family.
• No threshold, no alert. A bias metric that runs but never gates a release is a vanity metric.