Rethinking Verifier Design for Computer-Use Agents

Your agent books the flight. The final screenshot shows a confirmation page, the grader marks the run a pass, and the next morning the customer’s card has been charged twice. Who grades the grader that missed it?

Every evaluation rests on one assumption: the grader is right. For a computer use agent, that is exactly where most eval setups quietly break. The agent acts through a real interface over many steps, not a single answer you can check against a reference.

So a grader that checks only the final screen can pass a run that charged a card twice, deleted the wrong file, or clicked through a dialog it should have refused. When the grader is wrong, every score built on it is a guess.

What computer use agents are and why grading them is hard

Computer use agents are AI agents that operate a graphical interface the way a person does: they read the screen from a screenshot, then issue clicks, keystrokes, and scrolls to get a task done. Anthropic’s computer use and OpenAI’s operator-style agents work this way, perceiving the screen and acting on it in a loop.

That design is what makes grading them hard. A chatbot produces one answer you can score against a reference. A computer use agent produces a trajectory, a sequence of actions over many steps, and the thing you care about is whether the whole sequence reached the right end state without causing harm on the way.

Outcome-only grading misses most of that. Check the final screenshot and you learn whether the last page looks right, not whether the agent took a reckless path to get there. Two runs can end on the same confirmation page, one clean and one that double-submitted a form, scored identically.

Real computer use agent evaluation has to grade the path, not just the destination.

This is why cua evaluation is its own discipline. Public benchmarks run agents in real environments: OSWorld on a real desktop, WebArena and VisualWebArena on self-hosted sites, WebVoyager on live pages, and AndroidWorld on mobile.

They all wrestle with the same question: how do you check that an agent operating a live interface did the task, and only the task?

Why programmatic verifiers break on computer-use agents

The first instinct is a programmatic verifier: hard-coded checks that read the final state and assert what should be true. Did the order appear in the database, is the file in the right folder, does the confirmation number exist. When they work, they are fast, deterministic, and cheap.

They break on the interface. UIs drift: a button moves, a field is renamed, a flow adds a step, and a check pinned to the old layout starts failing runs that actually succeeded. Now you have false fails, and every one costs an engineer time to investigate and a rule to rewrite.

The reverse is worse. A rigid check can pass a run that went wrong in a way it does not look at. It confirms the order exists but not that it is the right order; it sees a confirmation page but not the duplicate charge behind it. Those false passes are the failures you built the verifier to catch.

And the maintenance never ends. Every app change can break a check, so a programmatic suite for a real interface becomes a second codebase you maintain alongside the agent. It is necessary for the things that are genuinely rule-checkable, but on its own it is brittle, and brittle graders produce scores you cannot trust.

Why an LLM or VLM judge is not automatically better

The usual fix is to swap the rigid check for judgment: an LLM as a judge, or a vision-language model that grades the screenshots and the trajectory against a rubric. A judge handles UI drift gracefully and can reason about whether the path made sense.

That is why benchmarks like WebVoyager use a vision model to score runs, and the pattern is common enough that eval SDKs ship it as an LLM or VLM judge you point at your own rubric.

But a judge is not automatically right, it just moves the problem. Model judges carry well-documented biases: they favor longer answers, lean toward their own style, and are swayed by the order the options appear in.

A subtler failure shows up when the judge reads only the final answer. It can mark a correct result wrong because an earlier step confused it, or pass a wrong one because the summary read well.

Rubric ambiguity is the quieter failure. Two reasonable people reading your grading prompt disagree on borderline cases, so the model does too, and scores wobble run to run. There is also the plain cost: a judge call per run adds latency and spend. An agent as a judge that inspects the whole trajectory helps, but inherits the same reliability questions.

Designing a verifier that resists reward hacking

Good verifier design assumes the agent will try to win the easy way. Reward hacking is when an agent earns a high score without doing the task, by exploiting a gap in the grader instead of solving it. If a check rewards reaching a confirmation page, an agent can learn to reach it without completing the booking behind it.

The defense is layers. Hard gates encode the non-negotiables, the things that must be true or the run fails outright: no charge without a matching order, no success claim without the record to back it. Softer, denser signals then grade quality on top, so the agent cannot farm points by gaming one loose metric.

Watch for the classic traps. Oracle reads, where the agent peeks at the answer or the success condition. Shortcut exploits, where it satisfies the letter of the check and not the intent.

The way to find these before the agent does is to mutation-test the verifier: feed it known-bad trajectories it should reject and confirm it does. Deterministic and judge-based checks get gamed differently, so a resistant design usually uses both.

None of this is about distrusting your agent. It is about not grading it with a ruler it can bend. A verifier that resists reward hacking is one you have actively tried to fool and could not, which is a very different thing from one that happened to pass on the runs you already saw.

How to validate the verifier itself

A verifier is the foundation of agent grading, and it is also a model, or a program, that can be wrong, so it needs its own evaluation.

The core question is agreement: when your verifier and a careful human grade the same runs, how often do they match? If they disagree a third of the time, the verifier is not grading, it is guessing.

Raw agreement overstates it, though, because some matches happen by chance. Cohen’s kappa corrects for that, measuring agreement beyond what two random graders would hit. The function below computes it between your judge and a human gold set, so you get one number for how much to trust the grader before you rely on it.

def cohen_kappa(judge, gold):
    """Agreement between an LLM judge and human gold labels; kappa corrects for chance."""
    assert len(judge) == len(gold) and judge
    n = len(judge)
    po = sum(j == g for j, g in zip(judge, gold)) / n
    labels = set(judge) | set(gold)
    pe = sum((judge.count(l) / n) * (gold.count(l) / n) for l in labels)
    return round((po - pe) / (1 - pe), 4) if pe != 1 else 1.0

def _demo():
    assert cohen_kappa(["pass", "fail", "pass"], ["pass", "fail", "pass"]) == 1.0  # perfect agreement
    assert cohen_kappa(["pass", "pass", "fail", "fail"], ["pass", "fail", "fail", "fail"]) == 0.5

if __name__ == "__main__":
    _demo()

A good kappa is necessary, not the whole picture. Spot-audit a sample of the verifier’s verdicts by hand, watch for scores that climb while real task success does not, and keep testing it against new failure modes.

Writing the judge prompt well matters too, since that is where most rubric ambiguity creeps in. Table B lists the checks worth running on any grader.

What does good evaluation for computer use agents look like in practice?

Put it together and good evaluation for computer use agents is layered, not a single grader. Hard programmatic gates catch the non-negotiables cheaply. A model judge handles the nuance the gates cannot express. And neither is trusted blindly, because both are validated against human judgment on a regular sample.

Human review stays in the loop, but sampled, not exhaustive. You cannot read every run, so you read a representative slice: a mix of passes and fails, weighted toward the high-stakes paths. That sample is what keeps the automated graders honest and catches the failure modes none of your rules anticipated.

Reproducibility matters too. Grading on your own private traces tells you about your agent; grading on a public benchmark like OSWorld or WebArena tells you how your setup compares and gives you a fixed target to tune against. Most teams want both: their own cases for relevance, a public one for a sanity check.

And the whole thing is continuous. Agents change, interfaces change, and a grader that was reliable last quarter drifts. Re-run the verifier audits on a schedule, so a grader that starts to slip gets caught the same week, not after a quarter of bad scores.

Building and checking CUA verifiers with Future AGI

The method in this post, gate the obvious, judge the nuance, and validate the grader, is what Future AGI’s evaluation stack is built for. With custom evals you write your own grading rule, pick a model judge or a deterministic check, and set the pass or fail threshold that defines a good run for your task.

The part that fits verifier design specifically is independence. In simulation, the agent, the simulator driving the scenario, and the verifier scoring it are kept separate, so the thing under test is not also grading itself. That separation is what makes a verdict mean something, rather than an agent marking its own homework.

For validating the grader, agreement metrics are built in: Cohen’s kappa for your verifier against a human gold set, and Fleiss’ kappa when three or more reviewers rate the same runs. You get the reliability number without writing the statistics yourself. The eval core is Apache-2.0 and self-hostable, so you can run these checks on your own infrastructure.

Grading computer-use agents you can actually trust

Go back to the flight that got booked, passed, and charged twice. A final-screenshot grader never had a chance; the damage was in the trajectory it did not read. A layered verifier catches it: a hard gate on no duplicate charge, a judge on whether the path made sense, and a kappa score telling you the grader can be trusted.

That is what grading computer-use agents you can trust takes. Gate the things that must be true, judge the things that need nuance, and audit the grader on a schedule so it never quietly drifts. Start by writing one custom eval for your agent’s riskiest task, then validate it against a handful of runs you have graded by hand.