From Traces to Experiments: A Loop for Improving AI Agents

You shipped three prompt tweaks to the checkout agent this month. Support says it feels worse. You have a month of traces and no way to prove which change to blame.

Those traces hold the answer to what is going wrong, but a folder of JSON is not a decision. Scrolling through runs gives you anecdotes, and an anecdote cannot tell you whether your next change helps or hurts.

TL;DR

• Traces tell you what to fix; experiments tell you whether a fix actually worked.
• Curate the interesting traces, the failing ones and a sample of normal ones, into an evaluation dataset.
• Test a change offline against that dataset first: fast, safe, and repeatable.
• Then run an online experiment on live traffic to confirm it holds, sized with real sample-size math.
• Ship the winner, keep watching, and feed the new failures back into the dataset.

Mining agent traces for what to fix

A trace is the full record of one agent run: the prompts, the tool calls, the intermediate steps, and the final output. Read a batch and patterns jump out, a tool that fails on certain inputs, a step where the agent loops, a class of request it handles badly.

That is the start of trace-driven experiments: the trace tells you what to test.

The move is to go from browsing to mining. Trace every run, then filter to the runs that matter: the failures, the slow ones, and a sample of normal ones for contrast. Attach eval scores where you have them, so you are sorting by what actually went wrong instead of reading text at random.

This turns a vague worry into a testable claim. Not a sense that checkout got worse, but a specific hypothesis: the agent picks the wrong shipping option when the cart has mixed items. A hypothesis that precise is something you can build a dataset around and prove or disprove, which is the whole point of the loop.

Offline evaluation and online experiments answer different questions

Once you have a hypothesis, there are two ways to test it, and they answer different questions. Offline evaluation scores a change against a fixed dataset before it ships. Online experimentation puts the change in front of real traffic and measures what happens. You want both, in that order.

Offline is fast, cheap, and safe: run the new prompt against your curated cases and compare scores, with no users at risk. It is also limited to the situations you captured. Offline evaluation tells you a change is probably good; it cannot tell you how real users will react.

That is what online LLM experimentation is for: an A/B test on live traffic that measures the effect, with the risk and the wait that come with real users. The table below lays out which one to reach for, and when.

Turning production traces into an evaluation dataset

The bridge between the two is a dataset. An evaluation dataset is a fixed set of inputs with the outcome you expect, so every agent version gets scored on the same cases. Built from traces, it reflects real usage instead of cases you imagined at your desk.

Build it in three steps. Select the traces that matter: the failures you found, plus representative normal runs so you do not overfit to disasters. Strip each to its inputs and the expected outcome. Then create a dataset and label it, and that set becomes your bar for every future change.

Keep it honest and current. A dataset frozen six months ago scores agents on problems you already solved. Add new failures as you find them, retire cases that no longer occur, and keep a mix of easy and hard so a single score still means something. The dataset is the asset that compounds, because every other step reuses it.

Running agent experiments that prove a change helps

With a dataset in hand, an experiment is simple in shape: take two variants, run both over the same cases, and compare. The function below scores variant A and variant B on each row and returns the means, the delta, and the winner. That compare step is the engine of the whole agent evaluation loop.

def run_experiment(dataset, variant_a, variant_b, score_fn):
    """Run two variants over a dataset, score each row, and compare means and delta."""
    if not dataset:
        raise ValueError("dataset is empty")
    a = [score_fn(variant_a, row) for row in dataset]
    b = [score_fn(variant_b, row) for row in dataset]
    ma, mb = sum(a) / len(a), sum(b) / len(b)
    delta = round(mb - ma, 3)
    winner = "B" if delta > 0 else ("A" if delta < 0 else "tie")
    return {"mean_a": round(ma, 3), "mean_b": round(mb, 3),
            "delta": delta, "winner": winner}

def _demo():
    data = [{"q": "a"}, {"q": "b"}]
    score = lambda v, row: 1.0 if v == "new" else 0.5  # the new variant scores higher
    out = run_experiment(data, "old", "new", score)
    assert out["winner"] == "B" and out["delta"] == 0.5, out

if __name__ == "__main__":
    _demo()

That offline delta is a green light, not proof. A higher score on fifty cases could be noise, so size the online test before you trust it live: how many requests per variant you need to call a real difference.

For a pass or fail metric across two equal arms, the standard approximation is n = K * [p1(1 - p1) + p2(1 - p2)] / delta^2, where K = (z_alpha/2 + z_beta)^2, about 7.85 at 80% power and alpha 0.05. Moving a success rate from 80% to 85% then needs roughly 900 requests per variant.

The pooled-variance form is slightly more precise, but this separate-variance version is the common, simpler one. Check significance before you call a winner, and keep the online test live long enough to hit that count. The offline side of the comparison runs as a dataset experiment on demand.

Turning one experiment into a repeating loop

One experiment is a fix. A loop is a practice. The agent improvement loop makes each change feed the next: ship the winner, keep watching it in production, and send the new failures you find straight back into the dataset.

That is what turns a one-off win into eval-driven development. A guardrail that catches a bad output becomes a new test case. A production regression becomes a dataset row. The same feedback loop that found the first problem now guards against it returning.

Run it once and you fixed a bug. Run it every week and the agent gets measurably better, because every failure you survive becomes a test the next version has to pass.

Running this loop end to end in Future AGI

The loop in this post is the one Future AGI is built around, so you can run it without stitching four tools together. Observe captures the traces, and you curate the interesting ones into a dataset without exporting them somewhere else first.

From that dataset you run an experiment, scoring variants against the same cases, then push the winner and watch it in production. Future AGI runs these stages as one connected loop, where production monitoring feeds your eval datasets and eval results inform the guardrails, so each stage hands work to the next instead of living in its own tool.

The step most teams skip is the last one, acting on the result. Optimization helps close it by turning what an experiment learned into the next candidate to test. Tracing, datasets, and offline evals run on the Apache-2.0 core you can self-host; the hosted Optimization product is a cloud feature, though its optimizer algorithms also ship in the open-source SDK to run locally.

From a pile of traces to agents that measurably improve

Go back to the checkout agent and the three tweaks no one could judge. With the loop, that stops being an argument. You pull the failing checkout traces, curate them into a dataset, and run each tweak against it, so the one that actually helps is a number, not an opinion.

That is the shift: from a pile of traces you cannot act on, to agents that measurably improve, one honest experiment at a time.

You do not need the whole system on day one. Take your loudest failure this week, turn it into five dataset cases, and run one experiment. That first measured delta is where improving your agent stops being a guess.