LLM Eval Data Drift Detection in 2026

The CI eval gate has been green for six weeks. Faithfulness sits at 0.87, task completion at 0.91. Then a customer-success thread surfaces a hallucination on a query about a feature you shipped in March. You pull the trace. The query is well-formed, the retrieval looks plausible, the answer is wrong. You go looking for the failure mode in the eval dataset. It is not there. Eight percent of production traffic this month is asking about that feature, and the golden set has zero examples of it. The eval was never wrong about what it tested. It tested the wrong product.

That is eval dataset drift. Your golden set was representative when you built it. By the time the CI gate runs in May, the production query distribution has shifted, the prompt template has moved, the retrieval corpus has rotated, and the eval is testing a frozen snapshot of yesterday’s traffic. This guide is the engineering methodology for catching it across the three places the dataset actually rots.

TL;DR: three drifts, one dataset, four-step refresh

Refresh cadence: monthly default, weekly under fast change, immediately after a vendor model bump or corpus re-index. Treat the golden set as a versioned artifact (golden-v3-2026-05), not a frozen one.

Why eval datasets drift (and the score does not warn you)

A golden set is a snapshot. Production is a distribution. The snapshot was a representative sample of the distribution on the day you authored it. Every day after, the distribution moves and the snapshot does not. By the time the gap is big enough to matter, the eval score has not moved at all, because the eval is still scoring the snapshot against itself.

Three forces drive the drift. New product surface adds intents the dataset never anticipated. Marketing campaigns and regional rollouts shift the persona and language mix. Users discover prompt patterns (longer queries, voice-to-text input, multi-step asks) that the original curators did not imagine. None of this breaks the API. None of it shows up in the dashboard. It just makes the golden set less and less representative of the system you ship to.

The frame that matters here is the same one in why your agent passes evals and fails in production: the eval was honest about an input distribution that no longer matches production. The fix is not a better rubric. The fix is dataset-drift detection that watches the eval stay calibrated to the world it scores. Golden-set construction is covered in the golden set design guide; this post is the operational sequel.

Drift 1: input-distribution drift

Input-distribution drift is the dataset-rot you can imagine before you measure it. Your golden set has 200 prompts spanning the intents you knew about at authoring time. Production users find an intent you did not anticipate. A persona shifts because a new campaign landed. A regional rollout brings prompts in a language whose tokenization stresses the model differently. Edge cases that were 0.5 percent of traffic on launch day are now 8 percent of traffic.

The signal is the cosine distance between two embedding centroids. Compute one centroid for the golden set on the same embedder you use for retrieval. Maintain a rolling 7-day centroid over production traces. Track the distance. When it crosses two standard deviations of a 30-day baseline, the distributions have separated and the dataset has stopped covering the long tail.

Add a KL divergence on intent labels for sharper attribution. If you tag traces with tag.tags (a traceAI span attribute) bucket the last 7 days by tag and compute KL against the golden-set tag distribution. If you do not tag, run HDBSCAN over the production embeddings nightly and watch for new clusters appearing in the last 7 days with zero representatives in the golden set. A new cluster with 50 traces and no golden-set neighbor is dataset rot you can ship to Linear before users complain.

from fi_instrumentation import register, ProjectType
from openinference.instrumentation.openai import OpenAIInstrumentor

trace_provider = register(
    project_type=ProjectType.OBSERVE,
    project_name="checkout-assistant-prod",
)
OpenAIInstrumentor().instrument(tracer_provider=trace_provider)

With traceAI capturing every span, the trace store doubles as the input-distribution feed. Span attributes carry session.id, user.id, and tag.tags, which gives you the slicing surface to ask “did the embedding centroid for the refund intent drift more than for the pricing intent” without a separate pipeline. The instrumentation walkthrough is in instrument your AI agent with traceAI.

The deeper landscape view sits in the best AI drift detection tools 2026, which compares the tooling across all five drift types you eventually need to monitor.

Drift 2: prompt-template drift

Prompt-template drift is the silent one. Your golden set is 200 well-curated inputs. The thing the LLM actually sees is those inputs wrapped in a system message, a few-shot block, and a tool schema. The wrapper is the prompt template. The dataset is not.

Here is how it bites. You shipped a system-message tweak on Friday to fix a tool-call failure on the checkout flow. The fix is good. The eval dataset still wraps inputs in last quarter’s template. Monday morning the CI gate regrades the dataset under a contract that no longer matches what production sees. The rubric scores something neither the user nor the PM cares about, and the score stays plausible because the rubric was generic enough to grade either contract.

Detection is conceptually simple. Hash the active prompt template nightly. Compare against the template hash stored alongside the eval dataset. Alert on mismatch. Pin every dataset version to the template hash it was authored against:

import hashlib

def template_hash(system_msg: str, few_shot: list, tool_schema: dict) -> str:
    canonical = system_msg + "\n".join(few_shot) + str(sorted(tool_schema.items()))
    return hashlib.sha256(canonical.encode()).hexdigest()[:12]

# Pin dataset version to template hash
dataset_meta = {
    "name": "checkout-golden-v3",
    "version": "2026-05",
    "template_hash": template_hash(SYSTEM_MSG, FEW_SHOT, TOOL_SCHEMA),
}

The remediation is dataset versioning. The Future AGI Dataset API is named-collection-with-versioned-rows: create checkout-golden-v3-2026-05 as a new collection when the template ships, add the labeled rows under the new template, and re-baseline the CI gate against the new version. Keep checkout-golden-v2-2026-04 around for at least one release cycle so you can A/B the two scores and tell whether the change reflects model behavior or dataset rot.

The wider dataset-management surface is covered in the best LLM dataset management tools 2026. The eval-stack way to wire this end-to-end is in the open-source evaluation library overview.

Drift 3: retrieval-corpus drift

Retrieval-corpus drift is the RAG-specific one and it is the meanest of the three because the eval score barely moves while it happens. Your golden set has fixed inputs and labeled expected outputs. The retrieval corpus is not part of the dataset. The retriever is configured against an index that grows, re-embeds, rotates sources, and tunes chunker parameters on a cadence the dataset does not see.

Concretely: the retriever you evaluated in March indexed 12,000 documents at chunk size 800. By May the index has 38,000 documents, the chunker re-embedded on a fresh model, and the same golden-set query lands on different top-k chunks. The generator faithfully grounds in whatever it was handed, so groundedness still scores 0.94. The user gets a confident-sounding answer drawn from a doc that does not apply to their case.

The signal that catches this is overlap between today’s top-k and the top-k captured when the dataset was authored. Snapshot the retrieved chunks for every golden-set query at dataset authoring time. On every refresh cadence, re-retrieve and compute BM25 overlap or cosine embedding overlap between the new top-k and the baseline:

def topk_overlap(baseline_chunks: list, current_chunks: list) -> float:
    baseline_ids = {c["doc_id"] for c in baseline_chunks}
    current_ids = {c["doc_id"] for c in current_chunks}
    if not baseline_ids:
        return 0.0
    return len(baseline_ids & current_ids) / len(baseline_ids)

# Alert when overlap drops more than 40 percent on the golden set
for query, baseline_topk in golden_set_topk_snapshot.items():
    current_topk = retriever.retrieve(query, k=5)
    if topk_overlap(baseline_topk, current_topk) < 0.6:
        alert(f"retrieval-corpus drift on '{query}'")

Pair the overlap signal with the split rubric layer the agent observability vs evaluation breakdown walks out. Score the retrieval rubrics (ContextRelevance, ChunkAttribution, ChunkUtilization) separately from the generation rubrics (Groundedness, FactualAccuracy). When context relevance drops with groundedness steady, the retriever moved. When groundedness drops with context relevance steady, the generator moved. One bisect instead of three days.

Detection methods: the three signals on one page

Three drifts, three primary signals. Run them on the same nightly batch and the eval team reads one dashboard instead of three.

Embedding cosine distance. Centroid of the golden set on the same embedder as retrieval, rolling 7-day centroid on production traces, cosine distance plotted over a 30-day baseline. The fastest signal to wire and the one that catches input drift before any score moves. Pin the embedder version explicitly so you are not chasing your own tail when the provider rotates text-embedding-3-small underneath you.

KL divergence on intent or topic labels. Bucket the last 7 days of production traces by tag.tags or by HDBSCAN cluster ID, compute KL against the golden-set distribution. Alert on movement past a baseline. The HDBSCAN variant is the right pick when you do not have a tag taxonomy yet, because the unsupervised cluster output is also a debugging surface.

Top-k overlap. BM25 overlap on document IDs is the fastest, embedding-cosine overlap on chunk vectors is more sensitive. A 40 percent drop on the golden set is a strong signal that the retrieval surface moved. The threshold is domain-specific, so calibrate it on a quiet week of traffic.

The classical statistical machinery underneath these three signals (KS, PSI, Wasserstein) is laid out in model vs data drift and the broader LLM-side framing is in what is LLM drift.

The refresh protocol: monthly, four steps

The detection signals tell you when the dataset has rotted. The refresh protocol is what you do about it. Monthly is the default cadence for production LLM applications. Weekly is right when traffic is changing fast (new product launch, regional rollout, marketing push). Immediately is right after a vendor model bump or a corpus re-index. The same four-step protocol applies regardless of cadence.

Step 1: stratified sample from production. Pull 200 to 500 recent production traces. Stratify by intent (so the new intents are over-represented), by persona, and by language. Pull from the last 14 to 28 days, not the last 7, so you do not over-fit the dataset to a single campaign or rollout.

Step 2: label against the active rubric. Apply the rubric (Groundedness, ContextRelevance, FactualAccuracy, AnswerRefusal, whatever is in CI) and hand-correct the labels. For applications with subject-matter complexity, route this through an annotation queue with SME reviewers; for simpler surfaces, a stronger judge model held to a higher bar will get you most of the way.

Step 3: version the dataset. Create golden-v4-2026-06 as a new named dataset (not an overwrite). Pin the prompt-template hash, the embedder version, and the retrieval-corpus snapshot to the dataset metadata. Keep the previous version (golden-v3-2026-05) live for at least one release cycle for A/B comparison.

Step 4: re-baseline the CI gate. Run the CI eval against the new dataset, capture the new baseline scores, update the gate thresholds. The score delta between v3 and v4 on the same model tells you how much of the prior “regression” was dataset rot vs. real model drift. Most teams find that 30 to 50 percent of the score movement between refreshes was the dataset, not the model.

# Refresh protocol — minimal Future AGI wiring
from fi.datasets import Dataset, DatasetConfig
from fi.utils.types import ModelTypes

# Step 3: version the dataset
config = DatasetConfig(
    name="checkout-golden-v4-2026-06",
    model_type=ModelTypes.GENERATIVE_LLM,
)
dataset = Dataset(dataset_config=config).create(source="labeled_traces_2026-06.csv")

# Step 4: re-baseline with the eval surface attached
from fi.evals.templates import Groundedness, ContextRelevance

dataset.add_evaluation(
    eval_templates=[Groundedness(), ContextRelevance()],
    name="ci_baseline_v4",
)

The wider monitoring surface that this protocol sits inside is in the production LLM monitoring checklist 2026 and the LLM evaluation playbook 2026.

How FAGI grounds the dataset-drift layer

Detection only works if the data plumbing is already in place. The Future AGI stack lines up the three feeds the protocol depends on.

traceAI as the production trace feed. With register(project_type=ProjectType.OBSERVE, project_name=...) and the per-framework instrumentors (OpenAIInstrumentor, LangChainInstrumentor, LangGraphInstrumentor, LlamaIndexInstrumentor, plus the rest of the 50-plus AI surfaces across Python, TypeScript, Java, and C#), every prompt, response, retrieval, and tool call lands as OTel spans with session.id, user.id, tag.tags. The same trace store you use for debugging is the input-distribution feed for the embedding-centroid signal and the HDBSCAN clustering pass.

Dataset API for versioned golden sets. Dataset in futureagi-sdk is name-and-version oriented. Create checkout-golden-v4-2026-06 as a new collection, pin the template hash and corpus snapshot to the metadata, attach the rubric via add_evaluation. The previous version stays live for A/B baselining. The wider landscape is in the dataset management tools 2026 roundup.

ai-evaluation for the rubric layer. 60-plus EvalTemplate classes (Groundedness, ContextRelevance, ChunkAttribution, ChunkUtilization, FactualAccuracy, AnswerRefusal, TaskCompletion, and the rest). Split the retrieval rubrics from the generation rubrics on the same run so the retrieval-corpus-drift bisect is a query into the same store, not a separate pipeline.

Error Feed as the dataset-gap surface. HDBSCAN soft-clustering over span embeddings runs nightly. A new cluster with zero golden-set neighbor is the cluster-shaped version of input-distribution drift. The Sonnet 4.5 Judge writes an immediate_fix annotation per cluster: “new intent pattern, customers asking about Q3 features, golden set lacks coverage.” That annotation routes into Linear as a ticket whose accept criteria is “stratified-sample these 50 traces into checkout-golden-v5.” Slack, GitHub, Jira, and PagerDuty are on the roadmap.

Honest framing: the trace-stream-to-dataset connector that auto-promotes drift clusters into the next dataset version without the manual sample-and-label step is on the active roadmap, not shipped. Today the cluster annotation is the trigger; the four-step refresh is the human response.

Anti-patterns that hide dataset drift

Four anti-patterns account for most of the dataset rot teams discover only after the fact.

Treating the golden set as ground truth instead of a versioned artifact. The dataset gets built once, reviewed by the team, declared canonical, and then never moves. Six months in, it is a snapshot of last quarter’s product. The fix is a name-and-version discipline (golden-v1, golden-v2) baked into the dataset API and a monthly refresh cadence on the calendar.

No input-distribution monitoring. The embedding centroid drift signal is small to wire, and the cost of skipping it is enormous: the production distribution shifts silently and the dataset stops covering the long tail with nobody watching. The embedding compute is already running for retrieval. Reuse it for the drift signal.

Prompt templates and eval datasets versioned in separate places. The system message lives in a YAML file that ships through CI on merge. The eval dataset lives in a different store reviewed by a different on-call. A template ship lands while the dataset still grades the old contract. The fix is to pin a template hash to the dataset metadata and alert on mismatch.

No retrieval-corpus snapshot. RAG-side dataset rot is silent until users complain. Snapshot the top-k document IDs for every golden-set query at authoring time, store them with the dataset, and monitor overlap on every refresh cadence. The mechanic is straightforward and almost no team does it.

Closing: the dataset that watches itself

The teams shipping reliable LLM applications in 2026 are not the ones with the largest golden sets. They are the ones whose datasets refresh on a cadence. Input drift moves the distribution, prompt-template drift moves the contract, retrieval-corpus drift moves the grounding. Three drifts, three signals, one monthly refresh protocol.

Build the three baselines on day zero: embedding centroid, template hash, top-k snapshot. Run the nightly rolling-window comparison. Version every refresh (golden-v3-2026-05). The drift will still arrive. You will just see it the sprint it shows up, instead of the quarter after.

For the closing-the-loop side, the LLM eval feedback loop design 2026 walks the path from in-product feedback to dataset refresh. For the production-side companion, 12 metrics for AI conversation monitoring lays out the metric set, and agent passes evals and fails in production covers the failure mode dataset-drift detection is designed to prevent.