What Is Overfitting in Machine Learning?

What Is Overfitting in Machine Learning?

Overfitting in machine learning is when a model fits the noise and exact records of its training set so closely that it stops generalizing to new data. The symptom is a model that performs very well on training inputs and noticeably worse on a held-out set or in production. It applies across model families: a decision tree grown to depth 30, a deep network trained too long, or an LLM fine-tuned on a small domain corpus. FutureAGI handles the production-side symptom: regression evals, drift comparisons, and per-cohort fail rates.

Why Overfitting Matters in Production LLM and Agent Systems

A model that overfits looks competent on the demo. It collapses the moment it meets a phrasing the training set did not contain, a longer context, or a different language. ML engineers see it as a wide train-test gap; SREs see it as eval-fail-rate spikes after a fine-tune ships; product teams see it as a sudden rise in thumbs-down rate on a specific user cohort.

For LLMs the failure is subtler than classical ML. A 7B model fine-tuned on 5,000 internal Q&A pairs can pass the static eval set, then return verbatim training answers when a user asks an unrelated question that shares a prefix. Symptoms include unusually low perplexity on training prefixes, repetition loops on familiar phrases, and tool-call payloads that match training values too literally.

In 2026 agentic stacks the blast radius grows. An overfit planner can pick the same tool every time because it memorized the trajectory in training. An overfit retrieval reranker can prefer training corpus chunks even when fresher context is supplied. An overfit judge model gives high scores to outputs that mimic training examples instead of meeting the rubric. Each of these is a quality bug at first, then a regression-eval failure, then an incident.

How FutureAGI Detects Overfitting Symptoms

FutureAGI does not train models, so it does not regularize weights or pick a stopping epoch for you. The honest connection is that FutureAGI measures the outputs of models that may be overfit, then makes the symptom visible. The standard workflow is to store the candidate model and the previous baseline in fi.datasets.Dataset, attach evaluators with Dataset.add_evaluation, and run them on both a held-out canonical eval set and a freshly sampled production cohort.

Real example: a support team fine-tunes a Llama variant on six months of resolved tickets. Before release, the engineer replays the candidate against a golden-dataset and a production-mirror cohort. Groundedness checks whether the response is supported by retrieved policy text. AnswerRelevancy checks whether it actually answers the user. JSONValidation checks structured payloads to the case-management tool. FutureAGI records model.version, adapter.id, dataset.id, prompt, response, and per-evaluator scores. If the fine-tuned model wins on the golden set but loses on the production-mirror set, that gap is the overfitting signature; the team rolls back, broadens the training set, or applies stronger regularization.

Compared with running these checks in a notebook, the FutureAGI workflow ties scores to model and dataset versions so a regression has a clean owner.

How to Measure or Detect Overfitting

Read the gap between training, held-out, and production data.

• Train-test gap — the canonical signal. Widening gap during training is the classic warning.
• Per-cohort eval-fail-rate — split by language, length bucket, tool route, and user segment. Overfitting often shows up in one cohort first.
• Regression-eval delta — compare the candidate model against the baseline on the same Dataset and the same evaluator suite.
• Verbatim-overlap rate for LLMs — percentage of responses with n-gram overlap above a threshold against suspected training documents.
• Out-of-distribution probes — synthetic personas built with ScenarioGenerator that fall outside the training distribution.

from fi.evals import AnswerRelevancy

evaluator = AnswerRelevancy()
result = evaluator.evaluate(
    input=user_query,
    output=model_response,
)
print(result.score, result.reason)

Common Mistakes

• Training to lowest validation loss only. Loss can decrease while task quality regresses; gate on task evaluators, not loss.
• Using a test set that overlaps with training. Even small leaks inflate scores and hide overfit behavior.
• Treating overfitting as a deep-learning-only problem. It happens to gradient-boosted trees, embedding models, and retrievers as well.
• Skipping per-cohort breakdowns. A model can look healthy on average and broken on the cohort that pays the bills.
• Ignoring memorization in LLM fine-tunes. Verbatim recall is the LLM-specific overfit signature; classical metrics miss it.