How is bagging different from boosting?

Bagging trains base models in parallel on independent bootstrap samples and averages results to reduce variance. Boosting trains models sequentially, with each one focusing on the prior model's mistakes, primarily reducing bias.

How do you evaluate a bagged model in production?

Run a regression eval against a versioned `Dataset` on each retrain, evaluate per-cohort accuracy and fairness, and — when the bagged model feeds an LLM — score the downstream LLM behaviour with `Groundedness` and `ContextRelevance`.

What Is Bagging in Machine Learning? FutureAGI Guide (2026)

Q: What is bagging in machine learning?

Bagging — bootstrap aggregating — is an ensemble technique that trains multiple base models on different bootstrap samples of the data and aggregates their predictions to produce a lower-variance, often more accurate model.

What Is Bagging in Machine Learning?

Bagging — short for bootstrap aggregating — is an ensemble technique that trains multiple base models on different bootstrap samples of a dataset and averages or votes their predictions to produce a single, lower-variance model. It is the idea behind random forests and most general-purpose ensemble classifiers. Bagging is a training-time technique, not used at LLM inference. FutureAGI sits above whichever trainer produced the ensemble: we score the resulting model with regression evals on a versioned Dataset and evaluators on the downstream LLM or agent it feeds.

Why Bagging Matters in Production LLM and Agent Systems

Bagging shows up in modern LLM stacks indirectly: as the classifier that routes prompts to the right model, the reranker that scores retrieval candidates, the fraud filter ahead of an agent action, or the AutoML output that feeds into a workflow decision. Bagging reduces variance, which is the right move when the base model overfits and the production cost of a wrong call is high. The risk is that bagging hides the kind of error the ensemble is making — the per-cohort failure pattern is averaged out, and a fairness regression on a minority slice can be invisible while aggregate accuracy looks fine.

The pain feels different by role. Data scientists see the bagged model win on cross-validation but lose on a specific user segment in production. ML platform engineers see latency triple because the ensemble runs N base models per prediction. Compliance leads find that explaining a single decision is harder when 100 trees voted on it. Product teams see a downstream LLM hallucinate more after a bagged reranker is swapped in — even though the reranker’s aggregate AP looks better.

In 2026, ensemble selection has become an MLOps decision rather than a model-research decision. The bagged classifier sits inside an LLM agent’s tool stack, which means its failures propagate as agent failures. Trajectory-level observability is required to attribute an agent regression to an upstream ensemble change rather than to the LLM itself.

How FutureAGI Handles Bagged Models

FutureAGI’s approach is to score bagged models on the surfaces they affect — the labelled dataset and the production trace. There is no bagging implementation inside fi.evals, and we do not pretend ensembling is a managed FutureAGI primitive.

The integration loop is straightforward. A team trains a bagged model in scikit-learn (BaggingClassifier, RandomForestClassifier), XGBoost, LightGBM, or AutoGluon, and registers the predictions against a versioned FutureAGI Dataset. They call Dataset.add_evaluation to attach FactualAccuracy for QA classification, BiasDetection plus NoGenderBias/NoRacialBias for fairness across cohorts, and RecallScore or PrecisionAtK for ranked retrieval. The regression eval against the previous bagged model on the same dataset slices is the release gate.

Once deployed, the bagged model’s outputs ride the agent trace. If the model is a reranker, retrieval spans flow through traceAI-llamaindex and FutureAGI scores live ContextRelevance. If it is a routing classifier, the predicted route rides each trace as a span_event, and the dashboard splits eval-fail-rate-by-cohort per route. Agent Command Center traffic-mirroring runs a candidate ensemble on shadow traffic, and a model fallback policy keeps the prior version warm during rollout. When Groundedness regresses on the LLM step, the trace makes it obvious whether the upstream bagged model was responsible.

How to Measure or Detect It

Pair model-native and downstream signals:

Aggregate accuracy and per-cohort accuracy: never report only the global number; minority-cohort regressions hide there.
BiasDetection, NoGenderBias, NoRacialBias: fairness evaluators applied per protected group.
FactualAccuracy and Groundedness: when the bagged model feeds an LLM, score the LLM step too.
Variance reduction check: standard deviation of base-model predictions on holdout — bagging should reduce it.
Latency p99 and inference cost: ensembles are inherently expensive; track the operational impact of bagging.
Calibration error: especially for risk-scoring use cases where the predicted probability matters as much as the label.

Quick downstream effect check on a bagged reranker:

from fi.evals import ContextRelevance

metric = ContextRelevance()
result = metric.evaluate(
    input="What is our refund window for EU customers?",
    context="EU returns: 14 days from delivery per regulation 2011/83/EU.",
)
print(result.score, result.reason)

Common Mistakes

Reporting only aggregate accuracy. Bagging hides per-cohort failures by design — look at slices.
Ignoring inference cost. Running 100 base models per prediction at production traffic is expensive and slow.
Skipping a regression eval on retrain. Bagging’s bootstrap stochasticity means each retrain produces a different ensemble; gate every release.
Confusing bagging and boosting use cases. Bagging is for variance; boosting is for bias. Picking the wrong one is a measurable accuracy regression.
Treating ensemble outputs as inherently calibrated. Voting and averaging do not produce calibrated probabilities by default; calibrate explicitly when probabilities matter.