What Is CI/CD for ML?

What Is CI/CD for ML?

CI/CD for ML is the discipline of running continuous integration and continuous delivery on machine learning systems. Every commit, prompt change, model swap, or dataset update triggers an automated pipeline that builds the system, runs unit tests, executes evaluator suites on a golden dataset, checks for regressions on production-traced cohorts, and only then promotes to staging or production. It is an MLOps surface, not a single tool: the pipeline ties Git history, dataset versions, evaluator thresholds, and deployment gates into one reliability loop that FutureAGI plugs into.

Why It Matters in Production LLM/Agent Systems

ML systems break in ways software CI/CD cannot catch. A prompt edit that improves one test case can silently regress a hallucination metric on three others. A model upgrade can drop tool-selection accuracy on a multi-step agent. A retrieved-chunk format change can collapse RAG faithfulness while keeping unit tests green. Without CI/CD for ML, these regressions reach production and surface as user complaints, runaway cost, or compliance incidents days later.

The pain spreads across roles. ML engineers see “it worked in the notebook” but production traces show 18% lower groundedness. SREs see latency p99 jump after a model change with no clear owner. Product managers cannot release prompt experiments because no one trusts the rollback path. Compliance teams cannot audit which model and prompt version produced a given response.

Agentic systems amplify this risk. A 2026-era agent might run six tool calls, two retrievals, and a planner step per request. Any one of those nodes can regress on a code change. Without an eval-driven CI gate, the team is debugging in production. With one, regressions are caught in the pull request before merge.

How FutureAGI handles CI/CD for ML

FutureAGI treats CI/CD for ML as an eval-gated workflow built on the Dataset SDK, the evaluator suite, and traceAI signals. FutureAGI’s approach is to make every pull request run the same evaluators that monitor production, on the same golden dataset, with the same thresholds.

A real workflow looks like this. The team maintains a Dataset of 400 versioned support-agent traces with ground-truth answers. On every pull request, GitHub Actions calls Dataset.add_evaluation with Groundedness, ContextRelevance, TaskCompletion, and JSONValidation. The pipeline also runs the new prompt or model against a 50-trace regression cohort sampled from production via traceAI. If Groundedness drops below 0.85 or TaskCompletion regresses by more than 3%, the merge is blocked and a Slack alert posts the failing examples with diffs.

For prompt changes specifically, the team can wire agent-opt optimizers like ProTeGi or GEPA into the same pipeline: the optimizer proposes candidates, the eval suite ranks them, and only the winner is promoted. Unlike a generic LangSmith experiment that requires a manual review, FutureAGI keeps the dataset, evaluator class, threshold, and approval state in the same audit trail. That trail is what compliance teams ask for during an EU AI Act review.

How to Measure or Detect It

Treat CI/CD for ML as a measurable system, not a process:

• Eval-fail rate per pull request: percentage of PRs blocked by evaluator regressions; rising rates often expose a brittle prompt or thinning dataset coverage.
• Time-to-merge for ML changes: median minutes from PR open to gated merge; long tails usually mean flaky evaluators or missing golden coverage.
• Regression-cohort delta: per-evaluator score change on the production-sampled cohort versus the previous main; the most direct quality signal.
• Dataset version drift: ratio of merged PRs that updated Dataset rows; very low values suggest the golden set has gone stale.
• Deployment frequency for prompts and models: a healthy team ships prompts daily, models weekly, both behind eval gates.

A minimal eval-gate snippet that fits in a CI step:

from fi.evals import Groundedness, TaskCompletion

g = Groundedness().evaluate(response=resp, context=ctx)
t = TaskCompletion().evaluate(input=task, response=resp)
assert g.score >= 0.85 and t.score >= 0.80, "block merge"

Common Mistakes

• Using only unit tests for prompt or model changes: prompts pass syntactic checks while regressing factual accuracy on real traces.
• Treating one offline benchmark as the gate: MMLU or HumanEval scores can rise while production task completion falls — gate on production-shaped data.
• Skipping the regression cohort: a golden dataset alone does not catch drift; pair it with a rolling sample of production traces from the last 7 days.
• Caching evaluator outputs across model swaps: a cached Groundedness score from gpt-4o is meaningless after a switch to claude-sonnet-4.
• Manual approval as the only gate: humans rubber-stamp green builds; numeric thresholds on evaluator scores are the actual safety net.