What Is a Decision Tree?

What Is a Decision Tree?

A decision tree is a predictive model that routes an example through if-then feature tests until it reaches a leaf prediction. In model engineering, teams use decision trees for classification, regression, routing, and guardrail decisions because they score quickly and expose the path behind each result. In LLM and agent systems, a tree may choose a model, queue, retrieval path, or escalation step; FutureAGI evaluates the user-visible effect of that choice through traces and downstream evaluators.

Why decision trees matter in production LLM and agent systems

A decision tree is the right answer to a surprising number of “AI” problems. Routing a customer ticket to one of five queues; deciding whether a user message warrants a guardrail; classifying a user intent into a small fixed taxonomy — all of these can be solved with a tree at near-zero latency, low cost, and full interpretability. Reaching for an LLM here is overkill; reaching for a deep neural network where a tree would do is also overkill.

The pain shows up when teams skip the tree. An LLM-based router runs on every request, costing 200ms and a few cents per inference, where a five-deep tree would have done it in microseconds. A guardrail uses a 7B-parameter classifier where a small tree on prompt features would have flagged 80% of cases just as well.

For LLM and agent systems, trees often play a supporting role: a small tree decides which model variant to route to; a tree-based fast-path classifier filters low-risk requests before any expensive model runs. The composite system is faster, cheaper, and easier to explain. FutureAGI’s job in that composition is to evaluate the final user-visible output, regardless of how many trees and LLM calls it took to produce it.

How FutureAGI handles decision-tree-driven systems

FutureAGI is not a tree-training tool, but tree-based decisions inside an AI system are visible through traces and evaluator results. FutureAGI’s approach is to treat the tree as one decision point inside the full production path, not as the only object being evaluated. When a tree is used inside Agent Command Center through the routing-policies resource or a routing policy: cost-optimized route, every routed request emits a span. Evaluators like Groundedness, AnswerRelevancy, and TaskCompletion score the final output. Eval-fail-rate-by-cohort is sliced by route, so the tree-driven routing decision is a first-class dimension in the dashboard.

A concrete example: a support team uses a small CART tree to triage tickets into “FAQ” (answered by a small distilled LLM), “policy” (answered by a fine-tuned model with retrieval), and “human” (escalated). FutureAGI scores every routed response with the appropriate evaluator and surfaces eval-fail-rate per route. After a deploy, the policy route’s Faithfulness drops 3 points; the team checks whether the tree’s routing changed (it did — drift in the request distribution shifted more requests to a route that wasn’t designed for them) and retrains the tree on a fresh sample. The tree is theirs to maintain; FutureAGI is what made the regression visible.

Unlike Ragas-style evaluation that is RAG-only, FutureAGI evaluates whatever the deployed system produces, whether the upstream is a tree, an LLM, or both.

How to measure tree-system quality

Measure the tree and the generated output together. A tree can look accurate on its own test set while still pushing the wrong requests into a weak prompt, stale retriever, or expensive fallback path. In production, the useful view joins route metadata, trace spans, model calls, and eval results:

• Route-level eval-fail-rate — slice by the tree’s leaf so a regression in one branch is visible before the aggregate score hides it.
• Groundedness, AnswerRelevancy, and TaskCompletion — evaluator scores on the downstream output, not only on the tree’s label.
• Routing distribution drift — count requests landing in each leaf over time and alert when a branch doubles without a planned release.
• Tree-output confidence — when the tree emits a probability, log it as a span attribute and compare low-confidence leaves against escalation rate.
• llm.model.name OTel attribute — confirm each downstream call used the model that the tree-selected route expected.

from fi.evals import TaskCompletion

eval = TaskCompletion()
result = eval.evaluate(
    input="Refund order 12345",
    output="Refund processed.",
)
print(result.score)

Common mistakes

• Training a deep tree without cross-validation; a depth-25 tree on a small routing dataset usually memorizes tickets instead of learning reusable decision boundaries.
• Reading feature importance from a single tree as causal explanation; trees rank correlations found in the training data, not causes of user outcomes.
• Monitoring tree accuracy alone while ignoring downstream Groundedness or TaskCompletion; the route can be correct while the selected model still fails.
• Pruning manually to “look explainable” without tracking lost recall, precision, or eval-fail-rate in the branches product teams care about.
• Encoding high-cardinality categories as arbitrary integers; the tree may split on meaningless numeric order and route similar requests to different leaves.