What Is Clustering?

What Is Clustering?

Clustering is the unsupervised-learning task of grouping a set of items so that items in the same group are more similar to each other than to items in other groups. There are no labels: the algorithm discovers structure in the data on its own. Common algorithms include k-means, hierarchical (agglomerative or divisive), DBSCAN, HDBSCAN, Gaussian mixture models, and affinity propagation. The choice of algorithm and similarity metric (Euclidean, cosine, Jaccard) shapes what the resulting groups look like. In LLM systems, clustering is preprocessing — it feeds retrieval, prompt design, and trace triage rather than producing a user-facing prediction.

Why It Matters in Production LLM and Agent Systems

Most LLM stacks need clustering somewhere. Engineers cluster production traces to surface recurring intents, deduplicate near-identical chunks before indexing, partition golden datasets into homogeneous slices for cohort eval, group user queries to design few-shot examples, and detect anomalies in inference telemetry. The choice of clustering shapes downstream quality directly.

The pain shows up across roles. An ML engineer runs k-means with k=8 on production trace embeddings, declares “we have 8 intents”, and ships a router based on it — three months later half the traffic falls into a single bloated cluster because k=8 was too few and the underlying embedding shifted. A platform engineer caches by exact prompt match because clustering the prompts with semantic-cache semantics was not implemented, and watches cache hit rate flatline at 4%. A product lead reviews “intent clusters” and finds they are syntactically similar but semantically incoherent.

In 2026 agent stacks where clustering feeds routing, prompt design, and dataset curation, picking the right algorithm is more than a hyperparameter — it is a contract with downstream evaluation. A bad cluster boundary becomes a bad routing rule becomes a bad eval cohort.

How FutureAGI Handles Clustering Outputs

FutureAGI does not implement clustering. We sit downstream: when your pipeline groups embeddings, queries, or traces and feeds the result into an LLM application, FutureAGI evaluates whether the grouping helped or hurt. The EmbeddingSimilarity evaluator scores pairwise semantic similarity — feed it cluster members to verify cohesion. The Groundedness and AnswerRelevancy evaluators score downstream LLM responses; segment by cluster_id to find groups that produce bad answers.

Concretely: a customer-support team clusters their last 5K queries with HDBSCAN, identifies 23 cluster centroids, and uses each centroid as a few-shot example in their system-prompt. They test two prompt versions through Prompt.commit() versions and run regression evals against a Dataset versioned at v6. FutureAGI’s AnswerRelevancy and TaskCompletion evaluators score both prompt variants segmented by cluster-id; the centroid-based prompt scores higher overall, but the dashboard reveals one cluster (cluster 17) actively regresses because the centroid is a poor representative. The team drops cluster 17’s centroid, picks a different exemplar via embedding-similarity ranking, and re-runs the eval.

For embedding-based RAG retrieval, EmbeddingSimilarity is the upstream sanity check; Groundedness is the downstream grade. We’ve found that in our 2026 evals, the strongest indicator of bad clustering is a high downstream groundedness variance within a single cluster — clusters whose members produce wildly different groundedness scores are not coherent.

How to Measure or Detect It

Clustering quality is graded by both intrinsic metrics and downstream eval:

• EmbeddingSimilarity (fi.evals): pairwise 0–1 cosine similarity; useful for measuring cluster-member cohesion.
• Silhouette score (intrinsic): combines cohesion and separation; higher is better; no labels needed.
• Davies-Bouldin index (intrinsic): lower is better; tracks intra-cluster dispersion vs. inter-cluster distance.
• Downstream eval lift: AnswerRelevancy or TaskCompletion delta when prompts/routes are built on different clustering outputs.
• Cluster stability: do the same clusters recur across re-runs? Variance suggests embedding-model drift or unstable hyperparameters.

from fi.evals import EmbeddingSimilarity

sim = EmbeddingSimilarity()
result = sim.evaluate(
    text_a="I can't log into my account",
    text_b="login is not working for me",
)
print(result.score, result.reason)

Common Mistakes

• Picking k by eye on a 2D PCA plot. The plot is a 2D shadow of high-dimensional structure; tune k by silhouette plus downstream eval.
• Mixing similarity metrics across the pipeline. Cluster on cosine but retrieve with dot product, and the cluster geometry doesn’t generalise.
• Treating clustering as a one-shot offline step. Production embeddings drift; re-cluster on a schedule and version the cluster output.
• Trusting cluster labels without inspecting members. A “cluster of 200 points” with an unrepresentative exemplar produces poor few-shot prompts.
• No downstream eval gate. Intrinsic silhouette can look great while the LLM application that consumes the clusters regresses; gate on downstream metrics.