What Is CX Analytics?

What Is CX Analytics?

CX analytics is the discipline of measuring customer experience across every channel using structured event data plus LLM-derived signals such as sentiment, intent, effort, satisfaction, and resolution. It augments CSAT, NPS, and CES with model-scored data from chats, calls, tickets, and product sessions. FutureAGI treats each LLM-scored signal as an evaluated metric in the production trace, so teams can see where customers struggle, where automation helps, and whether a trend is real or classifier drift.

Why It Matters in Production LLM and Agent Systems

CX analytics decisions move money — staffing levels, agent training budgets, automation investment, product roadmap. An uncalibrated sentiment classifier shifting 5 percentage points across a quarter looks like a customer-experience regression and triggers an org-level response that may not be warranted. A miscalibrated intent classifier pointing growth to the wrong product flow burns engineering capacity. Without calibration evidence, every dashboard is a story without a citation.

The pain is felt across roles. A VP of CX presents a board chart of declining sentiment, then learns the LLM behind the score was upgraded mid-quarter and now scores tougher on the same content. A support ops lead uses LLM-scored “effort” to prioritize friction fixes and cannot tell whether one channel really shows higher effort or the channel-specific transcripts confuse the classifier. A product manager A/B-tests a new flow and the test reads “neutral” because the sentiment judge has high variance on short responses.

In 2026 stacks the surface widens. CX analytics increasingly consumes voice and multi-modal interactions, where the chain is ASR → summarize → classify, and any upstream drift propagates into the score. Without per-step evaluation, the analytics team cannot tell whether the customer experience moved or whether the model upgraded.

For agentic customer-service systems, planner, retrieval, tool, and escalation steps can each change perceived effort before the final answer appears.

How FutureAGI Handles CX Analytics

FutureAGI’s approach is to keep every model-derived CX signal calibrated and auditable. The signal classifiers — sentiment, intent, effort, resolution — are implemented as fi.evals evaluators or CustomEvaluation instances, each calibrated against a human-labelled Dataset cohort. Calibration accuracy and Cohen’s kappa are tracked over time; a drift threshold triggers re-calibration. Tone, ConversationResolution, and CustomerAgentConversationQuality cover the most common CX dimensions out-of-the-box.

For multi-channel ingestion, traceAI integrations such as langchain, llamaindex, livekit, and pipecat annotate spans with the channel and customer cohort. The dashboard slices by channel and cohort, surfacing per-segment CX gaps that a global score hides. When the analytics layer reports a sentiment shift, the trace view shows whether the change came from the customer (real signal) or the classifier (drift). For voice, the upstream ASRAccuracy metric is monitored alongside the CX scores so a transcript regression does not silently corrupt the experience score.

For pre-release testing, a Scenario with Persona cases can run through LiveKitEngine; the same ConversationResolution and Tone scores then become regression gates before rollout.

We’ve found that the single biggest improvement teams get from this approach is honest dashboards — a CX trend that is real, not classifier drift dressed up as customer behaviour.

How to Measure or Detect It

Calibrate every LLM-scored CX dimension; never trust an uncalibrated classifier, and compare results with verified survey or escalation outcomes:

• ConversationResolution: percent of conversations that ended with the issue resolved.
• Tone: scored tone classification — calibrate per channel and per cohort.
• CustomerAgentConversationQuality: composite quality score for support-conversation analytics.
• Calibration kappa (dashboard signal): agreement between classifier and human labels — the trust score for each metric.
• Per-cohort sentiment delta: differences across customer segments; a real CX gap if calibration is solid, classifier noise if not.
• Channel-cohort eval-fail-rate: per-channel quality failures; isolates pipeline regressions from real CX shifts.

Minimal Python:

from fi.evals import ConversationResolution, Tone

resolution = ConversationResolution()
tone = Tone()

r = resolution.evaluate(input=conversation, output=summary)
t = tone.evaluate(output=customer_message)
print(r.score, t.score)

Common Mistakes

• Reporting LLM-scored sentiment without calibration evidence. The score’s accuracy versus humans is the headline number — without it, the trend line is opinion.
• Mixing model versions across the time series. A classifier upgrade mid-quarter breaks longitudinal comparisons. Pin the analytics-layer model build.
• Global CX score across channels. Voice and chat have different transcript characteristics; per-channel calibration is required.
• Confusing CSAT-proxy with CSAT. Predicted satisfaction does not replace verified customer surveys; correlate both as a check.
• Dropping per-cohort breakdowns. Underrepresented cohorts vanish from the global mean. Always slice.