What Is a Multi-Turn Conversation?

What Is a Multi-Turn Conversation?

A multi-turn conversation is an exchange between a user and an LLM that spans more than one prompt-response pair, where each new turn depends on context accumulated from prior turns. The model must track entities, references, and intent — usually by re-sending the running message history or by summarizing it into a memory store. Multi-turn is the default surface for chatbots, support agents, and voice agents. Each new turn is both a fresh inference and an extension of state, which is why single-turn evaluation misses most of the things that go wrong in production.

Why It Matters in Production LLM and Agent Systems

The failure modes that show up only in multi-turn conversations are precisely the ones users notice. The model contradicts itself between turns three and seven. It loses a constraint the user gave at turn two. It “agrees” with whatever the user just said, even if that contradicts what it asserted earlier — sycophancy that compounds across the dialogue. By turn ten, the conversation is technically still on-topic, but the model has drifted into giving advice that conflicts with the system prompt.

The pain is shared across roles. Support engineers see escalation rate climb when conversations exceed five turns. Product managers see CSAT drop on long sessions even when first-turn answers are excellent. Compliance leads find that PII redacted at turn one has been quoted back to the user at turn six because the running history was not re-scrubbed.

In 2026-era voice and chat agents, this gets worse. A voice agent’s “conversation” is a long-running stream of partial transcripts, tool calls, and TTS responses where every turn carries microphone noise, ASR errors, and barge-ins. Multi-step agent loops layered on top mean a single user request can produce internal “turns” the user never sees. Evaluating only the final answer ignores everything in between, and that is where the bug lives.

How FutureAGI Handles Multi-Turn Conversations

FutureAGI treats a conversation as a first-class trace, not a sequence of unrelated requests. Each user-assistant exchange becomes a span with agent.trajectory.step, and the entire session is grouped under a parent trace_id so coherence can be evaluated end-to-end. The ConversationCoherence evaluator scores whether the model stays on topic across turns. CustomerAgentContextRetention checks whether constraints, entities, and decisions established earlier in the session are honored later. CustomerAgentLoopDetection flags when the agent repeats the same response or revisits the same tool call, which is the classic multi-turn failure shape.

Concretely: a fintech support chatbot built on traceAI-langchain ingests every session into FutureAGI. Every five turns, the orchestrator runs ConversationCoherence and CustomerAgentContextRetention on the running transcript and writes scores back as span_events. A dashboard shows coherence-score-by-turn-number — the team sees coherence holding at 0.92 through turn six, then dropping to 0.71 by turn ten as the context window fills. They respond by introducing summarization-based memory at turn five so the prompt stays bounded. That is the kind of fix you cannot make if you only evaluate the final answer.

How to Measure or Detect It

Treat multi-turn quality as a rolling signal across the session, not a one-shot score:

• ConversationCoherence: returns 0–1 plus a reason for whether the conversation is logically consistent across turns.
• CustomerAgentContextRetention: scores whether prior-turn constraints are honored in later turns.
• CustomerAgentLoopDetection: flags repeated or redundant turns that signal the agent is stuck.
• Coherence-score-by-turn-number (dashboard signal): chart eval scores against turn position to see where degradation kicks in.
• Token usage per turn: if it climbs linearly, the context window is filling up and you are heading toward overflow.

Minimal Python:

from fi.evals import ConversationCoherence

coherence = ConversationCoherence()
result = coherence.evaluate(
    conversation=session_messages
)
print(result.score, result.reason)

Common Mistakes

• Evaluating only the last assistant message. Multi-turn failures are diffuse — score the trajectory, not just the final response.
• Letting the message history grow unbounded. Past a few thousand tokens, the model starts dropping early-turn constraints; truncate, summarize, or move to memory.
• Ignoring sycophancy across turns. Models often flip positions when contradicted; pin a factual-consistency check across turns to catch it.
• Re-injecting raw PII into every turn. Anything redacted at turn one must stay redacted in the running history; do not reconstruct it.
• Scoring CSAT at session-end only. Fail signals appear inside the session — track coherence per turn so you can intervene.