Span vs Trace in LLM Observability: What's the Difference in 2026?

Trace vs span comes down to scope. A trace is one user request from start to finish, identified by a single trace_id; a span is one operation inside it, such as an LLM call, a tool call or a retriever query. Each span points to its parent through parent_span_id, so the spans of one trace form a tree.

You debug a failing agent from that tree. Read 30 flat log lines and you have to rebuild which call ran inside which loop, which retry followed which tool failure, and where the loop stopped. Read the span tree and the nesting already shows it. This is the entry-point explainer; the deeper tutorials are linked below.

If you want depth, read these next:

• What is LLM Tracing? for spans, OTel GenAI, and sampling
• What is LLM Observability? for the full observability layer
• Best OTel Instrumentation Tools for LLMs for a comparison of the instrumentation SDKs

TL;DR

One trace contains many spans. The parent_span_id field on each span builds the tree: the root span has no parent; children point at their parent’s span_id. The OpenTelemetry data model is the same for HTTP services and for LLM applications. What changes are the span attributes; an LLM span carries gen_ai.* attributes, an HTTP span carries http.* attributes.

Spans vs traces at a glance

Why the vocabulary matters

Three reasons. First, debugging an agent run with 30 LLM calls, retriever queries, tool calls, and guardrail checks is intractable as a flat log file. You need the tree to see which tool call was inside which retry loop. The tooling depends on that vocabulary: tools that render the tree, query by parent, replay a single node.

Second, cross-service propagation requires the abstraction. When a Python frontend calls a Go retriever calls a Rust embedding service, the trace_id ties the spans across service boundaries. Without the trace_id concept, distributed debugging is back to log correlation by timestamp.

Third, the eval and monitoring layer attaches scores at the span level. An eval score that says “Faithfulness 0.7 on this trace” is harder to act on than “Faithfulness 0.7 on the generation span at depth 3, which received chunks from retriever span at depth 2 with similarity 0.81”. Span-level scores show which step to fix.

The transport caught up in parallel. OpenTelemetry, formed in 2019 from the merger of OpenTracing and OpenCensus, standardized spans and traces for cloud-native services. The OTel GenAI semantic conventions add the gen_ai.* attribute namespace for LLM workloads; since semantic conventions v1.42.0 (June 12, 2026) they live in their own repository and still carry Development status. Future AGI, Langfuse, Arize Phoenix and Datadog all accept OpenTelemetry spans.

What a span carries

Every span carries the same minimum metadata regardless of what it represents. Identity, timing, status, name and kind come from the OpenTelemetry data model, while the attribute bag holds the LLM-specific detail such as model, tokens and tool arguments.

Identity

• trace_id: ties the span to one user request.
• span_id: unique identifier of this span.
• parent_span_id: identifier of the span that owns this one; null for the root.

Timing

• start_time: Unix-nanosecond timestamp in OTel; actual clock precision depends on the SDK and runtime.
• end_time: Unix-nanosecond timestamp in OTel; actual clock precision depends on the SDK and runtime.
• The duration is end minus start.

Status

• OK, ERROR, or unset.
• Error spans can include exception events with an error message and stack trace when the instrumentation records them.

Name

• Human-readable, like openai.chat.completion, retriever.search, agent.tool_call, guardrail.check.

Span kind

• OpenTelemetry gives every span one of five kinds: Client, Server, Internal, Producer or Consumer.
• The OTel GenAI conventions recommend Client for model calls and remote agent calls, and Internal for tool execution and in-process agents.
• LLM tooling adds a second, semantic kind as an attribute: openinference.span.kind in OpenInference and gen_ai.span.kind in traceAI’s Python SDK. Future AGI Observe reads it as span types such as LLM, Tool, Retriever, Agent and Guardrail.

Attribute bag

• Typed key-value pairs.
• For LLM spans: gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.operation.name.
• For tool spans: gen_ai.tool.name, gen_ai.tool.call.arguments, gen_ai.tool.call.result.
• For retriever spans: gen_ai.retrieval.query.text, gen_ai.retrieval.documents, gen_ai.retrieval.top_k.
• Custom application attributes go here too: app.prompt.version, app.feature.flag, app.user_id_hash.

Events

• Discrete points in time inside the span.
• Useful for streaming first-token markers, intermediate state, and sub-events too small to warrant their own span.

Links

• References to other spans not in the parent chain.
• Useful for async patterns where one span starts work that another span finishes.

What a trace contains

A trace is the collection of every span that shares a trace_id, plus the implicit tree structure built from parent_span_id. The root span is the entry point: usually the user-visible operation like chat.respond or support_agent.run. Below it sit nested spans for each LLM call, retriever query, tool call, guardrail check, and sub-agent dispatch. Tree depth depends on the application:

• A simple chat completion is a shallow trace: a root request span plus one LLM call span beneath it.
• A RAG agent is a 2-3 level tree: root, retriever, generation, optional grounding check.
• A multi-step agent with tool calls and sub-agents is a deeper tree: root, agent, its LLM and tool calls, sub-agent dispatch, the sub-agent’s LLM and tool calls, evaluator.

Most observability backends render this tree natively as nested horizontal bars on a time ruler. The root sits at the top and each child is indented under its parent, so the depth of the tree mirrors the call structure of the application.

Trace ID vs span ID

A trace ID names the whole request and a span ID names one operation inside it. In the OpenTelemetry trace API, a trace ID is a 16-byte value written as 32 lowercase hex characters, and a span ID is an 8-byte value written as 16 hex characters. An all-zero value is invalid for both.

Every span in a trace shares the trace ID and gets its own span ID. The parent_span_id field holds the parent’s span ID, which is how the tree forms. Query by trace ID to pull the whole tree; query by span ID to pull one step.

Across services, the W3C Trace Context traceparent header carries the version, the trace ID, the caller’s span ID (called parent-id in the spec) and trace flags. The receiving service starts its spans with the same trace ID and points their parent at the caller’s span ID, so one trace crosses the service boundary.

traceparent: 00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01

Span types in an LLM trace

Each operation gets a span type with its own attributes: an LLM span records the model and token counts, a tool span the tool name and arguments, and a retriever span the query and chunks. The minimum useful set:

LLM call spans

One per call to a chat completion or text generation endpoint. Carry every gen_ai.* attribute plus the prompt and completion (opt-in). Nest under the application logic span that triggered the LLM call. Under the OTel GenAI conventions its name is {gen_ai.operation.name} {gen_ai.request.model}, for example chat gpt-4.1-mini, and its recommended kind is Client.

Tool call spans

One per function or tool invocation by the model. Carry the tool name, arguments, return value, latency, and status. Nest under the agent or chain span that ran the tool, as a sibling of the LLM span that requested it. The conventional name for this span is execute_tool {gen_ai.tool.name}, and its kind is Internal.

Retriever spans

One per vector search, BM25 search, or hybrid retrieval. Carry the query, the top-k chunks returned, similarity scores, and the index version. Its operation name is retrieval, with the query in gen_ai.retrieval.query.text and the results in gen_ai.retrieval.documents. For depth, see What is RAG Observability?.

Sub-agent spans

One per dispatch to a child agent. The child agent’s full trace tree nests under the parent. The shape is a sub-tree, not a flat span. In OTel terms an agent span uses the operation name invoke_agent, marked Client for a remote agent service and Internal for an in-process agent such as a LangChain or CrewAI agent.

Guardrail spans

One per input or output validator. Carry the rule name, the verdict, and the modified payload if the guardrail rewrote the input or output. Future AGI Observe shows these as Guardrail spans, so the verdict sits in the same tree as the LLM call it checked.

Evaluator spans

One per online scorer (LLM-as-judge, schema check, citation grounder). Either nested in the same trace as the span it scores or connected via a span link when the eval runs out-of-band. Future AGI Observe shows these as Evaluator spans that carry the metric and the score.

Custom spans

Anything your business logic does between LLM calls (preprocessing, postprocessing, business rule checks, persistence) gets its own span if you care about its latency or status. OpenTelemetry treats a span with no kind set as Internal, which fits most in-process business logic.

Common mistakes when modeling LLM spans and traces

• Treating spans as logs. Logs have timestamps and severity. Spans have parent_span_id, attribute bags, and structured status. Bolting span structure on top of log lines later means re-instrumenting every call site.
• Span-per-line-of-code. Too many spans crush the backend and make the tree unreadable. Pick a granularity (every LLM call, every tool call, every retriever call) and stick to it.
• Span-per-chunk in retrievers. Each retriever call is one span; chunks are an attribute on that span, not children. Span-per-chunk multiplies the span count by the number of chunks returned.
• Flat trace for agent runs. A LangGraph or CrewAI run is a tree. A flat span list buries the loop and the tool decisions. Force tree-structured trace views with parent_span_id discipline.
• No prompt version tag. Add app.prompt.version as a custom attribute on every LLM span. Without it, A/B prompt rollouts are not analyzable.
• No trace_id propagation across services. When a Python service calls a Go service, the trace_id has to pass in headers. Without propagation, the cross-service tree is broken.
• Forgetting redaction. gen_ai.input.messages and gen_ai.output.messages carry PII. Configure the SDK or collector to redact before storage. Treat retriever chunk_text the same way.
• Ignoring the root. Without a clean named root span (chat.respond, support_agent.run), traces are anonymous. Force a top-level span at every public entry point.

How OTel terminology compares to vendor terms

Different vendors use slightly different vocabulary on top of the OpenTelemetry data model. The table below maps each OTel term to its name in Langfuse, Phoenix and LangSmith, and the Datadog subsection after it covers Datadog’s terms.

Every vendor term maps back to an OTel term. The mapping matters when you switch backends: Langfuse, Phoenix and Future AGI all accept OpenTelemetry spans, so the labels change while the data model stays the same. For a side-by-side of the backends, see Best LLM Tracing Tools in 2026.

Datadog span vs trace terms

In Datadog Agent Observability, a trace represents the work involved in processing a request, and a span is a unit of work that builds the trace. The root span is the first span in a trace and marks its beginning and end. Datadog labels each span as LLM, Workflow, Agent, Tool, Task, Embedding or Retrieval, and it accepts OTel spans that follow the GenAI semantic conventions v1.37 or later.

From flat logs to a span tree you can debug

Future AGI gives you that tree without hand-written spans on supported frameworks. traceAI is Apache-2.0 instrumentation built on OpenTelemetry for Python, TypeScript, Java and C#. It turns model calls, tool calls, retrievals and agent steps into standard OTel spans with gen_ai.* attributes, and Future AGI Observe labels each span by type and nests it under the step that triggered it.

Eval scores attach to a single span or a whole trace, so a faithfulness score lands on the generation step that earned it (run evals on spans, traces or sessions). The same platform ingests OTLP, stores spans in ClickHouse and runs the Agent Command Center gateway. Start free and pay as you go. Future AGI is SOC 2 Type II, and a HIPAA BAA and SAML SSO come with paid add-ons (pricing).

Instrument one app with traceAI and read its first span tree in Future AGI: start with the Observe quickstart.

Sources

• OTLP trace data model (trace.proto)
• CNCF: A brief history of OpenTelemetry
• OpenInference GitHub repo
• Langfuse data model docs
• Langfuse observation types
• LangSmith SDK run types (schemas.py)
• Datadog Agent Observability docs