Engineering

Subagents: Why Failures Never Roll Up to the Parent Span

A subagent can fail, record the error correctly, and still leave the parent span green. Why OpenTelemetry defines no status roll-up, and how to fix it.

· 14 min read
subagents distributed-tracing opentelemetry agent-observability multi-agent-systems span-status
Editorial cover on a black blueprint grid reading SUBAGENTS FAIL. THE PARENT TRACE SAYS OK. On the right a thin line panel titled ONE TRACE FOUR SPANS shows a supervisor span marked OK above three nested subagent spans for research, pricing and summarise, where pricing is marked ERROR, with a dashed line labelled STATUS STOPS HERE and a footer reading ROOT SPAN STATUS OK and 10 subagents at 5 percent gives 40.1 percent of runs.
Table of Contents

A supervisor dispatches five subagents. Four return, one fails. The supervisor summarises what it has and reports the task complete.

Open the trace. The root span carries no error. Three levels down, the pricing subagent’s span reads ERROR, with the exception recorded and the stack trace attached. Both spans are correct.

That gap is not an instrumentation bug and it is not something your vendor forgot. It is what the OpenTelemetry specification leaves undefined, and every multi-agent system inherits it the moment a supervisor starts dispatching work.

The short version: span status has no roll-up. A child marked ERROR does not change its parent, so error rates keyed on the root span read clean straight through the incident.

What Does a Parent Span Actually Know About Its Children?

Very little, and the specification is precise about how little.

A trace is a tree assembled from parent pointers. Future AGI’s own traces documentation puts it plainly: “A trace is a tree of spans. The root span is the operation that kicked off the request, and every other span nests under the step that triggered it.”

Status is a separate concern from that tree. The OpenTelemetry tracing API defines three status codes, Unset, Ok, and Error, and says setting one “will override the default Span status, which is Unset.”

Hold onto that default. A span nobody touches is not Ok. It is Unset, which most backends render as the absence of a problem rather than as a positive verdict.

The decisive detail is what a child actually takes from its parent. The spec lists it exactly once:

“For a Span with a parent, the TraceId MUST be the same as the parent. Also, the child span MUST inherit all TraceState values of its parent by default.”

That is a closed list of two things, trace ID and trace state, and it runs downward. Status is not on it, and there is no upward mechanism anywhere in the specification. Status is per-span, set only by an explicit call, and nothing calls it on the parent for you.

Worth being careful here. The spec does not forbid roll-up, it simply defines none, and as of 2026-08-01 there is no open proposal in the specification to add one. If you want a parent to reflect its children, that is code you write.

This post is the distributed case. The single-process version, where a harness catches an exception and the step’s own span never turns red, is covered in the failures your agent harness catches and never tells you about. That one establishes why a span status is only ever set by an explicit call. This one is about why a correctly recorded error still does not travel upward.

Three adjacent subjects are covered elsewhere and not repeated here.

Scoring whether a dispatch was correct in the first place is an evaluation problem, handled in our guide to evaluating Claude sub-agents. The vocabulary itself, spans against traces, is set out in span vs trace. Building the hierarchy is covered in tracing multi-agent systems.

How Often Does a Clean Parent Trace Hide a Failed Child?

Often enough that it changes what your dashboard means.

If a supervisor dispatches k subagents and each fails independently with probability p, the chance at least one failed is 1 - (1-p)^k. That is the fraction of runs containing a failure. None of them turn the root span red on their own.

Subagentsp = 2%p = 5%p = 10%
35.9%14.3%27.1%
59.6%22.6%41.0%
1018.3%40.1%65.1%
2033.2%64.2%87.8%

A supervisor fanning out to ten subagents, each failing five percent of the time, contains at least one failed child in 40.1% of runs.

Now put a dashboard on it. If the orchestrator never re-raises, an error rate computed from root spans reports 0.0% against a true 40.1%. Re-raise a quarter of the time and it reports 10.0%. Re-raise half and it reports 20.1%.

The number on the dashboard is not measuring your system. It is measuring your exception handling.

There is a sharper edge if a supervisor marks itself successful. The spec sets a total order where “Ok > Error > Unset”, so setting Ok “will override any prior or future attempts” to set Error.

It is blunt about who should do that: “Generally, Instrumentation Libraries SHOULD NOT set the status code to Ok, unless explicitly configured to do so. Instrumentation Libraries SHOULD leave the status code as Unset unless there is an error.”

Read the next line before taking that as a prohibition, though. “Application developers and Operators may set the status code to Ok.” Your supervisor is application code, so the spec permits exactly what this post is about to argue against. The argument is ours, not the specification’s.

It is still the right call, because an eager Ok does not just fail to report a problem. It outranks the report.

Why Do Subagent Spans Go Missing Entirely?

A wrong status is the mild version. The worse one is a subagent whose spans are not in the trace at all.

Parenting is resolved from the current context, and the spec is explicit about what happens without one: “If there is no Span in the Context, the newly created Span will be a root span.” Root spans are not free-floating children. Implementations “MUST generate a new TraceId for each root span created.”

So a subagent that starts without context does not produce an orphan inside your trace. It produces a different trace, under a trace ID nobody is looking at, containing the failure you are hunting.

Context is scoped narrowly. The context specification carries it “across API boundaries and between logically associated execution units”, and the glossary defines an execution unit as “threads, coroutines or fibers”.

The Python SDK implements that on top of contextvars, which is exactly why a ThreadPoolExecutor is a common place to lose it.

Future AGI’s manual tracing walkthrough names the symptom directly: “Without the parent span, the retrieval and LLM spans would appear as separate traces since each top-level span gets its own trace ID.”

Our advanced tracing docs name the fix: “When tasks run in a ThreadPoolExecutor or via Promise.all, capture the context in the main thread and attach it in each worker so all tasks remain linked to the parent span.”

To be fair to the tooling, this is usually handled for you. The OpenTelemetry docs note that propagation “is usually handled by instrumentation libraries and is transparent to the user”. Subagent dispatch is where that stops being reliable, because fan-out is the moment your own code chooses how work crosses a boundary.

Which Failures Reach the Parent and Which Do Not?

Six outcomes, and only one of them turns the root red without help.

What happened in the childRoot span statusWhere the evidence lives
Child raised, exception escaped into the parentErrorThe root span, correctly
Child raised, parent caught and continuedUnsetThe child span only
Child called SetStatus(Error), parent read the resultUnsetThe child span only
Child returned a wrong value, no exceptionUnsetNowhere, until an eval reads the output
Context was not propagatedUnset, and incompleteA different trace entirely
Child never startedUnsetNowhere

Every row after the first reads as a clean run. Unset is the default a span carries when nothing set it, and dashboards count errors, not the absence of them.

The first row deserves a note, because it is the one that surprises people. The parent does not turn red through any roll-up rule. It turns red because language SDKs set Error when an exception escapes the span’s scope, which in Python is what start_as_current_span does on the way out.

That is not specification behaviour either. The spec never says an escaping exception sets Error. It is the SDK’s convenience.

The Python source is explicit that it reaches the parent, noting that this “causes parent spans to set their status to ERROR and to record an exception as an event if a child span raises an exception even if such child span was started with both record_exception and set_status_on_exception attributes set to False”.

So row one carries conditions worth stating. It holds when the exception escapes the parent’s own span scope, the parent was opened with start_as_current_span, set_status_on_exception is left at its default, the span is still recording, and its status is not already Ok. A parent opened with tracer.start_span() and closed with span.end() stays Unset no matter what escapes.

Read the second and third rows together. They are the common case in agent systems, because a supervisor that halts on any child failure is a supervisor nobody ships. Partial results are the design.

The last two rows are the ones that cost hours, because the trace looks complete. Nothing marks a span that was never created.

How Do You Make a Child Failure Visible at the Root?

Three moves, in order of how much they buy you.

Carry context across every boundary you own. Fan-out is the risk point. Capture context before dispatch and attach it inside each worker, so children land in the trace rather than starting their own.

Decide a roll-up policy and write it. The protocol will not choose for you. A workable default is to set an error status on the parent when a child failure changed the outcome, and leave it alone when the parent genuinely recovered. What matters is that the rule exists and is applied everywhere.

from concurrent.futures import ThreadPoolExecutor
from opentelemetry import context as otel_context, trace
from opentelemetry.trace import Status, StatusCode

tracer = trace.get_tracer(__name__)

def run_subagent(task, ctx):
    token = otel_context.attach(ctx)          # child lands in the parent's trace
    try:
        with tracer.start_as_current_span(f"subagent.{task.name}"):
            return task.run()                 # the SDK records the exception and sets ERROR on escape
    finally:
        otel_context.detach(token)            # workers are reused; a leaked token reparents the next task

def supervise(tasks):
    with tracer.start_as_current_span("supervisor") as parent:
        ctx = otel_context.get_current()      # capture inside the parent span
        with ThreadPoolExecutor() as pool:
            futures = [pool.submit(run_subagent, t, ctx) for t in tasks]

        results, failures = [], []
        for f in futures:
            try:
                results.append(f.result())
            except Exception as exc:
                failures.append(exc)

        if failures:                          # the roll-up nobody writes for you
            parent.set_status(
                Status(StatusCode.ERROR, f"{len(failures)} subagents failed")
            )
        return results, failures

Three details carry the whole snippet.

The context is captured inside the parent span, because that is what makes each worker’s span a child rather than a new trace.

The child does no manual error recording. start_as_current_span already calls record_exception and sets Error when an exception leaves the block, under the defaults record_exception=True and set_status_on_exception=True.

Writing those two lines by hand as well ships two exception events for one failure, and the SDK’s description overwrites the message you passed. Read the Python implementation before you add them.

The parent’s status is set on the parent handle while its with block is still open. Once that block exits the span has ended, and a status written to an ended span is dropped.

The roll-up does not have to be a write at all. There are two places to put it, and the choice is mostly about who can deploy.

Write time is the snippet above. It is the precise option, because only your orchestrator knows whether a child failure changed the outcome, and it is the expensive one, because it means touching every supervisor you run.

Query time skips the write. Grafana Tempo’s TraceQL has a descendant operator, {condA} >> {condB}, documented as spans matching condB “that are descendants of a span matching condA”.

So { name = "supervisor" } >> { status = error } returns the runs you care about without anything having been rolled up. You stop asking the root instead of correcting it.

The tradeoff is that this lives in one backend’s query language rather than in your traces, so alerts and dashboards have to be written there too.

Count children, do not just read statuses. Dispatching five subagents and finding four spans is a failure that no status field will ever report. Compare the number of children you started against the number that appear, and treat a shortfall as an incident.

One thing to avoid, and this is a recommendation rather than a rule the specification imposes: do not have the supervisor mark itself Ok on the happy path. Given Ok > Error, that single call can outrank a genuine error recorded later in the same span.

Where Does Future AGI Fit?

The checks above all need the tree, not the summary.

Status is filterable at the span level, where status takes OK or ERROR with is and is not, so you can search for failing spans directly rather than starting from traces that look fine.

That distinction matters here, because the trace that contains your failure is a trace whose root never went red.

A Future AGI multi-agent trace zoomed into the failed vendor_lookup span, showing Status ERROR from a 429 rate limit on the vendor directory API, the retry span directly beneath it, the error message in the result panel, and the full agent graph of the research_orchestrator run underneath.

That is a child step at ERROR inside a run that kept going. The span carries the evidence. The root span’s status carries none of it.

For assertions rather than searches, evals attach at three levels: “Spans (one step), Traces (a whole request), or Sessions (a whole conversation)”.

A trace-level eval can be given Trace context, documented as “the full trace tree, every span in the request”. That is what lets a check reason about children instead of the final message.

It is exposed through the connectors menu, which the docs note appears “only on a single Agent Evaluator”, not on a composite eval, an LLM-as-Judge, or a Code Eval.

A Future AGI trace with the Evals tab open, showing a task_completion eval result for the whole request with its pass status and written reasoning, and the agent graph of the full span flow beneath it.

That run passed. What matters in it is where the check is attached, not the verdict it returned: the eval is reading the request rather than the last message, which is the only position from which a failed child is visible at all.

For failures you did not think to assert on, Error Feed “reads the full span tree: inputs, outputs, tool calls, LLM responses, errors, metadata”, and its Agent Flow view “makes it obvious whether the error is in step one or step five”.

Check the sampling rate before you rely on it. The release notes record that for new tracing projects the rate “now defaults to 0%, so the Error Feed is off until you configure it”.

The sampling docs confirm it “doesn’t analyze every trace by default”. The guidance is 100% during development, and “10-20% is a reasonable starting point” in production.

Three honest notes about our own instrumentation, all read from the traceAI Claude Agent SDK integration, version 0.1.0, as of 2026-08-01.

It tracks subagent dispatches with, in its own words, “proper parent-child span relationships”, and _subagent_tracker.py sets an error status on a subagent span when the dispatch comes back flagged as an error. It does not mutate the parent, because that would be the wrong thing for a tracer to do. The roll-up decision stays yours.

The second note is uncomfortable. That same file sets Ok on every subagent span that does not fail, which is the eager Ok this post has been arguing against, and the reason the specification tells instrumentation libraries to leave the status Unset.

A subagent span’s status is therefore a verdict from the integration rather than an absence of one. If you are writing your own roll-up on top of it, read the children’s Error states, and treat Ok as “the dispatch returned” rather than “the work was right”.

The third note is the one that changes what you should do. _client_wrapper.py sets Ok on the assistant-turn span and on the conversation root span when a run completes normally.

Given Ok > Error, and given that the SDK ignores a later status write on a span already marked Ok, the roll-up recommended earlier in this post does not work if you write it onto that conversation span. The integration has already claimed it.

So if you are on this integration, put your roll-up on a supervisor span you own, above or beside the conversation span, and treat the conversation span’s Ok as the integration reporting that the stream finished rather than that the work was right. We would rather say that here than let you discover it from a dashboard.

Which Subagent Failures Should You Surface First?

Start with the missing spans, because they are the only category with no evidence anywhere. Count dispatched children against recorded children and alert on the gap.

Then write the roll-up rule. Until you do, your error rate is a measurement of how often your orchestrator re-raises, which is a fact about your code rather than your system.

Then move the assertion off the final answer and onto the tree. A supervisor that produced a confident summary from four results out of five is not a run that failed, and it is not a run that succeeded either.

Frequently Asked Questions About Subagent Tracing

Does a Child Span’s Error Status Propagate to Its Parent in OpenTelemetry?

No. The specification defines no status roll-up. It states exactly what a child takes from its parent, the trace ID and all trace state values, and status is not among them. Status is set only by an explicit call and defaults to Unset. A failing child does not make that call on its parent, so the parent stays clean unless your orchestrator sets it.

Why Do My Subagent Spans Show Up as Separate Traces?

Because context was lost before the span was created. OpenTelemetry resolves a parent from the current context, and the spec says that with no span in the context the new span becomes a root span, and every root span gets a new trace ID. Crossing a thread, a process, or a queue without carrying context explicitly produces exactly this.

How Do You Find a Run Where a Subagent Failed but the Parent Reported Success?

Not from root span status, which in this scenario is Unset or Ok, never Error. Search at the span level for an error status rather than filtering whole traces, or attach an eval that reads the trace tree instead of the final answer. Counting child spans against how many you dispatched also catches subagents that never started or were recorded elsewhere.

Should the Orchestrator Mark the Parent Span as Failed When a Child Fails?

That is a policy decision, because the protocol will not make it for you. A reasonable default is to mark the parent with an error status when a child failure changed the outcome, and leave it alone when the parent genuinely recovered. Whichever you pick, apply it consistently, or your error rate measures your exception handling.

Why Does Marking a Parent Span OK Make Things Worse?

Because Ok wins. The specification sets a total order where Ok is greater than Error, so setting a status to Ok overrides any prior or future attempt to set it to Error. It also advises instrumentation libraries not to set Ok unless explicitly configured. A supervisor that eagerly marks itself Ok will mask a real failure recorded later.

Frequently Asked Questions

Does a child span's error status propagate to its parent in OpenTelemetry?

No. The OpenTelemetry tracing specification defines no status roll-up. It states exactly what a child takes from its parent, the trace ID and all trace state values, and status is not among them. A span's status is set only by an explicit SetStatus call and defaults to Unset. A failing child does not make that call on its parent, so the parent stays clean unless your orchestrator sets it.

Why do my subagent spans show up as separate traces?

Because context was lost before the subagent span was created. OpenTelemetry determines a parent from the current context, and the specification says that if there is no span in the context, the new span becomes a root span, and every root span gets a new trace ID. Crossing a thread, a process, or a queue without carrying context explicitly produces exactly this.

How do you find a run where a subagent failed but the parent reported success?

Not from root span status, which in this scenario is Unset or Ok, never Error. Search at the span level for an error status rather than filtering traces, or attach an eval that reads the whole trace tree instead of the final answer. Counting child spans against how many you dispatched also catches the case where a subagent never started or was recorded somewhere else entirely.

Should the orchestrator mark the parent span as failed when a child fails?

That is a policy decision you have to make, because the protocol will not make it for you. A reasonable default is to mark the parent with an error status when a child failure changed the outcome, and to leave it alone when the parent genuinely recovered. Whichever you choose, apply it consistently, or your error rate measures your exception handling rather than your system.

Why does marking a parent span OK make things worse?

Because Ok wins. The specification sets a total order where Ok is greater than Error, so setting a span's status to Ok overrides any prior or future attempt to set it to Error. It also tells instrumentation libraries not to set Ok unless explicitly configured. A supervisor that eagerly marks itself Ok will mask a real failure recorded later.
Related Articles
View all