What is DSPy? Stanford's Compiled Prompt Framework in 2026

DSPy (Declarative Self-improving Python) is an open-source, MIT-licensed Python framework from Stanford NLP for programming language models instead of hand-writing prompts. You declare a task as a Signature, pick a Module such as ChainOfThought or ReAct, and an optimizer compiles the prompts and few-shot examples against your metric.

A team has a question-answering system over financial filings. The hand-tuned ChainOfThought prompt with three few-shot examples plateaus on their dev set, and no amount of system-prompt editing pushes the metric meaningfully higher. They rewrite the same logic in DSPy: a Signature with context, question -> reasoning, answer, wrapped in a ChainOfThought module, compiled with MIPROv2 against a labeled training set.

The compile pass takes the better part of a working session. The compiled program lands materially above the hand-tuned ceiling on the same dev set, with prompts and few-shot examples the team would not have written by hand.

This is the DSPy thesis: hand-prompting tops out, and optimizer-compiled programs push past that ceiling. The price is training inputs, a metric, and a compile pass. The payoff is a quality lift on workloads where prompting alone cannot reach the bar.

TL;DR

DSPy replaces hand-written prompts with declarative modules and compiled prompts. The repo lives at github.com/stanfordnlp/dspy, and the latest release is 3.4.1 (October 12, 2026). The primitives are Signature (input-output spec), Module (a strategy like Predict, ChainOfThought, or ReAct), Program (composed Modules), Optimizer (the compile algorithm), and Metric (the scoring function).

The 2026 optimizer menu includes MIPROv2 (Bayesian search over instructions and demos), GEPA (a newer reflective optimizer), SIMBA, BootstrapFewShot, and BetterTogether. In DSPy, you write the modules and the optimizer writes the prompts.

Why DSPy matters in 2026

Three forces pushed compiled prompts from research curiosity to production tool. First, hand-prompted quality stopped scaling. Adding more system-prompt instructions saturates, and adding more few-shot examples blows context budgets. The prompts that beat the ceiling were ones humans would not have written; they came from systematic search.

Second, the compile-once-deploy-many pattern matched production workflows. A compiled DSPy program is a frozen artifact. You compile it once on your training data, save the compiled prompts and few-shot examples to JSON, and run the program at inference time without re-running the optimizer. This separates the expensive optimization work from the cheap inference work.

Third, the 2026 optimizer menu broadened. The original BootstrapFewShot was useful but limited, while MIPROv2 runs a Bayesian search over instructions and few-shot examples together. GEPA brought a reflective-optimization approach with its own dedicated tutorials in DSPy’s docs, and SIMBA/BetterTogether cover additional regimes. Picking the optimizer is now a design decision of its own.

Is DSPy still relevant in 2026?

Yes. DSPy 3.4.0 shipped on September 25, 2026, and 3.4.1 followed on October 12, 2026. Version 3.4 made native LM engines the default, with LiteLLM as a compatibility fallback, and added a local CPython interpreter and async ReActV2. The DSPy site lists production users including Shopify, Dropbox, AWS, JetBlue, Replit, and Databricks.

The anatomy of a DSPy program

Signature. A declarative spec for a prompt-able task. Written as a string (“context, question -> answer”) or a class with typed input and output fields. The framework turns the Signature into a prompt automatically; you do not see the prompt unless you ask for it.

Module. A strategy class that wraps a Signature and implements a behavior. dspy.Predict asks the model the question, and dspy.ChainOfThought adds a reasoning step before the answer. dspy.ReAct adds a tool-using loop, and dspy.ProgramOfThought adds code execution. dspy.MultiChainComparison compares multiple ChainOfThought outputs.

Module variants. dspy.BestOfN and dspy.Refine re-run a module and use a reward function to pick the output to keep. The experimental dspy.RLM (Recursive Language Model) gives the model a sandboxed Python REPL so it can explore inputs too large to fit in the prompt.

Program. A composed dspy.Module subclass. You define a forward method that calls other modules in sequence or with branching. Programs can be nested arbitrarily; a Program can call another Program.

LM. The provider abstraction. dspy.LM takes a provider/model string such as openai/gpt-6-luna and is set globally with dspy.configure or for one block with dspy.context. Since DSPy 3.4, native engines run supported routes by default, and LiteLLM stays installed as the compatibility fallback for eligible routes.

Optimizer. The compile algorithm: it takes a program, a metric, and a training set, and returns a new program with frozen prompts and few-shot examples. Common optimizers in 2026: BootstrapFewShot (basic), BootstrapFewShotWithRandomSearch (better), MIPROv2 (Bayesian search over instructions and demos), GEPA (reflective), SIMBA, and BetterTogether (combines prompt search with model fine-tuning).

Metric. A function metric(example, pred) -> bool | float that scores a prediction against a gold example. Required by every optimizer. Can be exact-match, semantic similarity, an LLM-as-judge call, or any custom function.

Trainset and devset. Labeled dspy.Example lists used by the optimizer to compile and evaluate.

A complete DSPy example in Python

MIPROv2 needs DSPy’s optional Optuna extra, so install it with DSPy before you run the example:

pip install -U "dspy[optuna]"

import json
import dspy
from dspy.teleprompt import MIPROv2

# 1. Configure the LM.
dspy.configure(lm=dspy.LM("openai/gpt-6-luna"))

# 2. Define a Signature.
class QASignature(dspy.Signature):
    """Answer the question using the context."""
    context: str = dspy.InputField()
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

# 3. Wrap it in a Module.
class QAProgram(dspy.Module):
    def __init__(self):
        super().__init__()
        self.cot = dspy.ChainOfThought(QASignature)

    def forward(self, context, question):
        return self.cot(context=context, question=question)

# 4. Define a metric and a training set.
def exact_match(example, pred, trace=None):
    return example.answer.lower() == pred.answer.lower()

# qa_pairs.jsonl: one JSON object per line with context, question, and answer keys.
with open("qa_pairs.jsonl") as f:
    rows = [json.loads(line) for line in f]
trainset = [dspy.Example(**row).with_inputs("context", "question") for row in rows]

# 5. Compile with an Optimizer.
optimizer = MIPROv2(metric=exact_match, auto="medium")
compiled_program = optimizer.compile(QAProgram(), trainset=trainset, max_bootstrapped_demos=3)

# 6. Save and load.
compiled_program.save("./compiled_qa.json")
loaded = QAProgram()
loaded.load("./compiled_qa.json")
result = loaded(context="The filing reports headquarters in Austin, Texas.", question="Where is the company headquartered?")

The DSPy compiler works in one loop. compile() makes a fresh copy of your program, runs it on the trainset, scores each output with your metric, and keeps the instructions and demos that score best. The saved JSON holds those instructions, demos, and signature metadata, but not your LM configuration, so the same file loads against whichever model you configure.

How DSPy compares to alternatives

DSPy’s strength is the compile pass. If you do not have a metric and labeled data, the framework’s main feature does not apply. If you do, DSPy often beats hand-prompting. Without a metric yet, compare the wider set of prompt engineering tools.

DSPy vs LangChain and LangGraph

LangChain and LangGraph sit at a different layer from DSPy. LangChain gives you integrations and composable components, and you write the prompt strings yourself. LangGraph is a low-level orchestration framework for long-running, stateful agents with durable execution and human-in-the-loop. DSPy owns the prompt: you declare a Signature and an optimizer writes the instructions and picks the demos.

You can run DSPy and LangGraph in the same app. LangGraph nodes are Python functions, so a node can call a compiled DSPy program loaded from its saved JSON and write the result to graph state. LangGraph keeps routing and state, and DSPy controls what each step asks the model. Our LangGraph explainer covers the graph side.

Production patterns with DSPy

Pattern 1: Compile once, deploy the JSON artifact. Compile the program in CI on a labeled training set. Commit the resulting compiled JSON to your repo or artifact store, then load it at inference time and run the program. The compile pass is the expensive offline step; the inference is cheap.

Pattern 2: Module composition with a non-DSPy retriever. Wrap an existing retriever (LlamaIndex, Haystack, custom) as a callable and use it inside a dspy.Module’s forward method. The DSPy module owns the prompts and few-shot examples; the retriever owns the data layer.

Pattern 3: A/B compile against multiple metrics. Compile the same program with three different metrics (exact match, semantic similarity, LLM-as-judge). Compare the three compiled programs on a held-out set. The metric you optimize for is the metric the program optimizes for; running multiple variants surfaces the metric-quality tradeoff.

Common mistakes when adopting DSPy

• Skipping the metric. A weak metric produces a weak compile. Pick a metric that correlates with what you actually want; if your real goal is calibrated semantic correctness, do not optimize for exact match.
• Picking an optimizer that does not match your budget. The DSPy docs call BootstrapFewShot the safe first try; MIPROv2 and GEPA search harder and can spend hundreds of dollars in LM calls on one run.
• Not separating trainset from devset. Use trainset for the optimizer and devset for the final score. Reusing the same set for both inflates the reported quality.
• Editing the compiled prompts manually. The whole point is that the framework writes the prompts. Editing them by hand reintroduces hand-prompting drift.
• Recompiling on every deploy. Compile in CI on a stable training set. Commit the JSON. Loading is fast; recompiling on every deploy is expensive and slows iteration.
• Forgetting context budget. MIPROv2 can produce few-shot example sets that push prompts over context limits. Configure max_bootstrapped_demos and max_labeled_demos for tight context budgets.
• Using DSPy where prompt-engineering is enough. A trivial single-shot task with a hand-tuned prompt that already hits the bar does not need DSPy. The framework earns its weight when the bar is higher than hand-prompting can reach.

How to trace DSPy with Future AGI

traceAI instruments DSPy with OpenTelemetry spans and sends them to Future AGI’s observability platform or any OTel-compatible backend. Set FI_API_KEY and FI_SECRET_KEY as environment variables, then follow the DSPy integration guide:

pip install traceai-dspy

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_dspy import DSPyInstrumentor

trace_provider = register(
    project_type=ProjectType.OBSERVE,
    project_name="qa-pipeline",
)
DSPyInstrumentor().instrument(tracer_provider=trace_provider)

# Your DSPy programs and modules now emit span trees once instrumentation is registered.

How Future AGI implements DSPy observability and prompt optimization

Future AGI is the reliability layer around a DSPy program: tracing, evaluation, prompt optimization, and a gateway with guardrails on one platform. The Apache-2.0 core is self-hostable, with an Enterprise edition on top. Here is what each part does for a DSPy program:

• DSPy tracing, traceAI (Apache-2.0) instruments DSPy via traceai-dspy for Python, and the wider library covers Python, TypeScript, Java, and C#.
• Module-level evals, custom evals plus built-in evaluators (Groundedness, Detect Hallucination, Context Adherence, Task Completion, Tool Call Accuracy) run on traced spans; BYOK lets any LLM serve as the judge at zero platform fee.

• Prompt optimization, six optimizers (Random Search, Bayesian Search, ProTeGi, Meta-Prompt, PromptWizard, GEPA) score candidate prompts against your evals and ship open source in agent-learning-kit (pip install agent-learning-kit); the same engine optimizes an agent prompt from a Simulation run.
• Gateway and guardrails, the Agent Command Center is an OpenAI-compatible gateway with BYOK routing for the model wrappers DSPy modules call, and runtime guardrails enforce policy on the same plane.

Beyond those four parts, Future AGI also runs pre-production agent simulation, and agent-learning-kit ships a DSPy simulate adapter. The Free plan needs no credit card, and pay-as-you-go scales with usage. SOC 2 Type II, HIPAA BAA, SAML SSO + SCIM, and dedicated support come as paid add-ons (pricing).

Running DSPy in production adds four jobs around the compiled program: tracing, evals, prompt optimization, and a gateway with guardrails. Future AGI puts all four on one platform, so you do not stitch separate tools together to close the loop. For more on the tracing model, read What is LLM Tracing?.

Getting the filings team past its prompt ceiling

The filings team from the opening hit a ceiling that more prompt editing could not move. DSPy moved it by turning the prompt into a compiled program: a Signature, a ChainOfThought module, a metric, and an optimizer pass that wrote instructions and demos nobody on the team would have written.

Keeping that gain is the next job. Trace every compiled run with traceAI, score production outputs with evals that check what your compile metric checks, and recompile when those scores slip. A recompile writes a new JSON artifact, so the improved program ships as a file swap with no code change.

Start free on Future AGI and trace your first compiled DSPy program.

Sources

• DSPy paper: Compiling Declarative Language Model Calls into Self-Improving Pipelines (arXiv)
• Stanford NLP on GitHub
• DSPy optimizers API reference
• DSPy modules guide
• DSPy adapters: how signatures become prompts
• DSPy 3.2.0 release notes (Optuna made optional)
• OpenAI API deprecations