What Is a Meta-Prompt?

What Is a Meta-Prompt?

A meta-prompt is a prompt that tells another model, evaluator, or optimizer how to create, critique, or improve prompts for a target task. It is a prompt-engineering construct used in optimization pipelines, regression evals, and agent traces rather than a direct end-user instruction. FutureAGI surfaces it through optimizer:MetaPromptOptimizer, where a teacher model studies failed examples, rewrites the task prompt, and checks candidates with metrics such as PromptAdherence and TaskCompletion.

Why meta-prompts matter in production LLM and agent systems

Meta-prompt failures usually look like confident prompt churn. The optimizer keeps proposing new wording, the offline score rises, and production quality still drops because the meta-prompt rewarded the wrong behavior. Two common failure modes are instruction drift, where rewrites weaken a policy or output contract, and eval overfitting, where the teacher model optimizes for the examples it just saw instead of the distribution users actually hit.

Developers feel this as unstable prompts that cannot be explained after a release. SREs see wider latency variance when rewrites add long reasoning clauses or examples. Product teams see regressions in task completion for a customer cohort that was absent from the eval set. Compliance teams worry when a rewrite removes refusal boundaries, citation requirements, or data-handling instructions that were part of the original system prompt.

The symptoms show up in logs as prompt versions with rising token counts, falling PromptAdherence, higher schema-validation failure rate, or a sudden gap between offline eval scores and live thumbs-down rate. In 2026-era agent pipelines, the risk compounds because one rewritten planner prompt can alter retrieval, tool selection, final synthesis, and fallback behavior. A meta-prompt is therefore not just “a prompt about prompts.” It is a production control surface for how prompts change.

How FutureAGI handles meta-prompts

FutureAGI handles meta-prompts through the agent-opt optimizer surface optimizer:MetaPromptOptimizer. The inventory class is MetaPromptOptimizer, whose job is teacher-model rewrites: a stronger teacher analyzes failed examples and proposes a better task prompt. Unlike ProTeGi, which uses textual gradients and beam search, MetaPromptOptimizer asks the teacher to inspect failure rows directly and produce a candidate rewrite that can be evaluated against the same cohort.

A typical workflow starts with a baseline prompt for a support agent, a held-out eval dataset, and target metrics such as PromptAdherence, TaskCompletion, and cost per trace. The engineer runs MetaPromptOptimizer over failed rows where the agent missed policy constraints or chose an incomplete response. FutureAGI records candidate prompts as prompt assets through fi.prompt.Prompt, then compares each candidate against the baseline on the same eval split. The exact signals are the evaluator scores, prompt version, model route, and llm.token_count.prompt for token-cost regression.

FutureAGI’s approach is to treat the meta-prompt as auditable optimization code. If a candidate raises TaskCompletion from 0.74 to 0.82 but increases prompt tokens by 45%, the engineer can reject it, add a cost objective, or pass the candidate into GEPAOptimizer for multi-objective search. If the candidate passes, it is committed as a prompt version and watched in production traces before broad rollout.

How to measure or detect meta-prompts

Measure a meta-prompt by the quality of the prompts it produces, not by how polished its wording sounds:

• PromptAdherence: returns whether the output followed the candidate prompt’s instructions; compare mean score and fail rate against the baseline prompt.
• TaskCompletion: scores whether the agent completed the user goal; require improvement on a held-out cohort, not only the optimizer’s training rows.
• Score delta by cohort: split by customer tier, task type, locale, and tool path so one cohort does not hide another.
• Prompt-token regression: track llm.token_count.prompt for each candidate; better quality with a much larger prompt may still lose at scale.
• Live proxy metrics: monitor thumbs-down rate, escalation rate, fallback-response rate, and eval-fail-rate-by-cohort after rollout.

from fi.evals import PromptAdherence

evaluator = PromptAdherence()
result = evaluator.evaluate(
    input=task_input,
    output=model_output,
    prompt=candidate_prompt,
)
assert result.score >= 0.90

Common mistakes with meta-prompts

Most meta-prompt mistakes come from treating the teacher rewrite as authority instead of a hypothesis that needs measurement.

• Optimizing against the same failed rows that generated the rewrite; the next prompt memorizes the eval set instead of improving general behavior.
• Asking the teacher to rewrite the whole system prompt at once; smaller edits are easier to attribute, score, and roll back.
• Scoring only PromptAdherence; a prompt can follow instructions while reducing TaskCompletion, grounding, safety, or latency.
• Letting the meta-prompt remove policy text because it looks redundant; those clauses often exist for audit, refusal, or routing reasons.
• Shipping the winning candidate without prompt versioning; incident review then cannot tie live regressions to a specific rewrite.