Text to SQL LLM Systems: How to Evaluate Generated Queries Before They Run

A SQL query that fails loudly is a good day. The queries to worry about are the ones that run, return a clean table of numbers, and are wrong.

A finance analyst asks a text to SQL assistant for last quarter’s refund total. The model writes a query, the database accepts it, and a number comes back in under a second. The number is wrong by a factor of three, because the query joined orders to refunds on the customer ID instead of the order ID and multiplied every row.

Nothing errored. No exception was raised, no timeout fired, no log line turned red. The only signal that something went wrong was the number itself, and the person reading it had no way to know.

This post is about closing that gap. It covers where text to SQL breaks, a taxonomy of those failures, how to build a validation layer that runs before the database does, and why security scanning and correctness evaluation are two separate jobs.

Key takeaways

• Syntactic validity is the weakest possible correctness signal for generated SQL, because most wrong queries are still valid.
• The failure modes that matter most (hallucinated schema references, wrong joins, fabricated filter values) all produce queries a database will happily run.
• Pre-execution validation catches schema-level errors cheaply, before any rows are scanned.
• Post-execution scoring is the only reliable way to catch wrong business logic the schema itself does not encode, but it costs a real query against real data.
• Guardrails stop SQL injection. Evaluation catches a query that is perfectly safe and still returns the wrong answer. You need both, separately.

Why “It Ran Without an Error” Isn’t the Same as “It’s Correct”

Most text to SQL systems treat execution as the pass condition. The query compiled, the database returned rows, the job is done. That works for a compiler, where a program that runs is a program that parsed. It does not work for SQL, where the language is expressive enough that almost any confident guess is legal.

The gap comes from what the database is checking. It checks that identifiers resolve and types line up. It does not check that you meant this table, or that this join produces one row per order rather than one row per order line. Those are questions about intent, and the query planner has no access to intent.

Benchmark Accuracy Versus Production Reality

Public leaderboards make this look more solved than it is. The trouble is that at least the two most-scrutinized benchmarks carry heavy annotation errors, which means reported scores are measured against reference queries that are frequently wrong.

A January 2026 paper by Jin, Choi, Zhu and Kang, Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards, audited the reference data directly. They found an annotation error rate of 52.8% in BIRD Mini-Dev and 62.8% in Spider 2.0-Snow.

The effect on rankings is the part that should worry anyone using a leaderboard to pick a model. Re-evaluating sixteen open-source agents from the BIRD leaderboard against corrected annotations moved relative performance between −7% and 31%, and shifted individual ranks by as much as nine positions in either direction.

Even setting the annotation problem aside, benchmark schemas are small, clean and documented. Production schemas have thirty years of history, columns named flag_2, two tables that both look like they hold customers, and business logic that lives in a Confluence page nobody has read since 2023.

Syntactically Valid, Semantically Wrong

This is the failure mode that matters, and it is the one execution cannot detect. The query is well-formed, every identifier resolves, and the answer it returns has nothing to do with the question that was asked.

Consider a request for average order value. AVG(line_item_price) and AVG(order_total) are both valid SQL against a normal e-commerce schema. Both return a number. One answers the question and the other answers a different question that nobody asked, and the difference is invisible in the output.

Or take a filter on status = 'complete' when the column actually stores 'COMPLETED'. The query runs, returns zero rows, and a dashboard renders a confident zero. That is worse than an error, because a zero looks like an answer.

The general shape here is familiar from any generative system: the model produces something plausible rather than something checked. We go deeper on that pattern in our deep dive on LLM hallucination, and generated SQL is one of its sharpest instances, because the output is executable.

A Failure-Mode Taxonomy for LLM-Generated SQL

Before you can validate anything, you need to know what you are validating against. Grouping text to SQL failures by what they look like at the database boundary makes it clear which ones a pre-execution check can catch.

Schema Hallucinations

The model references a table or column that does not exist. It asks for customers.lifetime_value when the schema has customers.ltv_cents, or invents a subscriptions table in a database that never had one.

This is the friendliest failure in the taxonomy, because the database rejects it outright. It is also the easiest to catch early: you have the schema, the query names identifiers, and comparing the two is a lookup, not a judgment call.

The reason it still causes incidents is retry loops. An agent that gets an “unknown column” error and rewrites the query will often guess again rather than re-read the schema, burning attempts until it lands on something that resolves, correct or not.

Wrong Joins and Wrong Grain

Here the query is entirely legal and entirely wrong. Joining orders to order_items and then summing orders.total multiplies every order’s total by its line-item count. The result is a real number, computed from real data, that overstates revenue.

Grain errors are the most dangerous category because their output looks reasonable. Revenue is up. Nobody questions a number that moved in the direction they expected, and the query stays in production for months.

Catching this before execution needs more than an identifier lookup. You need to check the join keys against the actual foreign-key relationships in the schema and flag joins that do not follow a declared path.

Fabricated Filter Values

The schema is right, the joins are right, and the WHERE clause filters on a value that does not exist in the column. Region 'APAC' in a table that stores 'Asia-Pacific'. Status 'refunded' in a column whose enum only has 'REFUND_COMPLETE'.

These queries return empty result sets or partial ones. An empty set at least looks suspicious; a partial one does not, and a filter that silently drops half the rows produces a number that is wrong in a direction nobody will question.

Some of this is checkable pre-execution if you have column statistics or enum definitions available. Much of it is not, and that is a real limit of the approach.

Table 1 — Text to SQL failure mode taxonomy

Building a Pre-Execution Validation Layer for Text to SQL

A validation layer sits between the model and the database. It takes the generated query plus the schema, runs a series of checks, and only forwards a query that passes. The design goal is to spend cheap compute on checks so you spend expensive compute on fewer bad queries.

Dry-Run and Parse Checks

Start with the cheapest thing available: parse the SQL. A parser catches malformed syntax without touching a connection, and it also gives you a structured representation of the query, which is what every later check needs.

Most databases then offer a way to plan a query without running it. EXPLAIN, or a dry-run flag on the API, resolves every identifier against the catalog and returns an error if anything is missing, at no data cost.

That combination alone eliminates most of the schema-hallucination row of the taxonomy before a single row is scanned, though EXPLAIN and dry-run semantics vary by engine and do not cover every case, such as some dynamic SQL. It is still the highest-value check per unit of effort in the whole pipeline.

Groundedness Scoring Against the Real Schema

Parse checks tell you the identifiers resolve. Groundedness scoring asks a harder question: are these the right identifiers for this question, given the schema and the relationships in it.

In practice you pass the model-generated SQL, the natural-language question and the schema definition to an evaluator, and score whether the query’s references and relationships are supported by the schema. This is where wrong joins start becoming visible, because a join that does not follow a declared relationship is not grounded in the schema even when it parses.

Scoring like this is a judgment task, so it usually runs as a model-based evaluator rather than a rule. If you are choosing between the two approaches, our post on deterministic versus LLM-judge evals covers the tradeoff in detail.

Table and Join Verification

The last pre-execution check is narrow and specific: did the query select the right tables. Not whether the SQL is correct overall, just whether the tables it touches are the tables that could plausibly answer the question.

Isolate it as its own check, because it fails loudly and readably. When a query about refunds never touches the refunds table, you do not need a nuanced score to know something is wrong, and the fix is usually better schema context in the prompt rather than a different model.

What’s the Difference Between Pre-Execution and Post-Execution Evaluation?

Pre-execution checks run against the query text and the schema. Post-execution scoring runs against the query text, the returned rows and, where you have them, known-correct answers. They catch different things, and treating either as sufficient leaves a real gap.

What Each Catches and What Each Misses

Pre-execution validation is strong on anything the schema can adjudicate. Missing tables, missing columns, joins that ignore declared relationships. It is weak on anything requiring knowledge of the data itself or of the business definition behind a metric.

Post-execution scoring is the reverse. Once you have rows, you can compare against a reference result, check row counts against expectations, and catch a definitional error that no amount of schema inspection would reveal. What you cannot do is un-run the query.

Cost and Latency Tradeoffs

The cost argument favors pre-execution more than it first appears. A parse and a plan cost milliseconds and no data scan. A model-based groundedness score costs one extra inference call, which is small next to the generation call you already made.

Post-execution scoring costs a real query. On a warehouse billed by bytes scanned, a wrong query against a large fact table is expensive whether or not the answer was right, and running every candidate query to evaluate it multiplies that.

The practical arrangement is to run pre-execution checks on every query in production and post-execution scoring on a test set offline, plus a sample of production traffic. You get schema safety everywhere and semantic coverage where you can afford it.

Table 2 — Pre-execution versus post-execution evaluation

Guardrails Versus Evaluation: Two Different Jobs

These get conflated constantly, usually because both sit in the same part of the pipeline and both can block a query. They answer completely different questions and neither substitutes for the other.

Security Scanning Catches Injection Attempts

A guardrail asks whether the input or output is hostile. For text to SQL that means injection payloads smuggled through a natural-language question, shell commands, path traversal, and similar attack patterns aimed at the query layer.

This is a pattern-matching and classification problem with a clear adversary. It runs inline, it runs fast, and it blocks. We cover the broader category in our post on runtime guardrails for agents.

Correctness Evaluation Catches Safe but Wrong Queries

An evaluator asks whether the query answers the question. A SELECT that joins the wrong way and returns triple the real revenue contains no attack, passes every security scanner cleanly, and is still a production incident.

The two failure classes have nothing in common except the pipeline stage. Injection detection tuned to catch bad joins would fire constantly on legitimate queries, and a correctness scorer has no concept of an adversary. Keep them as separate steps with separate thresholds and separate owners.

Future AGI

Future AGI publishes a text-to-SQL cookbook that shows this pattern working end to end. It is a working reference, not a description of one.

The setup builds a LangChain SQL agent against a seven-table e-commerce schema with realistic constraints and relationships, then runs it against ground-truth test cases. Per question, the agent lists available tables, fetches schema and sample rows, generates SQL, and validates syntax before the database executes anything. That validation step is the same check this post argues for, instrumented so you can see it happen.

The cookbook applies five named evaluators through Future AGI’s tracing system, per the same documentation:

• TEXT_TO_SQL measures how accurately the agent converts user questions into syntactically correct and semantically appropriate SQL queries. It is a standalone built-in rather than a cookbook fixture, and it takes only the natural-language question and the generated SQL — no database connection, no gold query, no execution. That is the pre-execution check this post argues for, in two arguments.
• GROUNDEDNESS checks that the generated SQL references valid tables, columns and relationships within the actual schema.
• DETECT_HALLUCINATION identifies SQL referencing non-existent database structures.
• COMPLETENESS verifies that the response fully addresses every part of the SQL request.
• table_checker, a custom evaluator, confirms the agent picked appropriate tables for satisfying the user’s request.

The reported results from that run are a useful reality check on the taxonomy above. Table identification scored 75–80%, hallucination metrics stayed minimal, and text-to-SQL accuracy ranged from 25% to 75% depending on query complexity, with query execution averaging 43.6 milliseconds across a 29.5–77.1 millisecond range. Groundedness ranged from 53% to 81%, and that floor is the number worth sitting with: on a clean seven-table schema, with the agent handed the schema and sample rows, roughly half the generated queries were not fully grounded in it. Production schemas are neither clean nor seven tables. The spread across complexity, and even within the timing numbers themselves, is exactly the pattern that a single headline accuracy number hides.

Security scanning is a separate module and stays separate. The guardrails module documents a CodeInjectionScanner that identifies SQL injection, shell commands, path traversal, SSTI and XXE, alongside scanners for jailbreaks, secrets and prompt injection. That scanner will not tell you a join is wrong, and GROUNDEDNESS will not tell you a question carried an injection payload. Running both is the point.

Conclusion

A text to SQL system that runs without an error can still be returning confidently wrong answers, and the wrong answers are the ones nobody reports, because there is nothing to report. The absence of an exception is not evidence of correctness.

The fix is structural rather than clever. Parse and plan every query before it executes, score it for groundedness against the real schema, verify the tables and joins independently, and keep a post-execution comparison against known-correct results on a test set and a slice of live traffic.

And keep the two jobs apart. Injection scanning protects you from an attacker. Correctness evaluation protects you from your own model on an ordinary Tuesday, which is the failure you are far more likely to meet.