What Is SQL Injection in LLM Apps?

What Is SQL Injection in LLM Apps?

SQL injection in LLM apps is a security failure mode where a model, agent, or text-to-SQL layer produces database queries that let untrusted input alter SQL logic. It shows up in eval pipelines, production traces, and Agent Command Center guardrails when generated SQL can reach a database tool. FutureAGI maps this risk to eval:SQLInjectionDetector, which detects SQL injection vulnerabilities before execution and helps teams regression-test risky agent paths.

Why it matters in production LLM/agent systems

SQL injection in an LLM app turns a model output into a database boundary failure. The model may translate a natural-language request into SQL, patch a query template, choose a database tool, or write application code that builds SQL strings. If untrusted text can change the final query, the result can be cross-tenant data reads, bulk export, destructive writes, privilege escalation, or silent analytics corruption.

Two failure modes dominate incidents. Query logic takeover happens when attacker-controlled input changes WHERE, UNION, comment, or statement boundaries. Tool-side data exposure happens when an agent calls a database tool with SQL that broadens the row scope beyond the user’s authorization. The first looks like classic CWE-89; the second is easier to miss because the model appears to be using an approved tool.

Developers feel it as flaky text-to-SQL behavior: one phrasing returns the correct customer row, another returns every row in the table. SREs see query timeouts, p99 latency spikes, rejected statements, unexpected row counts, or a sudden rise in database permission errors. Compliance teams need a trace that proves which prompt, route, model, tool input, and database statement produced the unsafe query.

It matters more for 2026 multi-step agents because SQL is rarely generated in one clean turn. The agent may inspect schema, read prior tool output, rewrite the query, and retry after an error. Each step can carry attacker-controlled text into the next query unless the boundary is checked.

How FutureAGI handles SQL injection in LLM apps

FutureAGI handles this through the eval:SQLInjectionDetector surface. The inventory class is SQLInjectionDetector, a security detector for SQL injection vulnerabilities mapped to CWE-89. In an LLM workflow, the detector belongs at every point where model-controlled text can become SQL: text-to-SQL generation, function-call arguments, agent tool inputs, and code produced by a coding assistant.

A practical example is a support analytics agent instrumented with traceAI-langchain. The trace records the user request, agent.trajectory.step, selected tool, tool.input.sql, route name, model, and final database statement. A user asks, “Show my invoices for account 42’ OR ‘1’=‘1.” The model generates:

SELECT * FROM invoices WHERE account_id = '42' OR '1'='1';

Before the database tool runs, Agent Command Center applies a pre-guardrail on the analytics-sql route. SQLInjectionDetector evaluates the generated query and the surrounding tool argument. If the detector flags a SQL injection vulnerability, the route blocks execution, returns a safe fallback, records the guardrail decision on the trace, and sends the sample to a security review queue.

FutureAGI’s approach is to connect the detector result to the exact model and tool path that created the query. Compared with a Semgrep-only scan, which is strongest for code patterns before deployment, this catches runtime SQL assembled by an agent after schema inspection, retrieval, or tool retries. The engineer’s next step is concrete: add the prompt, generated SQL, trace ID, detector verdict, and expected safe refusal to a regression dataset; then fail release if a new prompt, model, or route can execute the same unsafe query.

How to measure or detect it

Measure SQL injection in LLM apps at the generated-query boundary, not only at the chat boundary:

• SQLInjectionDetector evaluator - detects SQL injection vulnerabilities, including CWE-89-style query manipulation, in generated SQL or query-building paths.
• Trace fields - inspect agent.trajectory.step, tool name, tool.input.sql, route name, model version, database statement, and guardrail decision.
• Dashboard signals - track sql-injection-eval-fail-rate, unsafe-query-block-rate, database-error-rate-after-generation, and row-count-anomaly rate by tenant or route.
• Authorization proxy - compare the rows returned by generated SQL against the caller’s allowed tenant, role, and object scope.
• User-feedback proxy - review escalations where an assistant returned another account’s records, over-broad analytics, or a refusal after a database request.

from fi.evals import SQLInjectionDetector

query = "SELECT * FROM users WHERE email = 'a' OR '1'='1'"
detector = SQLInjectionDetector()
result = detector.evaluate(input=query)
print(result)

Detection should be paired with database controls: parameterized queries, least-privilege credentials, read-only analytics roles, query allowlists for sensitive routes, and blocking before tool execution.

Common mistakes

SQL injection in LLM systems is usually a boundary-design problem, not just a prompt-quality problem.

• Trusting text-to-SQL because it passed demo questions. Attack phrasing must include comments, stacked statements, tautologies, and schema-discovery attempts.
• Checking the user prompt but not the generated query. The dangerous string may appear only after the model rewrites the request.
• Letting the agent retry unsafe SQL. Database errors can become a feedback loop that teaches the model table names and column shapes.
• Running generated SQL with broad credentials. A read-only, tenant-scoped role limits blast radius when detection misses a query.
• Treating syntax validity as safety. A syntactically valid query can still bypass authorization or widen result scope.