What Is Hardcoded Secrets (in AI Code)?

What Is Hardcoded Secrets (in AI Code)?

Hardcoded secrets are API keys, access tokens, passwords, private keys, or credentials embedded directly in AI code, prompts, notebooks, config files, or agent tool definitions. They are a security failure mode in eval pipelines, generated-code review, production traces, and multi-step agents because the secret can leak into logs, datasets, model context, or tool calls. FutureAGI detects this risk with HardcodedSecretsDetector before a workflow ships or a trace becomes incident evidence.

Why it matters in production LLM/agent systems

Hardcoded secrets turn ordinary AI reliability work into a credential incident. A developer may paste an OpenAI key into a notebook, ship a LangChain tool with a database password in a constructor, or let a coding agent create a .env example that contains a real token. Once committed, that value can be copied into eval datasets, traces, screenshots, prompt examples, customer support tickets, and model context. Rotation is then mandatory, but finding every copy is harder than replacing the original line of code.

The pain is shared. Developers lose time rotating keys and scrubbing repos. SREs see sudden API spend, 401 errors after emergency rotation, or unexplained traffic from an old service principal. Compliance teams need evidence that secrets were not stored in model logs or exported datasets. Product teams feel it as delayed launches when a release gate fails during security review.

Agentic systems make the issue worse because tools and memory expand where credentials can travel. A 2026 coding agent might read repo files, generate migration scripts, call cloud APIs, and summarize traces for a reviewer. If the secret is in a prompt template or tool config, it can be repeated by the model, stored in memory, or used by a downstream action. The common production symptom is not one obvious stack trace; it is a risky string appearing across spans, diffs, artifacts, and eval failures.

How FutureAGI detects hardcoded secrets

FutureAGI anchors hardcoded-secret checks to eval:HardcodedSecretsDetector, implemented by the HardcodedSecretsDetector class. It is a security detector for hardcoded secrets and credentials, including CWE-798, CWE-259, and CWE-321. In practice, teams run it on generated code, prompt templates, notebook cells, tool manifests, and config diffs before those artifacts enter a dataset or production route.

Real example: a coding assistant generates a FastAPI service that calls a LangChain tool. The response includes OPENAI_API_KEY = "sk-..." inside a sample module, and the same value appears in the trace summary. FutureAGI records the generation through the traceAI langchain integration; the trace includes the prompt version, artifact name, agent.trajectory.step, and route. A post-guardrail checks the generated artifact with HardcodedSecretsDetector, then the release gate uses SecretsSecurityScore to stop the merge until the key is removed, rotated, and replaced with a secret-manager reference.

FutureAGI’s approach is workflow-aware: scan the places where AI systems create, move, or summarize code, not only the final repository. Unlike a repo-only Gitleaks scan, this catches credential exposure while it is still in a model response, eval dataset, trace annotation, or agent scratchpad. The engineer can then block the output, quarantine the affected trace, add the example to a regression eval, and alert the owner of the exposed credential.

How to measure or detect it

Measure hardcoded secrets as detector findings plus operational impact:

• HardcodedSecretsDetector — flags code or config text that appears to contain hardcoded credentials; treat any finding as block-and-review until triaged.
• SecretsSecurityScore — tracks security score movement for hardcoded credential risk across releases, datasets, and generated-code cohorts.
• Trace evidence — inspect artifact paths, prompt version, agent.trajectory.step, tool.output, route name, and whether the value reached logs or eval data.
• Dashboard signals — detector-fail-rate-by-route, secrets findings per generated artifact, rotation count, time-to-remediation, and missed-finding rate from manual review.
• User-feedback proxy — support escalations or customer reports that mention leaked keys, unexpected billing, or unauthorized tool activity.

from fi.evals import HardcodedSecretsDetector

detector = HardcodedSecretsDetector()
finding = detector.evaluate(input=code_candidate)
if finding:
    print("block_review_rotate")

Do not stop at detection. A finding should trigger owner notification, credential rotation, log scrubbing, regression coverage, and a check that the same value did not enter traces or annotation exports.

Common mistakes

The repeated failure pattern is treating hardcoded secrets as a repository hygiene issue instead of an AI workflow issue.

• Scanning only committed code. Generated answers, notebooks, traces, prompt examples, and eval rows can expose the secret before any commit exists.
• Allowing realistic sample keys. Examples that look valid train reviewers to ignore dangerous strings and can bypass naive placeholder checks.
• Saving raw traces indefinitely. A blocked output can still become a stored incident if trace export captures the credential first.
• Rotating without backtracking copies. The original key may still live in prompt versions, screenshots, cached responses, and annotation queues.
• Trusting allowlists too broadly. A test token prefix can become a production token prefix after provider changes or internal key formats shift.