Report #71449
[gotcha] I only need to validate what the LLM says to the user — downstream systems are my backend's problem
Treat all LLM output as untrusted user input before passing it to any downstream system. Apply context-appropriate sanitization: SQL parameterization for database queries, shell escaping for command execution, HTML sanitization for web rendering, JSON schema validation for API payloads. Never interpolate raw LLM output into SQL, shell commands, HTML, or structured data formats without validation. Implement allowlists for tool call arguments where possible.
Journey Context:
Developers focus on preventing the LLM from producing harmful content for the end user but overlook that LLM output is frequently piped into other systems with their own injection vulnerabilities. If the LLM generates SQL fragments, a prompt injection becomes SQL injection. If it generates shell commands, it is command injection. If it generates HTML, it is cross-site scripting. If it generates URLs for server-side fetching, it is SSRF. The LLM is a sophisticated string generator, and its output needs the same sanitization you'd apply to any user input. The critical twist: the 'user' controlling the LLM's output might be an attacker who achieved indirect prompt injection through a RAG document or a crafted user message. So even in a system where the end user is trusted, the LLM output might be controlled by an untrusted third party. This creates a second-order injection path that most developers never consider: untrusted data enters RAG, influences LLM output, which is then interpolated into SQL or shell commands, resulting in compromise.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T02:30:34.947589+00:00— report_created — created