Skip to content

LLM05: add Markdown auto-render exfiltration to output handling guidance#9

Open
ottosulin wants to merge 1 commit intoGenAI-Security-Project:mainfrom
ottosulin:upgrade/LLM05-prevention-strategies
Open

LLM05: add Markdown auto-render exfiltration to output handling guidance#9
ottosulin wants to merge 1 commit intoGenAI-Security-Project:mainfrom
ottosulin:upgrade/LLM05-prevention-strategies

Conversation

@ottosulin
Copy link
Copy Markdown

@ottosulin ottosulin commented Apr 26, 2026

Adds prevention coverage for the Markdown image / link preview exfiltration pattern that has affected ChatGPT, Bing, Google Bard, NotebookLM, Writer.com, Amazon Q, GitHub Copilot Chat, Gemini, and Cursor.

CSP (current item 6) can mitigate this in browser-based UIs when img-src is properly used, but many LLM clients are not browsers (IDEs, native apps, terminal clients, email clients) and need application-layer controls at the renderer.

Adds:

  1. Impact condition (auto-fetched external resources in client renderers)
  2. Common example of the vulnerability (chat UI auto-renders Markdown images, exfil via URL)
  3. Prevention strategy (disable auto-render of Markdown images / link previews / iframes by default; allowlist or server-side image proxy when needed)
  4. Two references (Embrace The Red Copilot Chat writeup, Simon Willison Markdown exfiltration tracker)

@RicoKomenda
Copy link
Copy Markdown
Collaborator

Looks good from, reviewed in slack channel from @GTKlondike and myself

rocklambros pushed a commit that referenced this pull request May 2, 2026
- Added ALGEN (ACL 2025) and ZSInvert (arXiv:2504.00147) findings
- Updated recovery rate statistics (50-92% word recovery)
- Added GDPR/HIPAA compliance framing
- Added encryption and rate limiting mitigations (ref #9, #10)

Signed-off-by: azizrebhi <154744962+azizrebhi@users.noreply.github.com>
@rocklambros
Copy link
Copy Markdown
Collaborator

@ottosulin — thanks for the contribution.

@RicoKomenda @GTKlondike — LLM05 entry leads, please review.

@rocklambros (project owner) would like entry leads to review this content before it merges. Once your review is complete, please tag @rocklambros and let him know it's ready for merge.

@RicoKomenda
Copy link
Copy Markdown
Collaborator

@ottosulin I think you need to resolve the conflicts. After that, we can merge this PR. :)

@ottosulin ottosulin force-pushed the upgrade/LLM05-prevention-strategies branch from 0ae92c3 to 21f8b42 Compare May 3, 2026 13:12
@ottosulin
Copy link
Copy Markdown
Author

@RicoKomenda fixed!

@RicoKomenda
Copy link
Copy Markdown
Collaborator

@ottosulin per policy you need to have verified commits through, please do that on both your PRs. 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants