Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

Applications that search, read documents, or inspect code place externally supplied text inside a language model's context. That text can contain instructions from someone other than the user or application developer. When the model treats those instructions as authoritative, ordinary retrieval becomes a route for influencing answers and downstream actions.

The paper studies this application-level boundary, rather than only direct attempts to override a chatbot through its public prompt box. Its scope includes effects on the user, the integrated application, and other systems reachable through the application's tools.

A threat model distinguishing indirect injection delivery paths, attacker objectives, and affected parties.

A threat taxonomy covering information exposure, fraud, intrusion, malware-related behavior, content manipulation, and availability.

A collection of controlled demonstrations and historical application tests, including multi-stage and encoded delivery, with prompts and transcripts for inspection.

The analysis follows the path from an attacker-controlled source to the application's retrieval or context-building step. The model then processes that source alongside the legitimate interaction and may use the application's interfaces or generated answer to pursue the injected objective. The critical transition is from consuming external data to accepting behavioral direction.

The authors distinguish passive delivery through retrieved sources from active delivery through material such as messages. They also discuss user-mediated and hidden delivery. The taxonomy organizes consequences by the threatened asset or behavior, so that different delivery techniques can be compared without equating them.

Controlled prototypes provide selected search, page-reading, URL-retrieval, email, address-book, and memory interfaces. Prepared responses allow the authors to observe how an injected instruction affects a subsequent interaction. Additional tests examine page context supplied to Bing Chat's sidebar and code context supplied to GitHub Copilot; these tests concern the integrations available during the study.

The synthetic applications use prepared tool content and mock targets, without issuing requests to real external systems. Application demonstrations use controlled local material, including local HTML for sidebar tests. The authors inspect conversation and tool-use traces against the intended attacker objective across the threat categories.

This is primarily a qualitative feasibility study, not a standardized success-rate benchmark. Multi-stage and encoded cases investigate delivery mechanisms. The paper leaves systematic measurement across prompt variations, repeated generations, interaction histories, and user persuasion to future work, and notes that changing black-box services limit exact reproduction.