Skip to content
    All guides

    AI Architect Case Files

    Case: EchoLeak, Indirect Injection in an AI Assistant

    A sourced case on EchoLeak (CVE-2025-32711): how one email could steer Microsoft 365 Copilot toward data theft, and the layered defences one paper proposes.

    By Published 7 min read

    • ai-architect
    • case-file
    • industry-saas-b2b
    • pattern-prompt-injection-defense
    • failure-injection
    • kind-public
    Share

    Public case (sourced). Based on the paper "EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System" (Reddy and Gujral, arXiv 2509.10540), the CVE-2025-32711 record and Aim Labs' own post; every fact below is attributed to one of them.

    This case belongs to Security and Threat Modeling for AI Systems. The paper analyses already-public data and says its authors did not reproduce the attack. Aim Labs' pages return HTTP 403 to automated fetches, so they were read as Wayback Machine copies. Defensive framing only: this page leaves out the wording of the injected text, the addresses and the endpoint names.

    Context

    • The paper describes Microsoft 365 Copilot as an assistant that helps users draft emails, summarize documents and query organizational knowledge.
    • It says researchers at Aim Security disclosed EchoLeak in June 2025 and that Microsoft assigned CVE-2025-32711.
    • Its threat model: the attacker cannot query Copilot and has no insider access, but can email the victim's organization. The victim is an employee with Copilot enabled; Copilot can read what that employee can read.

    Problem statement

    Item What the sources say
    CVE description "Ai command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network." (CVE record)
    Weakness CWE-74, improper neutralization of special elements in output used by a downstream component (CVE record)
    Published 2025-06-11 (CVE record)
    Severity Microsoft's entry in the record: CVSS 3.1 base score 9.3, critical, user interaction none. NVD's own score in the same listing: 7.5, high

    This page's framing of the defence goal: keep outside text from steering the assistant and stop its output moving data out.

    Constraints

    • The paper assumes the attacker's email is text only, with no malware attachment, and is delivered normally.
    • Three defences were already in place, per the paper: an input classifier called XPIA (Cross Prompt Injection Attempt), filtering that redacts certain output such as external links, and a browser content security policy (CSP) that blocks loads from unapproved domains.
    • The paper says Microsoft's filters and models are proprietary, so some of its conclusions are inferential.

    Why the traditional approach fell short

    The paper says prompts are free-form natural language, so they cannot be sanitized like structured input, and that unlike SQL injection prompt injection "cannot be reliably mitigated through schema validation". Ordinary email security is assumed to pass the message.

    Its discussion says the classifier, the link redaction and the CSP rules "each failed under modest adversarial pressure". The paper names no other Copilot controls and does not say how XPIA works.

    Options considered

    Option What the paper says Verdict
    A. Rely on the injection classifier alone The paper says the injected text was phrased to evade it Rejected: one layer, and it failed
    B. Keep external email out of answers about internal work by default (simplest, no new model) "If emails from unknown external senders are never used to answer questions about internal projects, EchoLeak fails by design" Accepted as the default, at a cost in coverage: the paper says it reduces functionality, and Aim's post says excluding external email "also degrades the capabilities of Copilot"
    C. Layered controls: partitioning, an output policy gate, strict CSP, a signed media proxy, confirmation for external fetches Proposed in the paper; its Table 3 is labelled theoretical Accepted as the complement to B

    Solution architecture

    Figure 1. The controls and steps the paper names

    flowchart LR accTitle: Components and controls in the EchoLeak chain accDescr: An external sender's email lands in the mailbox. Retrieval pulls it into the prompt together with internal items, and an input classifier checks the prompt before the model. The model's answer passes link redaction and is rendered by the client, which applies a content security policy. An allow-listed Microsoft service is the route the paper says carried the final request to an attacker-controlled server outside the organisation. The paper does not state where the classifier runs, so it is drawn at the input. subgraph OUT["Outside the organisation"] S["External sender"] A["Attacker-controlled server"] end subgraph IN["Inside the organisation"] MB["Mailbox and files"] R["Retrieval"] C["Input classifier (XPIA)"] M["Copilot model"] F["Output handling: link redaction"] U["Client (Outlook or Teams) with CSP"] end P["Allow-listed Microsoft service"] S -->|"email, text only"| MB MB --> R --> C --> M --> F --> U U -->|"image fetch allowed by CSP"| P P -.->|"fetches on the client's behalf"| A

    How to read this: each box between the mailbox and the client is a control or step the paper names; the dotted arrow is the path out of the organisation.

    Figure 2. The chain as the paper describes it, at defender level

    sequenceDiagram accTitle: EchoLeak chain at defender level accDescr: An external email arrives and is later retrieved when an employee asks an ordinary question. The answer carries a link or image reference that encodes data from the employee's context. The client fetches it automatically, an allow-listed Microsoft service performs the request, and data reaches the attacker's server. Notes mark the control the paper says failed at each stage. autonumber participant S as External sender participant M as Mailbox participant E as Employee participant C as Copilot participant U as Client participant P as Allow-listed service participant A as Attacker server S->>M: email, delivered normally E->>C: ordinary question C->>M: retrieve context M-->>C: internal items plus the external email Note over C,M: Paper: the text evaded the XPIA classifier C-->>U: answer with a reference-style link or image Note over C,U: Paper: link redaction did not remove this form U->>P: automatic image fetch, no click Note over U,P: Paper: CSP allows this Microsoft domain P->>A: request to the attacker-controlled address

    How to read this: the employee does nothing unusual; every failure is a control that let one step through.

    How AI solved it, step by step

    The stepper compares the paper's account of the chain with its proposed layers, which it says were not comprehensively evaluated.

    EchoLeak: the chain and the proposed layers (interactive when JavaScript is on)

    As disclosed (paper's account)

    1. Email arrives. An external text-only email is delivered normally and sits in the mailbox.
    2. Retrieval pulls it in. Later an ordinary question makes retrieval place the email in the prompt beside internal content.
    3. Classifier passes it. Phrased like a normal request to the human recipient, the text evaded the XPIA classifier.
    4. Answer encodes data. The answer carries a reference-style link or image that encodes details from the employee's context; redaction did not remove this form.
    5. Client fetches by itself. The client fetches the image automatically, so no click is needed.
    6. Allow-listed route out. A CSP-allowed Microsoft service made the request on the client's behalf, so it reached the attacker's server.

    With the paper's proposed layers

    1. Email arrives, tagged external. Provenance records that the sender is outside the organisation.
    2. Partitioned retrieval. External text is marked non-authoritative and, by default, not used for internal questions.
    3. Output policy gate. Links and images are checked against an allow-list and a safe Markdown subset.
    4. Strict CSP and signed proxy. The client cannot load from unknown origins; permitted fetches go through a signing proxy.
    5. Confirmation and monitoring. External fetches need consent, and unusual egress is logged.

    Under the hood

    The paper's account is that retrieved content is inserted into the prompt and that, because "LLMs treat all prompt content as potential instructions", malicious text in retrieved data can cross a trust boundary. Its discussion calls the result a "collapse of the boundary between content and commands". Aim Labs' name for it is "LLM Scope Violation": instructions from untrusted input make the model attend to trusted data in its context "without the user's explicit consent", which Aim says breaks the Principle of Least Privilege.

    The shape is three checks on input, output and network, each beaten separately. The paper's Table 3, its own judgement of which proposed defences cover which step, is labelled theoretical.

    What went wrong / failure modes

    Failure (as the paper reports) What a defender takes from it
    The classifier missed text phrased as a normal request A classifier lowers risk; it cannot be the boundary
    Redaction handled one Markdown link form but not another Check output against a format allow-list, not a list of known bad forms
    The client fetched images without a click Rendering is an egress path; restrict what the client can load
    An allow-listed Microsoft service fetched external URLs A proxying allow-listed domain undoes the allow-list; harden such proxies (paper)
    The email told Copilot not to mention it, so the user saw no sign of outside influence Show provenance in the interface

    Results

    Only what the sources state. The CVE record's own text is quoted under Problem statement.

    Item Source says
    Found and reported Aim Labs built a proof of concept in January 2025 and reported it privately to Microsoft (paper, Table 1)
    Fixed and disclosed Server-side fix in May 2025, "no customer action required"; advisory and public research on 2025-06-11 (paper, Table 1). Microsoft's advisory data says it "has already been fully mitigated by Microsoft"
    Exploitation The paper says Microsoft stated there was no evidence of in-the-wild exploitation. Aim: "not aware of any customers being impacted to date". The record's CISA entry (2025-06-14) lists exploitation as "none"
    Not stated Number of organisations exposed, how long data could be taken, what Microsoft's fix changed in detail

    Tradeoffs and when NOT to use this approach

    This is the page's reading, not the sources' claims.

    • Option B costs usefulness: some legitimate external mail will not inform answers.
    • The paper's defence matrix is analysis, not measurement (a note says "in our tests", yet its limitations say it ran no experiments), and it concerns one vendor's proprietary system. Aim's page also promotes its own guardrails.
    • On a hosted assistant most of these controls belong to the vendor; a customer can set policy and monitor.

    Design rules extracted

    1. Keep text from outside the organisation out of prompts that also hold internal data, unless the user asks for it.
    2. Treat model output as untrusted: allow-list link and image destinations and the Markdown subset before rendering.
    3. Make the render surface unable to load arbitrary origins, and audit allow-listed domains for proxy behaviour.
    4. Use a classifier as one layer, never as the boundary.
    5. Show the user when external content shaped an answer.

    Sources

    Last verified: 2026-10-22

    Ask about this article

    Get answers grounded in this post. AI-generated — based on this article, and may be imperfect.

    Scaled AI Weekly

    Enjoyed this? Get more like it every Monday.

    Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.

    Get new articles by email

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Join engineers building production AI systems

    Comments

    Back to all guides