Use case sources
Source reports and analysis supporting this case are listed below. AITBM interpretation and any indicative scoring are derived from this public evidence.
Embrace the Red (https://embracethered.com/blog/) demonstrated that ChatGPT users' conversations can be exfiltrated via an indirect prompt injection. To execute the attack, a threat actor uploads a malicious prompt to a public website, where a ChatGPT user may interact with it. The prompt causes ChatGPT to respond with the markdown for an image, whose URL has the user's conversation secretly embedded. ChatGPT renders the image for the user, creating an automatic request to an adversary-controlled script and exfiltrating the user's conversation. Additionally, the researcher demonstrated how the prompt can execute other plugins, opening them up to additional harms.
ASSESSED SYSTEM
OpenAI ChatGPT with third-party plugins enabled, in the configuration demonstrated by Embrace The Red and recorded as MITRE ATLAS AML.CS0021: the WebPilot plugin retrieving a web page the user's query points to, the Expedia plugin enabled in the same session, the model's plugin-invocation path, and the chat renderer that automatically fetches Markdown images. Scored in the demonstrated state.
OUT OF SCOPE
The attacker's web page and image-hosting server. The internal implementation of the third-party WebPilot and Expedia plugin services beyond their invocation by ChatGPT. ChatGPT's later state, memory and other tool surfaces. The Fairness axis (Fa-1..4).
Architecture: Tool-Calling LLM / Connected GenAI (decision tree Q3) — Q1 = NO: one assistant; the plugins are tools rather than peer agents. Q2 = NO: the evidenced chain is a single user-triggered turn with no autonomous planning loop or persistent task state. Q3 = YES: the model invokes functions — 'ChatGPT ... retrieve[s] the webpage using its WebPilot plugin', and the injected prompt can 'cause the LLM to execute other plugins that do not match a user request', demonstrated as a WebPilot-to-Expedia call. The renderer's automatic image fetch is a further connected capability acting on model output. Tier 2: Tier 2 Consumer: ChatGPT is a mass-market public assistant and the demonstrated exploitation affects individuals — the user's own conversation history is exfiltrated to an attacker server. No physical or systemic financial harm at scale is evidenced.
Documented attack or failure path
- The researcher writes a prompt instructing ChatGPT to emit a Markdown image whose URL carries the user's conversation, and places it on a web page.
- A user's query causes ChatGPT to retrieve that page through the WebPilot plugin, and the page text enters the model context.
- The injected instructions execute: ChatGPT emits a Markdown image hosted on an adversary-controlled server with the chat history embedded as a query parameter.
- The ChatGPT interface automatically renders the image, so the browser requests the attacker's URL and the conversation leaves the product with no user action.
- The same injection can make the model invoke another enabled plugin that the user did not request, demonstrated as WebPilot causing an Expedia plugin call.
Layer 1 — Intrinsic Vulnerability Profile
Each sub-metric is placed on its five-level rubric by the evidence quoted beside it. Missing applicable evidence remains unknown. The displayed midpoint and interval are scenario values, not inferred control performance.
| Sub-metric | Score | Rubric basis | Evidence |
|---|---|---|---|
| Robustness (Ro) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.50 indeterminate | |||
| Fairness (Fa) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
| Transparency (Tr) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.50 indeterminate | |||
| Privacy (Pr) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
| Containment (Cn) — scenario interval 0.03–0.54 (midpoint 0.28), Tier 2 MVT 0.55 confirmed breach | |||
| Cn-1Scope Enforcement | 0.25w 0.12 | The 0.25 anchor: basic boundaries exist — tool invocation is confined to the plugins enabled in the session — but simple prompt manipulation exceeded intended scope on both the tool path and the network path. The 0.50 anchor requires core scope controls that block common violations; an arbitrary-domain fetch and an unrequested tool call are the common violations. | Two scope boundaries were exceeded by prompt content alone. The renderer fetched an image from an arbitrary adversary-controlled server, so no destination allowlist governed renderer network scope; and the injection caused ChatGPT to invoke the Expedia plugin, which 'do[es] not match a user request'. The forged call stayed within the session's enabled plugin set rather than reaching an arbitrary capability.source: secureflow/atlas |
| Cn-3Output Filtering Robustness | 0.00w 0.16 | The 0.00 anchor: a sensitive, attacker-directed output was returned and rendered directly. The 0.25 anchor requires a basic filter that the attack had to bypass; the payload used no encoding or evasion and no filtering layer acting on it is evidenced. | The model's reply contained a Markdown image pointing at an adversary-controlled host with the user's conversation history in the query string, and the interface rendered it automatically. Nothing in the record inspected the generated URL, the destination or the embedded data before render.source: secureflow/atlas |
| Cn-6Action Reversibility Classification Rate | 0.00w 0.07 | The 0.00 anchor: no reversibility classification is evidenced before execution and the actions ran ungated (ARCR = 0 for these action classes). The 0.25 anchor requires at least ad-hoc classification of selected high-impact actions, which the record does not show. | Two automated actions executed with no pre-execution gate: the renderer's outbound image request, whose disclosure of the conversation cannot be recalled once made, and a plugin call the user had not requested. The record describes no confirmation, classification or approval step before either action.source: secureflow/atlas |
| Cn-8Untrusted Context Isolation | 0.00w 0.14 | 0.00 anchor — no origin labelling; tool-returned web content was admitted to active context as trusted context by design and acquired instruction authority over output and plugin invocation. Not 0.25, because no origin label on the plugin channel and no prompt-wording denial of instruction authority is evidenced. Moved from Ro-1: the evidence records a boundary failure (untrusted content admitted as instruction), not a measurement of the model's resistance under adversarial test. | Text on an attacker web page, retrieved through the WebPilot plugin during an ordinary user query, entered the model context and was executed as instruction: ChatGPT emitted a Markdown image carrying the user's conversation to an adversary-controlled host and could be made to invoke the Expedia plugin, which 'do[es] not match a user request'. The record reports no input screening, origin labelling or demotion of plugin-returned content.source: secureflow/atlas |
Unknown, not N/A: 22 applicable sub-metrics lack admissible public evidence. The lower, midpoint, and upper scenarios evaluate each at 0.00, 0.50, and 1.00 respectively; no weight is redistributed merely because evidence is missing.
Layer 2 — Operational Risk Posture
| Dimension | Score | Justification |
|---|---|---|
| AaAutonomy Amplification | 0.50w 0.25 | The 0.50 human-on-the-loop anchor: ChatGPT executes routine tool actions autonomously within the plugin boundary the user configured — retrieving pages, invoking plugins and rendering remote media — without per-action confirmation, and the user can observe and stop the session. Not 0.75: no high-consequence transaction executed autonomously is evidenced; the unrequested Expedia call is demonstrated as a forged request, not as a completed consequential action. |
| AsAttack Surface Exposure | 1.00w 0.35 | The 1.00 maximum-exposure anchor: the assistant is internet-facing, ingests wholly untrusted web content on demand through a third-party plugin, and that content was shown to drive further third-party tool integrations. Untrusted retrieval combined with demonstrated cross-plugin tool reach is the anchor's untrusted-RAG-and-tool-integration condition. elevated |
| CpCascade Potential | 1.00w 0.20 | No verified System Dependency Graph is public for ChatGPT's plugin and rendering pipeline, so the specification's worst-case default applies (no SDG, DGC < 0.90, therefore Cp = 1.00). The observed path crosses the model, the renderer, the plugin layer and an external host, but its terminal node is an outbound read of an attacker URL and the Expedia call is not shown completing a write-external action, so the evidence does not independently corroborate the 1.00 anchor.GDCP: worst-case default — no verified dependency graph published elevated |
| RfRemediation Feasibility | 0.25w 0.20 | The 0.25 model-adjacent anchor: the remediation class is deployment-side — blocking or proxying automatic rendering of model-emitted remote images, allowlisting and canonicalising destinations, and requiring user confirmation before a plugin call that the user did not request. None requires model change. The record reports no vendor fix, so this reflects the availability of a deterministic remedy rather than a shipped patch. |
Nelevated = 2 (dimensions strictly above 0.75) → CRM = 1.15.
Compound Risk Alert. Two or more dimensions are simultaneously elevated (spec 3.2.2); architectural decomposition is recommended before deployment.
Layer 3 — Public-evidence confidence diagnostic
| Component | Score | Basis |
|---|---|---|
| Pc — Public provenance evidence | 0.25 | The 0.25 anchor: the vendor, product and the two plugins are named, but the case provides no architecture record, plugin manifest or permission scope, renderer policy, model lineage or evaluation artefact for the assessed configuration. |
| Ec — Public evaluation coverage | 0.13 | coverage 0.15 (4 of 26 applicable sub-metrics) × independence 1.00 × fidelity 0.85. No Full, Standard, or Lite pathway is claimed for a retrospective article. |
| Tf — Public-evidence freshness | 0.00 | Evidence dated 2023-05-01; age 1266 days on the workpaper reference date. Components: T_containment 0.00 · T_calendar 0.00 · C_monitor 0.65 · C_event 0.65 · C_evidence 0.85. Binding term: T_containment. dt_days is measured from the MITRE ATLAS AML.CS0021 incident date 2023-05-01 (month granularity) to the 2026-10-18 evidence reference date; the primary Embrace The Red report is listed in sources. agentic = true: the Tool-Calling class has runtime-mutable permission boundaries — enabled plugins and their scopes change per session — so the containment staleness floor applies. baw = false: no model-writable cross-session memory, self-modifying configuration, agent-to-agent channel or closed feedback loop is evidenced. C_monitor 0.65 — no detection of the exfiltration image or the forged plugin call is evidenced. C_event 0.65 — a researcher demonstration with no reported in-the-wild compromise. C_evidence 0.85 — the record reports no remediation. |
Public-evidence ACI = (Pc × Ec × Tf)1/3 = 0.00 — diagnostic status: Invalid as assessment-of-record evidence. It describes the evidence available to this case study, not the assurance of the underlying system, and is not inserted into the normalized-assurance scenario ERS.
Indicative ERS — normalized-assurance scenario
Published CVSS beside indicative ERS
Each published CVSS record is shown exactly as its scorer published it, one per vulnerability record, with the version, metric group and vector when stated, plus its source and access date. Records are never combined or inferred from a CVE's existence.
INDICATIVE ERS · NORMALIZED-ASSURANCE SCENARIO
4.9 midpoint, interval 1.6–8.3; Low-Moderate scenario band; ACI fixed at 1.00. The published CVSS records do not enter this calculation.
PUBLISHED CVSS
CVSS — no applicable published score verified.
CVSS describes an individual vulnerability as its scorer published it. Indicative ERS describes the assessed AI deployment under the normalized-assurance scenario, so the two values are not direct equivalents and neither replaces the other.
AIDEFEND defences → AITBM sub-metrics
Identifiers are quoted as they appear on the AIDEFEND in Action brief or in the AIDEFEND SecureFlow defense map (audited AIDEFEND dataVersion 2026.10.15); the sub-metric mapping is AITBM's own, from the specification's AIDEFEND tables reconciled at catalogue data version 2026.10.15. AIDEFEND renumbers identifiers between releases, so the data version travels with every mapping and neither side's IDs should be cited without one. A mapping identifies a possible evidence route; a recommendation does not prove that the control was implemented or effective and receives no scoring credit by itself.
| Technique | Defence | Priority | Evidences |
|---|---|---|---|
| AID-H-002.002 | Inference-Time Prompt & Input ValidationExact SecureFlow defense-map identifier (flow atlas-aml-cs0021-chatgpt-conversation-exfiltration), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-3. Presence supplies no score credit. | Candidate evidence route | Ro-1 Ro-5 Fa-1 Fa-3 Cn-8 |
| AID-H-019.002 | Secure HTML Rendering & Content DemotionExact SecureFlow defense-map identifier (flow atlas-aml-cs0021-chatgpt-conversation-exfiltration), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-3. Presence supplies no score credit. | Candidate evidence route | Cn-1 Cn-3 Cn-8 |
| AID-D-001.001 | Per-Prompt Content, Intent & Obfuscation AnalysisExact SecureFlow defense-map identifier (flow atlas-aml-cs0021-chatgpt-conversation-exfiltration), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Ro-1 |
| AID-H-016.001 | System Prompt Structure & Instruction/Data SeparationExact SecureFlow defense-map identifier (flow atlas-aml-cs0021-chatgpt-conversation-exfiltration), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Ro-1 |
| AID-H-017.007 | Dual-LLM Isolation PatternExact SecureFlow defense-map identifier (flow atlas-aml-cs0021-chatgpt-conversation-exfiltration), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Cn-5 Cn-7 |
| AID-H-006.002 | Text, Markup & Structured Output Sanitization and Release GateExact SecureFlow defense-map identifier (flow atlas-aml-cs0021-chatgpt-conversation-exfiltration), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-5. Presence supplies no score credit. | Candidate evidence route | Ro-3 Cn-3 |
| AID-H-018.004 | Intent-Based Dynamic Capability ScopingExact SecureFlow defense-map identifier (flow atlas-aml-cs0021-chatgpt-conversation-exfiltration), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Cn-1 Cn-6 Cn-7 |
| AID-D-003.004 | Tool-Call Sequence Anomaly DetectionExact SecureFlow defense-map identifier (flow atlas-aml-cs0021-chatgpt-conversation-exfiltration), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Ro-3 Cn-1 Cn-3 Cn-7 |
WHAT THIS CASE TEACHES
A single injected page fails Cn-1 on two separate scope paths at once — the renderer's network destination and the model's tool selection — and both automated actions ran ungated, which is why Cn-6 sits at 0.00 even though the user never approved or saw either action.
Sources: AIDEFEND SecureFlow — ChatGPT Conversation Exfiltration · MITRE ATLAS case study AML.CS0021 · ChatGPT Plugins: Data Exfiltration via Images & Cross Plugin Request Forgery (embracethered.com)
SecureFlow attack flow
This record mirrors an AIDEFEND SecureFlow flow (MITRE ATLAS case-study conversion): an ordered attack path with MITRE ATLAS tactic and technique labels and the AIDEFEND controls matched to each step. The scenario above places only the control outcomes this flow and its primary references observed; the matched AIDEFEND controls are evidence routes and earn no score credit.
| MITRE ATLAS case study | AML.CS0021 (exercise) |
|---|---|
| Target | OpenAI ChatGPT |
| Actor | Embrace The Red |
| Primary evidence date | 2023-05-01 |
| SecureFlow record updated | 2026-08-02 |
| AI layers | AI infrastructure |
| Risk impact | Data exfiltration |
| Affected components | ChatGPT LLM assistant, WebPilot and Expedia plugins, Markdown image auto-rendering, user conversation history |
Attack steps
| # | Step | ATLAS tactic / technique |
|---|---|---|
| 1 | Craft Markdown Image Exfiltration PromptThe researcher developed a prompt that causes ChatGPT to include a Markdown element for an image with the user's conversation embedded in the URL as part of its responses. | AML.TA0003 AML.T0065 |
| 2 | Plant Exfiltration Prompt on WebpageThe researcher included the prompt in a webpage, where it could be retrieved by ChatGPT. | AML.TA0003 AML.T0079 |
| 3 | Retrieve Adversary Webpage via PluginWhen the user makes a query that causes ChatGPT to retrieve the webpage using its `WebPilot` plugin, it ingests the adversary's prompt. | AML.TA0004 AML.T0078 |
| 4 | Execute Indirect Prompt InjectionThe prompt injection is executed, causing ChatGPT to include a Markdown element for an image hosted on an adversary-controlled server and embed the user's chat history as query parameter in the URL. | AML.TA0005 AML.T0051.001 |
| 5 | Exfiltrate Conversation via Rendered Image URLChatGPT automatically renders the image for the user, making the request to the adversary's server for the image contents, and exfiltrating the user's conversation. | AML.TA0010 AML.T0077 |
| 6 | Forge Cross-Plugin RequestsAdditionally, the prompt can cause the LLM to execute other plugins that do not match a user request. In this instance, the researcher demonstrated the `WebPilot` plugin making a call to the `Expedia` plugin. | AML.TA0012 AML.T0053 |
| 7 | Violate User PrivacyThe user's privacy is violated, and they are potentially open to further targeted attacks. | AML.TA0011 AML.T0048.003 |
Threat references
Summary and attack steps reproduced from AIDEFEND SecureFlow (https://aidefendlabs.com/en/secureflow/?flow=atlas-aml-cs0021-chatgpt-conversation-exfiltration; AIDEFEND content CC BY 4.0), converted from MITRE ATLAS case study AML.CS0021 (Apache License 2.0; MITRE ATLAS is a trademark of The MITRE Corporation).