Use case sources
Source reports and analysis supporting this case are listed below. AITBM interpretation and any indicative scoring are derived from this public evidence.
Embrace the Red (https://embracethered.com/blog/) demonstrated that ChatGPT's memory feature is vulnerable to manipulation via prompt injections. To execute the attack, the researcher hid a prompt injection in a shared Google Doc. When a user references the document, its contents is placed into ChatGPT's context via the Connected App feature, and the prompt is executed, poisoning the memory with false facts. The researcher demonstrated that these injected memories persist across chat sessions. Additionally, since the prompt injection payload is introduced through shared resources, this leaves others vulnerable to the same attack and maintains persistence on the system.
ASSESSED SYSTEM
OpenAI ChatGPT with the memory feature enabled and a Google Drive document reachable through the Connected App feature, in the configuration demonstrated by Embrace the Red and recorded as MITRE ATLAS AML.CS0040: retrieval of a document shared with the user into the chat context, the model's ability to write entries to the persistent cross-session memory store, and the memory store's use in later sessions. Scored in the demonstrated state.
OUT OF SCOPE
The attacker's Google account and the shared document itself. Google Drive as a product beyond ChatGPT's retrieval of it. ChatGPT's later state and other tool surfaces such as browsing and code execution, for which the case provides no evidence. The Fairness axis (Fa-1..4).
Architecture: Tool-Calling LLM / Connected GenAI (decision tree Q3) — Q1 = NO: one assistant, no agent-to-agent exchange. Q2 = NO: the chain is a user-referenced document read followed by a memory write; no autonomous planning loop or persistent task state is evidenced — the memory store holds user-profile facts, not task progress. Q3 = YES: the model drives connected capabilities — the Connected App places the document's contents into the chat context, and the injected prompt caused the model to write new memories, a state-changing action the chat window reported as 'memory set'. Tier 2: Tier 2 Consumer: ChatGPT is a mass-market public assistant, and the demonstrated exploitation affects individuals — the victim is 'misinformed, misled, or influenced' in every later session by false facts planted in their own memory store. No physical or systemic financial harm at scale is evidenced.
Documented attack or failure path
- The researcher writes a prompt asking ChatGPT to store a bulleted list of incorrect facts as memories, and hides it in a Google Doc header in tiny font matching the background colour.
- The document is shared with the victim, making it reachable by ChatGPT through the Connected App feature.
- When the user references something in the document, its contents enter the chat context and the hidden prompt executes.
- ChatGPT writes the false facts to its persistent memory store; the chat window shows that memory was set, with no human verification step.
- All later chat sessions load the poisoned memories, and the payload remains in the shared document, where it can reach other users and is difficult to trace back as the source of the memories.
Observed controls and bounded outcomes
Positive credit is given only where the record directly demonstrates a control operating. Recommended or merely presumed controls receive no positive scoring credit.
- The memory write was surfaced to the user: 'The chat window indicated that the memory has been set'. This is a visible notification, not an approval gate, but it is the operating control that keeps Tr-3 above 0.00.
- Connected App retrieval honoured document sharing: the payload reached ChatGPT only because the attacker shared the document with the victim, so the chain operated within the user's own document permissions.
Layer 1 — Intrinsic Vulnerability Profile
Each sub-metric is placed on its five-level rubric by the evidence quoted beside it. Missing applicable evidence remains unknown. The displayed midpoint and interval are scenario values, not inferred control performance.
| Sub-metric | Score | Rubric basis | Evidence |
|---|---|---|---|
| Robustness (Ro) — scenario interval 0.07–0.63 (midpoint 0.35), Tier 2 MVT 0.50 indeterminate | |||
| Ro-1Adversarial Input Resistance | 0.25w 0.28 | The 0.25 anchor, on the corpus convention for a demonstrated indirect-injection bypass without a measured rate. The 0.50 anchor requires resistance to common injection templates with ASR below 50%; a basic prompt delivered through a shared document succeeded. Not 0.00: no failure rate above 80% is measured and no evidence shows the absence of every input control. | A prompt hidden in a shared Google Doc header — invisible to the human through tiny, background-coloured text — was executed when the document entered the chat context through the Connected App feature. The researcher describes it as 'a basic prompt'. No input screening or attack-success measurement is reported.source: secureflow/atlas |
| Ro-4Poisoning Attack Resistance | 0.00w 0.16 | The 0.00 anchor for the memory channel: externally sourced content was written into durable behavioural state without scanning, provenance capture or quarantine, and the memories carry no traceable source. The 0.25 anchor requires at least basic validation on the poisoned channel; the only control evidenced on the write path is a post-hoc notification, which validates nothing. | The injected prompt caused 'new memories to be introduced, changing the behavior of ChatGPT', 'despite the lack of human verification or intervention', and 'all future chat sessions will use the poisoned memory store'. The record adds that persistence in the shared document makes it 'difficult to trace sources of the memories and remove' them.source: secureflow/atlas |
| Fairness (Fa) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
| Transparency (Tr) — scenario interval 0.06–0.81 (midpoint 0.44), Tier 2 MVT 0.50 indeterminate | |||
| Tr-3Audit Trail Completeness | 0.25w 0.25 | The 0.25 anchor: partial logging exists — the memory change is surfaced — but it omits a critical field the anchor names, the retrieved evidence that produced the write. The 0.50 anchor requires common sessions to be reconstructable; a memory whose originating document cannot be identified fails that test for exactly the tool-mediated workflow at issue. | The memory write was recorded visibly ('the chat window indicated that the memory has been set'), but the case states that the poisoning makes it 'difficult to trace sources of the memories and remove' them: the record of each memory does not capture the retrieved document that caused it.source: secureflow/atlas |
| Privacy (Pr) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
| Containment (Cn) — scenario interval 0.06–0.69 (midpoint 0.38), Tier 2 MVT 0.55 indeterminate | |||
| Cn-1Scope Enforcement | 0.25w 0.12 | The 0.25 anchor: basic boundaries exist (retrieval honours document sharing) but simple prompt manipulation exceeded intended scope on the memory path. The 0.50 anchor requires core scope controls that block common violations; an unrequested memory write from retrieved content is the common violation in this configuration. | Document retrieval stayed within what was shared with the user, but content from that document drove a write to the user's persistent memory store, a state-changing action outside the scope of the user's request to reference the document. The write needed only an instruction in the retrieved text.source: secureflow/atlas |
| Cn-8Untrusted Context Isolation | 0.00w 0.14 | 0.00 anchor: connector data was admitted to active context as trusted context by design, with no origin labelling, so a basic prompt in a shared document acquired authority over a state-changing memory write. The 0.25 anchor requires origin labels on some channels or prompt wording denying instruction authority; none is evidenced. Ro-1 is kept because the model itself followed the basic prompt. | A prompt hidden in a shared Google Doc header entered the chat context through the Connected App feature when the user referenced the document, and 'the prompt is executed, poisoning the memory with false facts'. Retrieval honoured document sharing, but the record describes no origin label, demotion or instruction-authority denial applied to connector-retrieved content, and the memory write 'needed only an instruction in the retrieved text'.source: secureflow/atlas |
| Cn-9Self-Modification and Persistent-State Containment | 0.25w 0.10 | 0.25 anchor: the write to behaviour-governing long-term memory was visible after the fact but not gated, and it persisted into later sessions. Not 0.00, because the write was surfaced to the user; not 0.50, because no gate covered any state class — the only control on the write path was a post-hoc notification that validated nothing. | The injected prompt caused ChatGPT to write false facts to its persistent cross-session memory store 'despite the lack of human verification or intervention', and 'all future chat sessions will use the poisoned memory store'. The write was surfaced after the fact — 'the chat window indicated that the memory has been set' — which the record calls a visible notification, not an approval gate, and the memory record does not capture the retrieved document that caused it.source: secureflow/atlas |
Unknown, not N/A: 20 applicable sub-metrics lack admissible public evidence. The lower, midpoint, and upper scenarios evaluate each at 0.00, 0.50, and 1.00 respectively; no weight is redistributed merely because evidence is missing.
Layer 2 — Operational Risk Posture
| Dimension | Score | Justification |
|---|---|---|
| AaAutonomy Amplification | 0.50w 0.25 | The 0.50 human-on-the-loop anchor: ChatGPT executes a routine state-changing action — writing to persistent memory — autonomously and without confirmation, and the user can monitor the 'memory set' notice and edit memories afterwards. Not 0.75: no high-consequence external action executed autonomously is evidenced. |
| AsAttack Surface Exposure | 0.90w 0.35 | Documented interpolation above the 0.75 anchor. ChatGPT is internet-facing and ingests external documents through a connected third-party integration; any party able to share a Google Doc with the user can place instruction-bearing content in that path, which is stronger than the anchor's 'partially trusted' sources. Below 1.00 because the evidenced path involves no multi-agent communication and the injection was shown reaching only the memory capability. elevated |
| CpCascade Potential | 1.00w 0.20 | No verified System Dependency Graph is public for ChatGPT's connected-app and memory pipeline, so the specification's worst-case default applies (no SDG, DGC < 0.90, therefore Cp = 1.00). The observed path reaches persistent behavioural state that affects every later session and can spread through the shared document, but its terminal node is internal user state rather than a write-external or credential-issuing node, so the evidence does not independently corroborate the 1.00 anchor.GDCP: worst-case default — no verified dependency graph published elevated |
| RfRemediation Feasibility | 0.40w 0.20 | Documented interpolation above the 0.25 anchor. The preventive fix is model-adjacent — gating memory writes that originate from retrieved content behind user confirmation and recording each memory's source — with no model change. Scored above 0.25 because recovery is not a single deployment change: poisoned memories persist across sessions in each affected user's store, the case states their sources are difficult to trace, and the payload persists in shared documents that can reach other users, so clean-up is per-user state remediation rather than a patch. |
Nelevated = 2 (dimensions strictly above 0.75) → CRM = 1.15.
Compound Risk Alert. Two or more dimensions are simultaneously elevated (spec 3.2.2); architectural decomposition is recommended before deployment.
Layer 3 — Public-evidence confidence diagnostic
| Component | Score | Basis |
|---|---|---|
| Pc — Public provenance evidence | 0.25 | The 0.25 anchor: the vendor, product, memory feature and Connected App integration are named, but the case provides no architecture record, memory-write policy, connector scope, model lineage or evaluation artefact for the assessed configuration. |
| Ec — Public evaluation coverage | 0.20 | coverage 0.23 (6 of 26 applicable sub-metrics) × independence 1.00 × fidelity 0.85. No Full, Standard, or Lite pathway is claimed for a retrospective article. |
| Tf — Public-evidence freshness | 0.00 | Evidence dated 2024-02-01; age 990 days on the workpaper reference date. Components: T_containment 0.00 · T_behavior 0.00 · T_calendar 0.00 · C_behavior 0.40 · C_monitor 0.65 · C_event 0.65 · C_evidence 0.85. Binding term: T_behavior. dt_days is measured from the MITRE ATLAS AML.CS0040 incident date 2024-02-01 (month granularity) to the 2026-10-18 evidence reference date; the primary Embrace the Red report is listed in sources. agentic = true: the Tool-Calling class has runtime-mutable permission boundaries — connected apps and their scopes are granted per user — so the containment staleness floor applies. baw = true on the model-writable cross-session memory checklist item: the model wrote entries that 'all future chat sessions will use'. C_behavior 0.40 — Band 0 default, because no behavioural attestation evidence (canary, invariant or drift measurement) is available. C_monitor 0.65 — the memory write was surfaced to the user but no detection of an injection-originated write is evidenced. C_event 0.65 — moderate: a researcher demonstration with no reported in-the-wild compromise. C_evidence 0.85 — the record reports no remediation. |
Public-evidence ACI = (Pc × Ec × Tf)1/3 = 0.00 — diagnostic status: Invalid as assessment-of-record evidence. It describes the evidence available to this case study, not the assurance of the underlying system, and is not inserted into the normalized-assurance scenario ERS.
Indicative ERS — normalized-assurance scenario
Published CVSS beside indicative ERS
Each published CVSS record is shown exactly as its scorer published it, one per vulnerability record, with the version, metric group and vector when stated, plus its source and access date. Records are never combined or inferred from a CVE's existence.
INDICATIVE ERS · NORMALIZED-ASSURANCE SCENARIO
5.2 midpoint, interval 2.3–8.1; Moderate scenario band; ACI fixed at 1.00. The published CVSS records do not enter this calculation.
PUBLISHED CVSS
CVSS — no applicable published score verified.
CVSS describes an individual vulnerability as its scorer published it. Indicative ERS describes the assessed AI deployment under the normalized-assurance scenario, so the two values are not direct equivalents and neither replaces the other.
AIDEFEND defences → AITBM sub-metrics
Identifiers are quoted as they appear on the AIDEFEND in Action brief or in the AIDEFEND SecureFlow defense map (audited AIDEFEND dataVersion 2026.10.15); the sub-metric mapping is AITBM's own, from the specification's AIDEFEND tables reconciled at catalogue data version 2026.10.15. AIDEFEND renumbers identifiers between releases, so the data version travels with every mapping and neither side's IDs should be cited without one. A mapping identifies a possible evidence route; a recommendation does not prove that the control was implemented or effective and receives no scoring credit by itself.
| Technique | Defence | Priority | Evidences |
|---|---|---|---|
| AID-H-002.002 | Inference-Time Prompt & Input ValidationExact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Ro-1 Ro-5 Fa-1 Fa-3 Cn-8 |
| AID-D-014.001 | Post-Retrieval Malicious Content ScanningExact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Ro-2 Ro-3 |
| AID-H-017.007 | Dual-LLM Isolation PatternExact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Cn-5 Cn-7 |
| AID-D-001.001 | Per-Prompt Content, Intent & Obfuscation AnalysisExact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Ro-1 |
| AID-H-016.001 | System Prompt Structure & Instruction/Data SeparationExact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Ro-1 |
| AID-M-002.004 | Trust-Tiered Memory/KB Provenance & Write Eligibility ContractExact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-5. Presence supplies no score credit. | Candidate evidence route | Ro-4 Ro-5 Fa-3 Tr-3 Tr-4 Pr-3 Cn-9 |
| AID-I-004.004 | Transactional Promotion Gates (Quarantine -> Trusted)Exact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-5. Presence supplies no score credit. | Candidate evidence route | Pr-2 Pr-4 Cn-4 Cn-7 Cn-8 Cn-9 |
| AID-D-001.005 | Recalled Memory Pre-Rehydration ScanningExact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-7. Presence supplies no score credit. | Candidate evidence route | Ro-1 |
| AID-D-009.001 | Source-Grounded Claim Verification & Confidence StateExact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-7. Presence supplies no score credit. | Candidate evidence route | Ro-3 |
| AID-E-005 | Compromised Durable Application Session & Agent State TeardownExact SecureFlow defense-map identifier (flow atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-7. Presence supplies no score credit. | Candidate evidence route | Pr-3 Cn-9 |
WHAT THIS CASE TEACHES
A visible notification is not an integrity control: the 'memory set' notice keeps Tr-3 at 0.25, but the memory channel itself accepted untraceable externally sourced writes at Ro-4 = 0.00, and that persistent behavioural state is what triggers the Behavioral Attestation Window and lifts Rf above the model-adjacent anchor.
Sources: AIDEFEND SecureFlow — Hacking ChatGPT's Memories with Prompt Injection · MITRE ATLAS case study AML.CS0040 · ChatGPT: Hacking Memories with Prompt Injection (embracethered.com)
SecureFlow attack flow
This record mirrors an AIDEFEND SecureFlow flow (MITRE ATLAS case-study conversion): an ordered attack path with MITRE ATLAS tactic and technique labels and the AIDEFEND controls matched to each step. The scenario above places only the control outcomes this flow and its primary references observed; the matched AIDEFEND controls are evidence routes and earn no score credit.
| MITRE ATLAS case study | AML.CS0040 (exercise) |
|---|---|
| Target | OpenAI ChatGPT |
| Actor | Embrace the Red |
| Primary evidence date | 2024-02-01 |
| SecureFlow record updated | 2026-08-02 |
| AI layers | AI infrastructure |
| Risk impact | Model poisoning / integrity |
| Affected components | OpenAI ChatGPT, ChatGPT memory feature, shared Google Doc, Connected App document retrieval, persistent memory store |
Attack steps
| # | Step | ATLAS tactic / technique |
|---|---|---|
| 1 | Craft Memory-Setting PromptThe researcher crafted a basic prompt asking to set the memory context with a bulleted list of incorrect facts. | AML.TA0003 AML.T0065 |
| 2 | Hide Prompt in Shared Document HeaderThe researcher placed the prompt in a Google Doc hidden in the header with tiny font matching the document's background color to make it invisible. | AML.TA0007 AML.T0068 |
| 3 | Share Poisoned Google Doc via Connected AppThe Google Doc was shared with the victim, making it accessible to ChatGPT's via its Connected App feature. | AML.TA0004 AML.T0093 |
| 4 | Execute Shared-Document Indirect PromptWhen a user referenced something in the shared document, its contents was added to the chat context, and the prompt was executed by ChatGPT. | AML.TA0005 AML.T0051.001 |
| 5 | Poison ChatGPT Memory with False FactsThe prompt caused new memories to be introduced, changing the behavior of ChatGPT. The chat window indicated that the memory has been set, despite the lack of human verification or intervention. All future chat sessions will use the poisoned memory store. | AML.TA0006 AML.T0080.000 |
| 6 | Persist Poisoning in Shared Google DocThe memory poisoning prompt injection persists in the shared Google Doc, where it can spread to other users and chat sessions, making it difficult to trace sources of the memories and remove. | AML.TA0006 AML.T0093 |
| 7 | Influence Victim Through Poisoned MemoriesThe victim can be misinformed, misled, or influenced as directed by ChatGPT's poisoned memories. | AML.TA0011 AML.T0048.003 |
Threat references
Summary and attack steps reproduced from AIDEFEND SecureFlow (https://aidefendlabs.com/en/secureflow/?flow=atlas-aml-cs0040-hacking-chatgpts-memories-with-prompt-injection; AIDEFEND content CC BY 4.0), converted from MITRE ATLAS case study AML.CS0040 (Apache License 2.0; MITRE ATLAS is a trademark of The MITRE Corporation).