PUBLIC-EVIDENCE TOPIC COLLECTION

RAG Security Case Studies and Evaluation References

RAG security cases covering retrieval authorization, vector databases, poisoned content, cross-user data, prompt-layer compromise, and cloud takeover paths.

5 matched references · all link to complete source and scoring records

Why this topic matters

Retrieval-augmented generation adds ingestion, storage, ranking, authorization, and prompt assembly to the model boundary. A meaningful RAG evaluation must test those components as one system and preserve tenant, provenance, and instruction-trust information end to end.

Evaluation questions

  • Can one user retrieve another tenant's documents, embeddings, prompts, or metadata?
  • Does retrieved content retain provenance and an untrusted-instruction label through prompt assembly?
  • Can ingestion, model loading, database access, or helper APIs reach code execution or cloud credentials?
  • Are retrieval relevance, authorization, poisoning resistance, and output controls tested together?

Framework routes: OWASP LLM Top 10 · NIST AI RMF. External frameworks guide threat, control, and evidence selection; measured deployment evidence determines AITBM scores.

RAG security case library

Use these records as incident references, test-design inputs, and examples of evidence-to-rubric reasoning. They are retrospective scenarios, not current vendor ratings or assessments of record.

2026-08-05 · RAG / Retrieval-Augmented System

Vector Databases Are Not Just Vectors: Orca Exposure Findings and Milvus Authentication Failures

The source combines an Internet-wide exposure study, several unrelated vector-database deployments, credential validation against external SaaS systems, and two different Milvus vulnerabilities with separate preconditions and fixed versions. There is no single assessed database, architecture, dependency graph, or remediation state. Combining them into one score would conflate population prevalence, two software defects, and several organizations' configurations.

Research note · no ERS

2026-05-26 · RAG / Retrieval-Augmented System

ChromaToast: ChromaDB Pre-Auth RCE Through Malicious Hugging Face Model Loading

Remediation Feasibility sits at its bottom anchor — a patchable ordering bug with a CVE and a fixed release — while Cascade Potential still forces 1.00, which is exactly the layer separation AITBM is built for: how easily a defect is fixed and how far it reaches are scored independently, so a cleanly patchable flaw never gets to look harmless.

Indicative ERS 4.8 · 2.7–7.0

2026-04-29 · RAG / Retrieval-Augmented System

Kimi Privacy Leak Report: When a Translation Request Returns Another User's Resume

An incident with no established root cause is still scorable — the enforced boundary and the missing release gate are directly observable from the outcome — and the unresolved mechanism belongs in ACI (thin coverage, a critical-invariant C_event cap) rather than being smoothed over with invented sub-metric scores; it is also the clearest example of a 'default' Cp, where 1.00 comes from the missing dependency graph and not from the impact.

Indicative ERS 3.6 · 1.5–5.6

2026-04-23 · RAG / Retrieval-Augmented System

Microsoft "EchoLeak": How One Email Turned Microsoft 365 Copilot into a Zero-Click Data Exfiltration Path

This is the batch's clearest demonstration that a control which held still gets scored, and scored up: CSP blocked the direct egress domain and the XPIA classifier forced the attacker to craft plain prose, which is exactly why Cn-3 and Cn-1 sit at 0.50 and Ro-1 at 0.40 rather than at failure anchors — and the residual risk is carried instead by Cp, where a single demonstrated chain crossing a declared-blocked boundary invalidates that gate's containment claim and corroborates the 1.00 anchor.

Indicative ERS 4.3 · 2.0–6.6

2026-04-17 · RAG / Retrieval-Augmented System

McKinsey Lilli Compromise: When SQL Injection Reaches the AI Prompt Layer

The sub-metric that should have stopped this is Ro-4, not a web-application control: once the prompt and retrieval tables were writable, the AI platform's integrity depended on chunk and configuration signing that did not exist - and AITBM scores that AI asset store, not the endpoint. The case also shows the attacker's autonomy must not leak into the victim's architecture class; Lilli is RAG, and classifying it as agentic because CodeWall's agent was would have silently changed the intra-axis weight set.

Indicative ERS 4.5 · 2.3–6.7

Use the evidence, not just the incident name

Each record distinguishes observed controls, missing evidence, architecture-based exclusions, operational risk, confidence limits, and the uncertainty interval. When using a case as a reference, compare the documented path to your own system boundary and rerun the applicable test methods rather than copying its indicative ERS.