PUBLIC-EVIDENCE TOPIC COLLECTION
AI model security cases covering malicious model loading, inference backdoors, tokenizer blind spots, exposed inference, namespace reuse, and distillation.
23 matched references · all link to complete source and scoring records
Model security extends beyond adversarial examples. Serialization formats, chat templates, tokenizers, serving messages, namespaces, exposed compute, and distillation pipelines can all change behavior or open a code-execution boundary.
Framework routes: MITRE ATLAS · OWASP AISVS. External frameworks guide threat, control, and evidence selection; measured deployment evidence determines AITBM scores.
Use these records as incident references, test-design inputs, and examples of evidence-to-rubric reasoning. They are retrospective scenarios, not current vendor ratings or assessments of record.
2026-09-08 · Agentic / MCP System
Anthropic reported a September 2025 campaign that used Claude Code, its AI coding agent, against roughly 30 organizations and compromised a small number. Anthropic attributes GTG-1002 to a Chinese state-sponsored actor with high confidence. The attackers posed as legitimate security testers and fragmented malicious objectives into plausible technical tasks, depriving the model of the context needed to judge their combined purpose. Their orchestration connected Claude to external tools through MCP, and the technical report describes an SSRF path from scanning and payload generation to validation, human approval, and exploitation, followed by internal discovery and credential abuse. Anthropic estimated AI performed 80 to 90 percent of tactical work while humans retained critical decisions, and notes that Claude fabricated credentials and misrepresented public information as secret discoveries, so generated reports are not proof of compromise. Anthropic investigated Claude usage, banned accounts, and notified affected entities; its account is not a complete independent reconstruction of every local command or victim outcome. MITRE ATLAS records the campaign as case study AML.CS0069.
Research note · no ERS
2026-08-25 · Tool-Calling LLM / Connected GenAI
SentinelLABS analyzed macOS.Gaslight, a Rust backdoor containing an approximately 3,500-byte prompt-injection payload with 38 fabricated system messages. The messages imitate an LLM-assisted malware-triage harness and report fake token expiry, memory and disk failures, and bogus analysis warnings, aiming to make the workflow abort, truncate, or refuse analysis. Recovered strings include a {{DATA}} boundary imitation paired with fabricated worker-OOM and static-analysis warnings. SentinelLABS attributed the activity with high confidence to a DPRK-aligned cluster and documented conventional backdoor capabilities — Telegram command and control, a shell, collection, upload, and LaunchAgent persistence — but the published research did not test an LLM or demonstrate a successful AI-analysis bypass.
Research note · no ERS
2026-08-06 · Incident
The Microsoft Incident Response - Detection and Response Team (DART) investigated a compromised system where a threat actor utilized SesameOp, a backdoor implant that abuses the OpenAI Assistants API as a covert command and control channel, for espionage activities. The SesameOp malware used the OpenAI API to fetch and execute the threat actor's commands and to exfiltrate encrypted results from the victim system. The threat actor had maintained a presence on the compromised system for several months. They had control of multiple internal web shells which executed commands from malicious processes that relied on compromised Visual Studio utilities. Investigation of other Visual Studio utilities led to the discovery of the novel SesameOp backdoor.
Research note · no ERS
2026-08-06 · Agentic / MCP System
Rules files are mutable behavioural state, so AITBM scores the backdoor through Ro-5 and Cn-5 (both 0.00) and applies the behavioural staleness window; the invisible Unicode defeats human review rather than a model filter, which is why it does not lift Cn-8 above the class precedent's 0.00.
Indicative ERS 6.2 · 2.9–9.5
2026-08-06 · Traditional ML / Classifier
A scanner that must fully deserialize an artifact to judge it sits on the wrong side of the code-execution boundary: the evasion holds Ro-5 at 0.25 only because a hub-side scan existed, while the loader compromise is carried by Containment rather than by any model property.
Indicative ERS 3.7 · 1.4–5.9
2026-08-06 · Traditional ML / Classifier
Shipping a model inside a client package moves its integrity boundary onto the device, and an app that loads that model without verification scores Ro-5 = 0.00 however accurate the clean model is; a single observed sub-metric also shows how thin evidence widens the scenario interval.
Indicative ERS 3.8 · 1.2–6.4
2026-08-05 · Standalone LLM / Generative AI
An internal AI message bus is still an untrusted code boundary: peer identity, segmentation, and safe serialization must hold before bytes reach a deserializer.
Indicative ERS 3.5 · 1.5–5.5
2026-08-02 · Validated Research
Many recent large-scale datasets are distributed as a list of URLs pointing to individual datapoints. The researchers show that many of these datasets are vulnerable to a "split-view" poisoning attack. The attack exploits the fact that the data viewed when it was initially collected may differ from the data viewed by a user during training. The researchers identify expired and buyable domains that once hosted dataset content, making it possible to replace portions of the dataset with poisoned data. They demonstrate that for 10 popular web-scale datasets, enough of the domains are purchasable to successfully carry out a poisoning attack.
Research note · no ERS
2026-08-02 · Incident
McAfee Advanced Threat Research noticed an increase in reports of a certain ransomware family that was out of the ordinary. Case investigation revealed that many samples of that particular ransomware family were submitted through a popular virus-sharing platform within a short amount of time. Further investigation revealed that based on string similarity the samples were all equivalent, and based on code similarity they were between 98 and 74 percent similar. Interestingly enough, the compile time was the same for all the samples. After more digging, researchers discovered that someone used 'metame' a metamorphic code manipulating tool to manipulate the original file towards mutant variants. The variants would not always be executable, but are still classified as the same ransomware family.
Research note · no ERS
2026-08-02 · Agentic / MCP System
Model weights are only one executable release component; chat templates must be digest-bound, provenance-checked, and security-regression-tested with the same rigor as the weights.
Indicative ERS 4.8 · 1.9–7.7
2026-08-02 · Standalone LLM / Generative AI
A familiar model name is not an identity: deployment admission must bind approved bytes, signer, source ownership, and loader policy before first load and every reload.
Indicative ERS 4.0 · 1.7–6.4
2026-08-02 · Standalone LLM / Generative AI
A hosted model's caller identity and its extraction monitoring are both security controls: when one actor can operate thousands of authenticated accounts behind resale proxies, authentication and regional policy pass while capability extraction succeeds, so AITBM records the failure on Cn-5 and Cn-1 and the late detection on Pr-2 at 0.25 rather than treating authenticated traffic as trusted.
Indicative ERS 4.0 · 1.3–6.7
2026-08-02 · Standalone LLM / Generative AI
For a hosted model service the decisive control is credential binding, not the model: authorisation worked as designed, yet a replayable credential plus an error-oracle side channel let attackers confirm access, check logging, and resell inference, so AITBM records the loss on Cn-5 and Cn-4 and leaves the model's own behaviour unscored.
Indicative ERS 3.5 · 1.1–5.9
2026-08-02 · Incident
Linux packages for PyTorch's pre-release version, called Pytorch-nightly, were compromised from December 25 to 30, 2022 by a malicious binary uploaded to the Python Package Index (PyPI) code repository. The malicious binary had the same name as a PyTorch dependency and the PyPI package manager (pip) installed this malicious package instead of the legitimate one. This supply chain attack, also known as "dependency confusion," exposed sensitive information of Linux machines with the affected pip-installed versions of PyTorch-nightly. On December 30, 2022, PyTorch announced the incident and initial steps towards mitigation, including the rename and removal of `torchtriton` dependencies.
Research note · no ERS
2026-07-07 · Traditional ML / Classifier
A serving runtime's own error path can hand an unauthenticated caller the key to its private memory: Cn-4 and Cn-5 at 0.00 and Cn-1 at 0.25 show that information leakage, caller identity, and API scope compose into one chain that no single sub-metric captures, while a deterministic vendor patch keeps Rf at 0.00.
Indicative ERS 3.2 · 1.1–5.2
2026-07-06 · Agentic / MCP System
JADEPUFFER is the case that forces the assessor to keep the attacker out of the assessed system: the agentic behaviour that compressed the kill chain to 31-second self-repair belonged to the offence, so it drives no ORP or BAW score here — what AITBM scores is a Tier 1 workflow host with Cn-1, Cn-2 and Cn-5 all at 0.00 and a corroborated Cp = 1.00 traced from an unauthenticated origin to a P4 backdoor-administrator terminal.
Indicative ERS 7.0 · 3.2–10.0
2026-07-02 · Standalone LLM / Generative AI
Architecture class is scored on the assessed boundary, not on the attacker's behaviour: the exposed backend is a Standalone LLM with Aa = 0.25 and baw/agentic both false, yet three Containment sub-metrics sit at 0.00 and Cp is corroborated at 1.00 — an unauthenticated endpoint can be maximally exposed and trivially remediable (Rf = 0.00) at the same time, and the thin two-axis evidence base correctly forces the Lite Ec cap.
Indicative ERS 4.4 · 1.9–6.8
2026-06-29 · Agentic / MCP System
When a backdoor lives in model-writable memory, the egress fix and the persistence fix are different remediations: closing the font-src channel under CVE-2026-24299 does not retire the unaudited record_memory write path, which is why Rf sits at 0.50, Tr-3 is scored from the Purview gap, and the behavioural staleness window applies.
Indicative ERS 5.9 · 2.9–8.9
2026-06-27 · Multi-Agent / MCP System
Detection capability and evidence freshness are scored in different layers, and this case separates them cleanly: the monitor's blindness is a point-in-time IVP finding at Ro-1 and Cn-3, while the same sensor loss independently caps ACI Temporal Freshness through C_monitor and Band 0 C_behavior — and because the report measured visibility rather than consequence, Cn-1 and Cn-6 are correctly omitted instead of guessed.
Indicative ERS 5.0 · 1.9–8.0
2026-05-26 · RAG / Retrieval-Augmented System
Remediation Feasibility sits at its bottom anchor — a patchable ordering bug with a CVE and a fixed release — while Cascade Potential still forces 1.00, which is exactly the layer separation AITBM is built for: how easily a defect is fixed and how far it reaches are scored independently, so a cleanly patchable flaw never gets to look harmless.
Indicative ERS 4.8 · 2.7–6.9
2026-05-15 · Agentic / MCP System
Nothing in this case reaches the model — there is no prompt, no injection, and no Robustness-1 signal at all — so AITBM's not-applicable redistribution rule carries the whole assessment on Containment, identity, and posture evidence, which is the correct answer for an incident where, as the brief puts it, security was lost before the model saw any prompt.
Indicative ERS 7.5 · 4.1–10.0
2026-04-24 · Traditional ML / Classifier
An artifact-supply-chain incident with no agent, no prompt and no model output is still fully scorable — the evidence lands on Ro-5, Tr-3/Tr-4 and Cn-1/Cn-2 — and because the assessed configuration was observed in February 2024, Temporal Freshness rather than the attack itself is what collapses assurance confidence, which is the temporal layer behaving exactly as designed.
Indicative ERS 5.0 · 2.6–7.4
2026-04-16 · Research
Researchers showed that a student model can inherit a teacher model's behavioural traits from distillation data that carries no visible trace of the trait. A teacher is first nudged toward a preference (the paper's example is a stated fondness for owls) or fine-tuned into broader misalignment; the teacher then emits ordinary-looking content such as number sequences, code snippets, or chain-of-thought; a student fine-tuned on that content exhibits the same preference or misalignment when prompted on unrelated topics. Ordinary defences miss it: format restrictions, banned-number lists, and LLM-judge filtering removed the obvious clues while transfer still occurred, and the researchers' in-context-learning and LLM-judge probes could not detect the trait in the training data. Transfer depends on shared or behaviourally matched initialisation between teacher and student, which makes base-model lineage - not dataset content - the reliable predictor. No system was attacked and no operator was affected.
Research note · no ERS
Each record distinguishes observed controls, missing evidence, architecture-based exclusions, operational risk, confidence limits, and the uncertainty interval. When using a case as a reference, compare the documented path to your own system boundary and rerun the applicable test methods rather than copying its indicative ERS.
