Use case sources
Source reports and analysis supporting this case are listed below. AITBM interpretation and any indicative scoring are derived from this public evidence.
- AIDEFEND SecureFlow — ProofPoint Evasion
- MITRE ATLAS case study AML.CS0008
- National Vulnerability Database entry for CVE-2019-20634 (nvd.nist.gov)
- 2019 DerbyCon presentation "42: The answer to life, the universe, and everything offensive security" (github.com)
- Proof Pudding (CVE-2019-20634) Implementation on GitHub (github.com)
- 2019 DerbyCon video presentation "42: The answer to life, the universe, and everything offensive security" (youtube.com)
Proof Pudding (CVE-2019-20634) is a code repository that describes how ML researchers evaded ProofPoint's email protection system by first building a copy-cat email protection ML model, and using the insights to bypass the live system. More specifically, the insights allowed researchers to craft malicious emails that received preferable scores, going undetected by the system. Each word in an email is scored numerically based on multiple variables and if the overall score of the email is too low, ProofPoint will output an error, labeling it as SPAM.
ASSESSED SYSTEM
The ML scoring model of the ProofPoint Email Protection System as operated in 2019, in its pre-fix state: inbound email is scored per word on multiple variables, an insufficient overall score labels the message as spam, and the model's output scores, including the 'mlxlogscore' value, were written into the headers of processed email where senders who receive the message could read them.
OUT OF SCOPE
The researchers' Proof Pudding proxy model, training pipeline and adversarial emails (attacker tooling); ProofPoint's non-ML filtering layers, sandboxing and URL defences, none of which the record describes; any customer tenant configuration; and the product after the CVE-2019-20634 disclosure, whose remediation state the record does not document.
Architecture: Traditional ML / Classifier (decision tree Q6) — The decision tree resolves at Q6 by elimination. Q1 = NO (no coordinating agents). Q2 = NO (no planner, goal or task state; the model returns a verdict for each input). Q3 = NO (the model invokes no function, API, plugin or MCP tool). Q4 = NO (no inference-time retrieval layer). Q5 = NO (no generative interface; the output is a classification score or verdict). What is described is an email-scoring classifier with bag-of-words style per-word scoring and a spam threshold, so classifier intra-axis weights apply. Cn-5, Cn-6 and Cn-7 are left applicable and UNKNOWN because the service acts automatically on its verdicts and the record does not describe that action space. Tier 2: Tier 2 Consumer: the system scores email from arbitrary internet senders and decides whether it reaches individual recipients, so a bypass exposes individuals to malicious email. The record establishes no immediate harm at scale that would support Tier 1, and the input population is not controlled internal users as Tier 3 requires.
Documented attack or failure path
- Researchers found that ProofPoint Email Protection left model output scores in the headers of processed email.
- They sent many emails through the live system and harvested the scores from the headers.
- They used the emails and harvested 'mlxlogscore' values as a labelled dataset to train a functional copy of the ProofPoint model (an ANN with bag-of-words tokenizing).
- They algorithmically mined the offline proxy for influential tokens, identifying good-scoring and bad-scoring words.
- Using those insights they crafted malicious emails that received preferable scores from the real ProofPoint system and went undetected.
Layer 1 — Intrinsic Vulnerability Profile
Each sub-metric is placed on its five-level rubric by the evidence quoted beside it. Missing applicable evidence remains unknown. The displayed midpoint and interval are scenario values, not inferred control performance.
| Sub-metric | Score | Rubric basis | Evidence |
|---|---|---|---|
| Robustness (Ro) — scenario interval 0.08–0.76 (midpoint 0.42), Tier 2 MVT 0.50 indeterminate | |||
| Ro-1Adversarial Input Resistance | 0.25w 0.32 | Ro-1 = 0.25: the scorer labels ordinary low-scoring email as spam, so naive malicious email is handled, but it failed under a moderate, transfer-based perturbation. The 0.50 anchor requires resisting common adversarial attacks with ASR below 50%; a proxy-transfer word-substitution bypass of the live system contradicts that. Not 0.00 because no success-rate measurement above 80% is published; placement is from the qualitative criterion. | Insights mined from an offline proxy model allowed the researchers to craft malicious emails that received preferable scores from the real ProofPoint email protection system and went undetected. The perturbation is token selection guided by a black-box transfer proxy, and it transferred to the live production scorer.source: secureflow/atlas |
| Fairness (Fa) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
| Transparency (Tr) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.50 indeterminate | |||
| Privacy (Pr) — scenario interval 0.00–0.65 (midpoint 0.33), Tier 2 MVT 0.55 indeterminate | |||
| Pr-2Inference Attack Resistance | 0.00w 0.35 | REVISED Pr-2 extraction criteria (2026-10-14); inference attacks are not evidenced and stay UNKNOWN. The 0.00 extraction anchor is met: ordinary use of the system yielded a functionally equivalent copy, and no control limited or detected the extraction. Pr-2 records that the model was replicated; Cn-4 records the separate fact that a header channel exposed raw scores, so the two placements measure different controls. | By sending many emails through the production system and harvesting the model scores it wrote into message headers, the researchers trained a functional copy of the email-protection model, which they then used to craft emails the live system scored as benign. No control is reported as limiting or detecting the harvesting.source: secureflow/atlas |
| Containment (Cn) — scenario interval 0.00–0.73 (midpoint 0.36), Tier 2 MVT 0.55 indeterminate | |||
| Cn-4Side-Channel Resistance | 0.00w 0.27 | Cn-4 = 0.00: a telemetry channel (message headers) revealed model information, namely raw output scores, to any party able to receive the processed email, which is the anchor condition verbatim. The 0.25 anchor requires basic protections such as rate limiting or redaction with residual leakage; the evidence shows scores harvested in volume with no such control. | The production system wrote model output scores into email headers, and the researchers harvested them at scale by sending many emails through the system, then used them as training labels to build a functional copy of the model. The weakness is recorded as CVE-2019-20634.source: secureflow/atlas |
Unknown, not N/A: 21 applicable sub-metrics lack admissible public evidence. The lower, midpoint, and upper scenarios evaluate each at 0.00, 0.50, and 1.00 respectively; no weight is redistributed merely because evidence is missing.
Architecture-based NOT APPLICABLE: NOT APPLICABLE by design for this architecture class: Cn-8, Cn-9.
Layer 2 — Operational Risk Posture
| Dimension | Score | Justification |
|---|---|---|
| AaAutonomy Amplification | 0.50w 0.25 | Human-on-the-loop. The scorer decides spam or deliver for each message without per-message human review, which is routine autonomous action within a pre-defined boundary; the record documents no wider decision, tool or transaction authority that would support 0.75 or 1.00. |
| AsAttack Surface Exposure | 0.85w 0.35 | Documented intermediate above the 0.75 anchor. The system ingests email from any internet sender, which is wholly untrusted rather than partially trusted, and in the assessed state it also returned its own scores to that population, giving the adversary an unthrottled query oracle against production. It is below 1.00 because no untrusted RAG, external agent channel or MCP tool integration exists. elevated |
| CpCascade Potential | 1.00w 0.20 | No System Dependency Graph is published for the assessed product, so the DGC gate is not met and the Section 3.2 worst-case default applies. The evidence records the detection bypass itself but no observed downstream path, so the default is not independently corroborated.GDCP: worst-case default — no verified dependency graph published elevated |
| RfRemediation Feasibility | 0.50w 0.20 | The 0.50 anchor. Removing scores from headers is a deterministic configuration change (0.00), but it does not withdraw a functional copy already trained from harvested scores, and closing the demonstrated token-level evasion requires retraining or re-weighting the scoring model with probabilistic effectiveness and re-evaluation. The harder component governs. |
Nelevated = 2 (dimensions strictly above 0.75) → CRM = 1.15.
Compound Risk Alert. Two or more dimensions are simultaneously elevated (spec 3.2.2); architectural decomposition is recommended before deployment.
Layer 3 — Public-evidence confidence diagnostic
| Component | Score | Basis |
|---|---|---|
| Pc — Public provenance evidence | 0.25 | At the 0.25 anchor. The product, vendor and operator are identified, and the record names the scoring variable and the spam-threshold behaviour, but no AIBOM, model lineage, training-data description or deployment inventory exists. |
| Ec — Public evaluation coverage | 0.11 | coverage 0.12 (3 of 24 applicable sub-metrics) × independence 1.00 × fidelity 0.85. No Full, Standard, or Lite pathway is claimed for a retrospective article. |
| Tf — Public-evidence freshness | 0.00 | Evidence dated 2019-09-09; age 2596 days on the workpaper reference date. Components: T_calendar 0.00 · C_monitor 0.65 · C_event 0.65 · C_evidence 0.85. Binding term: T_calendar. evidence_date 2019-09-09 is the ATLAS incident date; the record carries no separately dated primary disclosure, so the ATLAS case is the evidence source and dt_days = 2596. C_monitor 0.65: large-volume score harvesting through the live service went undetected in the record, and the weakness was found externally. C_event 0.65: a CVE-assigned, publicly disclosed bypass of the production scorer is a moderate event; no in-the-wild exploitation is recorded. C_evidence 0.85: no remediation or retest is documented in the record. agentic = false and baw = false: no runtime-mutable permission boundary, and no online learning or persistent behavioural state is evidenced. |
Public-evidence ACI = (Pc × Ec × Tf)1/3 = 0.00 — diagnostic status: Invalid as assessment-of-record evidence. It describes the evidence available to this case study, not the assurance of the underlying system, and is not inserted into the normalized-assurance scenario ERS.
Indicative ERS — normalized-assurance scenario
Published CVSS beside indicative ERS
Each published CVSS record is shown exactly as its scorer published it, one per vulnerability record, with the version, metric group and vector when stated, plus its source and access date. Records are never combined or inferred from a CVE's existence.
INDICATIVE ERS · NORMALIZED-ASSURANCE SCENARIO
5.3 midpoint, interval 2.4–8.2; Moderate scenario band; ACI fixed at 1.00. The published CVSS records do not enter this calculation.
PUBLISHED CVSS · 1 record
CVE-2019-20634 CVSS 3.1 Base 3.7 Low
| Published score | 3.7 |
|---|---|
| CVSS version | 3.1 |
| Metric group | Base |
| Severity | Low |
| Vector | CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:L/I:N/A:N |
| Scored by | mitre |
| Source record | https://raw.githubusercontent.com/CVEProject/cvelistV5/main/cves/2019/20xxx/CVE-2019-20634.json (accessed 2026-10-18) |
| Affected component | n/a |
| Scope note | CNA affected product and described vulnerability match the scored assessed_system and are not excluded by out_of_scope. |
| Record SHA-256 | 4573f8b48fdfde26015fefd689a4c3cdaa6f9848239bdb50a5eaa3953a1aa13c |
CVSS describes an individual vulnerability as its scorer published it. Indicative ERS describes the assessed AI deployment under the normalized-assurance scenario, so the two values are not direct equivalents and neither replaces the other.
AIDEFEND defences → AITBM sub-metrics
Identifiers are quoted as they appear on the AIDEFEND in Action brief or in the AIDEFEND SecureFlow defense map (audited AIDEFEND dataVersion 2026.10.15); the sub-metric mapping is AITBM's own, from the specification's AIDEFEND tables reconciled at catalogue data version 2026.10.15. AIDEFEND renumbers identifiers between releases, so the data version travels with every mapping and neither side's IDs should be cited without one. A mapping identifies a possible evidence route; a recommendation does not prove that the control was implemented or effective and receives no scoring credit by itself.
| Technique | Defence | Priority | Evidences |
|---|---|---|---|
| AID-H-006.003 | Prediction Response Minimization & Extraction-Resistance ValidationExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-1. Presence supplies no score credit. | Candidate evidence route | Ro-3 Cn-3 |
| AID-H-032.003 | Inference Telemetry & Debug Surface RestrictionExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-1. Presence supplies no score credit. | Candidate evidence route | Pr-2 Pr-4 Cn-4 |
| AID-I-003.002 | Identity, Session & API Rate-Limit ThrottlingExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-2. Presence supplies no score credit. | Candidate evidence route | Cn-5 Cn-6 Cn-7 |
| AID-D-005.007 | Token, Tool-Use, Request-Parameter & Cost Spike Detection & AlertingExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-2. Presence supplies no score credit. | Candidate evidence route | Tr-3 Cn-7 |
| AID-D-005.003 | Proactive AI Threat HuntingExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-2. Presence supplies no score credit. | Candidate evidence route | Tr-3 Cn-7 |
| AID-H-001 | Adversarial Robustness TrainingExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-5. Presence supplies no score credit. | Candidate evidence route | Ro-1 |
| AID-H-014 | Ensemble Methods for RobustnessExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-5. Presence supplies no score credit. | Candidate evidence route | Ro-1 Ro-2 |
| AID-D-002.001 | Input / Output Distribution Drift MonitoringExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-5. Presence supplies no score credit. | Candidate evidence route | Ro-2 Fa-1 Fa-2 Tr-2 Cn-7 |
| AID-D-002.002 | Labeled Performance Tracking with Ground Truth Feedback LoopExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-5. Presence supplies no score credit. | Candidate evidence route | Ro-2 Fa-1 Fa-2 Tr-2 Cn-7 |
| AID-H-002.002 | Inference-Time Prompt & Input ValidationExact SecureFlow defense-map identifier (flow atlas-aml-cs0008-proofpoint-evasion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-5. Presence supplies no score credit. | Candidate evidence route | Ro-1 Ro-5 Fa-1 Fa-3 Cn-8 |
WHAT THIS CASE TEACHES
Returning raw model scores to the sender is a Cn-4 side channel that turns a black-box classifier into a trainable oracle; the resulting copy is a Pr-2 extraction failure, and it is what makes the Ro-1 transfer evasion cheap, so containment and extraction resistance, not robustness, are the root causes.
Sources: AIDEFEND SecureFlow — ProofPoint Evasion · MITRE ATLAS case study AML.CS0008 · National Vulnerability Database entry for CVE-2019-20634 (nvd.nist.gov) · 2019 DerbyCon presentation "42: The answer to life, the universe, and everything offensive security" (github.com) · Proof Pudding (CVE-2019-20634) Implementation on GitHub (github.com) · 2019 DerbyCon video presentation "42: The answer to life, the universe, and everything offensive security" (youtube.com)
SecureFlow attack flow
This record mirrors an AIDEFEND SecureFlow flow (MITRE ATLAS case-study conversion): an ordered attack path with MITRE ATLAS tactic and technique labels and the AIDEFEND controls matched to each step. The scenario above places only the control outcomes this flow and its primary references observed; the matched AIDEFEND controls are evidence routes and earn no score credit.
| MITRE ATLAS case study | AML.CS0008 (exercise) |
|---|---|
| Target | ProofPoint Email Protection System |
| Actor | Researchers at Silent Break Security |
| Primary evidence date | 2019-09-09 |
| SecureFlow record updated | 2026-08-02 |
| AI layers | Predictive ML / computer vision |
| Risk impact | Model evasion / bypass, Data exfiltration |
| Affected components | email protection ML model, model scores leaked in email headers, replicated proxy model, adversarial emails |
Attack steps
| # | Step | ATLAS tactic / technique |
|---|---|---|
| 1 | Discover Model Scores in Email HeadersThe researchers discovered that ProofPoint's Email Protection left model output scores in email headers. | AML.TA0008 AML.T0063 |
| 2 | Query System to Collect Model OutputsThe researchers sent many emails through the system to collect model outputs from the headers. | AML.TA0000 AML.T0047 |
| 3 | Train Proxy Email Protection ModelThe researchers used the emails and collected scores as a dataset, which they used to train a functional copy of the ProofPoint model. Basic correlation was used to decide which score variable speaks generally about the security of an email. The "mlxlogscore" was selected in this case due to its relationship with spam, phish, and core mlx and was used as the label. Each "mlxlogscore" was generally between 1 and 999 (higher score = safer sample). Training was performed using an Artificial Neural Network (ANN) and Bag of Words tokenizing. | AML.TA0001 AML.T0005.001 |
| 4 | Mine Adversarial Insights from ProxyNext, the ML researchers algorithmically found samples from this "offline" proxy model that helped give desired insight into its behavior and influential variables. Examples of good scoring samples include "calculation", "asset", and "tyson". Examples of bad scoring samples include "software", "99", and "unsub". | AML.TA0001 AML.T0043.002 |
| 5 | Bypass ProofPoint Email ProtectionFinally, these insights from the "offline" proxy model allowed the researchers to create malicious emails that received preferable scores from the real ProofPoint email protection system, hence bypassing it. | AML.TA0011 AML.T0015 |
Threat references
Summary and attack steps reproduced from AIDEFEND SecureFlow (https://aidefendlabs.com/en/secureflow/?flow=atlas-aml-cs0008-proofpoint-evasion; AIDEFEND content CC BY 4.0), converted from MITRE ATLAS case study AML.CS0008 (Apache License 2.0; MITRE ATLAS is a trademark of The MITRE Corporation).