Use case sources
Source reports and analysis supporting this case are listed below. AITBM interpretation and any indicative scoring are derived from this public evidence.
Deep learning models are increasingly used in mobile applications as critical components. Researchers from Microsoft Research demonstrated that many deep learning models deployed in mobile apps are vulnerable to backdoor attacks via "neural payload injection." They conducted an empirical study on real-world mobile deep learning apps collected from Google Play. They identified 54 apps that were vulnerable to attack, including popular security and safety critical applications used for cash recognition, parental control, face authentication, and financial services.
ASSESSED SYSTEM
Representative configuration: an Android application distributed through Google Play that embeds a TensorFlow or TFLite deep learning model in its APK and loads it at runtime without verifying the model's integrity, as found in the 54 vulnerable apps of the DeepPayload study (including cash-recognition, parental-control, face-authentication, and financial-services apps). The apps share one mechanism and one configuration class and are scored as a single representative class in the state the researchers tested.
OUT OF SCOPE
Google Play's distribution controls and the supply-chain compromise needed to install a repackaged APK on victim devices, which the record states but does not demonstrate; any individually named app; the researchers' trigger-detector training and repackaging tooling, which is attacker tooling; and extraction of the model as an intellectual-property question, which no rubric anchor in this boundary measures.
Architecture: Traditional ML / Classifier (decision tree Q6) — Host-inheritance rule: the AI workload is the on-device deep learning model the app runs, for example image classification for cash recognition or face authentication. No generative interface, inference-time retrieval, tool or function calling, autonomous planning, or agent coordination is described, so Q1-Q5 resolve NO and the decision tree ends at Q6. Tier 2: Tier 2 Consumer: the apps are distributed to the public through Google Play and include face-authentication and financial-services uses that decide on behalf of consumer users; the evidence does not establish a critical deployment that would support Tier 1.
Documented attack or failure path
- The researchers searched Google Play for apps likely to embed deep learning models and acquired their APKs.
- They filtered for TensorFlow or TFLite metadata and model binary formats and extracted the models with Apktool, obtaining full access to each model in compiled form.
- They developed a neural payload — a trigger detector plus conditional logic — trained with a general same-modality dataset and several photos of a physical trigger.
- They injected the payload by directly modifying each compiled model's computation graph and repackaged the poisoned model into the APK.
- They confirmed that the app did not crash with the malicious model in place and that the trigger detector worked, identifying 54 vulnerable apps.
- At inference time a physical trigger captured by the device camera bypasses the victim model and yields attacker-chosen outputs; delivery to victims would require a separate supply-chain compromise.
Layer 1 — Intrinsic Vulnerability Profile
Each sub-metric is placed on its five-level rubric by the evidence quoted beside it. Missing applicable evidence remains unknown. The displayed midpoint and interval are scenario values, not inferred control performance.
| Sub-metric | Score | Rubric basis | Evidence |
|---|---|---|---|
| Robustness (Ro) — scenario interval 0.00–0.80 (midpoint 0.40), Tier 2 MVT 0.50 indeterminate | |||
| Ro-5Supply-Chain Artifact Admission Integrity | 0.00w 0.20 | The 0.00 anchor: the model artifact was admitted and loaded with no integrity, signature or provenance check and the modified graph ran without any check failing. The 0.25 anchor requires pinning or allow-listing for at least some artifact classes, and none operated on the embedded-model channel. (Moved from Ro-4; the evidence concerns load-time admission of an unverified model artifact, not content poisoning.) | The assessed apps embed a TensorFlow or TFLite model in the APK and load it at runtime without verifying the model's integrity. The researchers modified each compiled model's computation graph, repackaged the poisoned model into the APK, and 'confirmed that the app did not crash with the malicious model in place and that the trigger detector worked', identifying 54 vulnerable apps.source: secureflow/atlas |
| Fairness (Fa) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
| Transparency (Tr) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.50 indeterminate | |||
| Privacy (Pr) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
| Containment (Cn) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
Unknown, not N/A: 21 applicable sub-metrics lack admissible public evidence. The lower, midpoint, and upper scenarios evaluate each at 0.00, 0.50, and 1.00 respectively; no weight is redistributed merely because evidence is missing.
Architecture-based NOT APPLICABLE: declared with an architecture rationale in the workpaper: Cn-5, Cn-6; NOT APPLICABLE by design for this architecture class: Cn-8, Cn-9.
Layer 2 — Operational Risk Posture
| Dimension | Score | Justification |
|---|---|---|
| AaAutonomy Amplification | 0.50w 0.25 | Model predictions drive in-app decisions such as face authentication, cash recognition, and parental-control filtering automatically, with no per-decision human review; the app acts within pre-defined boundaries, matching the human-on-the-loop anchor rather than suggestion-only operation. High-consequence autonomous execution is not evidenced, so 0.75 is not supported. |
| AsAttack Surface Exposure | 0.75w 0.35 | The model consumes camera input from the physical environment, which an attacker can influence with a placed trigger, and the model artifact ships in a publicly downloadable APK that can be extracted and repackaged; this is external, partially trusted input without multi-agent or MCP exposure. |
| CpCascade Potential | 1.00w 0.20 | No verified System Dependency Graph with DGC at least 0.90 is public for this boundary, so the specification's worst-case graph default governs rather than an assessor-estimated blast radius.GDCP: worst-case default — no verified dependency graph published elevated |
| RfRemediation Feasibility | 0.25w 0.20 | Remediation is application and deployment infrastructure: verify model integrity at load with signed or hashed model files, protect the packaged model, and ship a corrected release. The legitimate model needs no retraining because the backdoor exists only in repackaged copies. |
Nelevated = 1 (dimensions strictly above 0.75) → CRM = 1.00.
Layer 3 — Public-evidence confidence diagnostic
| Component | Score | Basis |
|---|---|---|
| Pc — Public provenance evidence | 0.35 | The public record identifies the representative configuration, attack path, and principal control boundaries, but does not provide a complete asset manifest, verified dependency graph, configuration export, or assessment evidence manifest. |
| Ec — Public evaluation coverage | 0.04 | coverage 0.05 (1 of 22 applicable sub-metrics) × independence 1.00 × fidelity 0.85. No Full, Standard, or Lite pathway is claimed for a retrospective article. |
| Tf — Public-evidence freshness | 0.00 | Evidence dated 2021-01-18; age 2099 days on the workpaper reference date. Components: T_calendar 0.00 · C_monitor 0.65 · C_event 0.65 · C_evidence 0.85. Binding term: T_calendar. Evidence age is measured from the MITRE ATLAS incident date 2021-01-18 (day granularity) to the 2026-10-18 evidence reference date, so the calendar term dominates. C_monitor 0.65: no app detected the modified model; the finding came from external research. C_event 0.65: a research exercise with no reported in-the-wild exploitation. C_evidence 0.85: the record does not establish that the affected apps added model integrity verification. agentic = false and baw = false: the on-device model is static and has no runtime tool, memory, or configuration mutation. |
Public-evidence ACI = (Pc × Ec × Tf)1/3 = 0.00 — diagnostic status: Invalid as assessment-of-record evidence. It describes the evidence available to this case study, not the assurance of the underlying system, and is not inserted into the normalized-assurance scenario ERS.
Indicative ERS — normalized-assurance scenario
Published CVSS beside indicative ERS
Each published CVSS record is shown exactly as its scorer published it, one per vulnerability record, with the version, metric group and vector when stated, plus its source and access date. Records are never combined or inferred from a CVE's existence.
INDICATIVE ERS · NORMALIZED-ASSURANCE SCENARIO
3.8 midpoint, interval 1.2–6.4; Low-Moderate scenario band; ACI fixed at 1.00. The published CVSS records do not enter this calculation.
PUBLISHED CVSS
CVSS — no applicable published score verified.
CVSS describes an individual vulnerability as its scorer published it. Indicative ERS describes the assessed AI deployment under the normalized-assurance scenario, so the two values are not direct equivalents and neither replaces the other.
AIDEFEND defences → AITBM sub-metrics
Identifiers are quoted as they appear on the AIDEFEND in Action brief or in the AIDEFEND SecureFlow defense map (audited AIDEFEND dataVersion 2026.10.15); the sub-metric mapping is AITBM's own, from the specification's AIDEFEND tables reconciled at catalogue data version 2026.10.15. AIDEFEND renumbers identifiers between releases, so the data version travels with every mapping and neither side's IDs should be cited without one. A mapping identifies a possible evidence route; a recommendation does not prove that the control was implemented or effective and receives no scoring credit by itself.
| Technique | Defence | Priority | Evidences |
|---|---|---|---|
| AID-H-003.002 | CI/CD Release Gating, Model Artifact Signing & Secure DistributionExact SecureFlow defense-map identifier (flow atlas-aml-cs0013-mobile-app-model-backdoor), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-7. Presence supplies no score credit. | Candidate evidence route | Ro-4 Ro-5 Tr-4 |
| AID-H-003.009 | Model Artifact Static Malware & Unsafe-Serialization ScanningExact SecureFlow defense-map identifier (flow atlas-aml-cs0013-mobile-app-model-backdoor), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-7. Presence supplies no score credit. | Candidate evidence route | Ro-4 Ro-5 Tr-4 |
| AID-D-004.001 | Static Artifact Integrity Re-verification & Drift MonitoringExact SecureFlow defense-map identifier (flow atlas-aml-cs0013-mobile-app-model-backdoor), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-7. Presence supplies no score credit. | Candidate evidence route | Ro-5 Tr-3 Cn-7 Cn-9 |
| AID-E-003.001 | Compromised Model Quarantine & Recovery RoutingExact SecureFlow defense-map identifier (flow atlas-aml-cs0013-mobile-app-model-backdoor), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-10. Presence supplies no score credit. | Candidate evidence route | Ro-4 |
| AID-R-001.002 | Model Retraining for RemediationExact SecureFlow defense-map identifier (flow atlas-aml-cs0013-mobile-app-model-backdoor), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-10. Presence supplies no score credit. | Candidate evidence route | Ro-4 |
| AID-D-002.002 | Labeled Performance Tracking with Ground Truth Feedback LoopExact SecureFlow defense-map identifier (flow atlas-aml-cs0013-mobile-app-model-backdoor), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-10. Presence supplies no score credit. | Candidate evidence route | Ro-2 Fa-1 Fa-2 Tr-2 Cn-7 |
WHAT THIS CASE TEACHES
Shipping a model inside a client package moves its integrity boundary onto the device, and an app that loads that model without verification scores Ro-5 = 0.00 however accurate the clean model is; a single observed sub-metric also shows how thin evidence widens the scenario interval.
Sources: AIDEFEND SecureFlow — Backdoor Attack on Deep Learning Models in Mobile Apps · MITRE ATLAS case study AML.CS0013 · DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection (arxiv.org)
SecureFlow attack flow
This record mirrors an AIDEFEND SecureFlow flow (MITRE ATLAS case-study conversion): an ordered attack path with MITRE ATLAS tactic and technique labels and the AIDEFEND controls matched to each step. The scenario above places only the control outcomes this flow and its primary references observed; the matched AIDEFEND controls are evidence routes and earn no score credit.
| MITRE ATLAS case study | AML.CS0013 (exercise) |
|---|---|
| Target | ML-based Android Apps |
| Actor | Yuanchun Li, Jiayi Hua, Haoyu Wang, Chunyang Chen, Yunxin Liu |
| Primary evidence date | 2021-01-18 |
| SecureFlow record updated | 2026-08-06 |
| AI layers | AI supply chain, Edge and client AI |
| Risk impact | Model poisoning / integrity |
| Affected components | mobile deep learning model, APK model binary, injected neural-payload backdoor, visual trigger |
Attack steps
| # | Step | ATLAS tactic / technique |
|---|---|---|
| 1 | Search App Store for ML AppsTo identify a list of potential target models, the researchers searched the Google Play store for apps that may contain embedded deep learning models by searching for deep learning related keywords. | AML.TA0002 AML.T0004 |
| 2 | Extract Models from APKsThe researchers acquired the apps' APKs from the Google Play store. They filtered the list of potential target applications by searching the code metadata for keywords related to TensorFlow or TFLite and their model binary formats (.tf and .tflite). The models were extracted from the APKs using Apktool. | AML.TA0003 AML.T0002.001 |
| 3 | Gain Full Model AccessThis provided the researchers with full access to the ML model, albeit in compiled, binary form. | AML.TA0000 AML.T0044 |
| 4 | Develop Neural Payload BackdoorThe researchers developed a novel approach to insert a backdoor into a compiled model that can be activated with a visual trigger. They inject a "neural payload" into the model that consists of a trigger detection network and conditional logic. The trigger detector is trained to detect a visual trigger that will be placed in the real world. The conditional logic allows the researchers to bypass the victim model when the trigger is detected and provide model outputs of their choosing. The only requirements for training a trigger detector are a general dataset from the same modality as the target model (e.g. ImageNet for image classification) and several photos of the desired trigger. | AML.TA0003 AML.T0017.000 |
| 5 | Inject Backdoor into ModelThe researchers poisoned the victim model by injecting the neural payload into the compiled models by directly modifying the computation graph. The researchers then repackage the poisoned model back into the APK | AML.TA0006 AML.T0018.001 |
| 6 | Verify Backdoor FunctionalityTo verify the success of the attack, the researchers confirmed the app did not crash with the malicious model in place, and that the trigger detector successfully detects the trigger. | AML.TA0001 AML.T0042 |
| 7 | Distribute Poisoned APK via Supply ChainIn practice, the malicious APK would need to be installed on victim's devices via a supply chain compromise. | AML.TA0004 AML.T0010.003 |
| 8 | Place Visual Trigger in EnvironmentThe trigger is placed in the physical environment, where it is captured by the victim's device camera and processed by the backdoored ML model. | AML.TA0001 AML.T0043.004 |
| 9 | Trigger Backdoor via Physical AccessAt inference time, only physical environment access is required to trigger the attack. | AML.TA0000 AML.T0041 |
| 10 | Evade Victim ModelPresenting the visual trigger causes the victim model to be bypassed. The researchers demonstrated this can be used to evade ML models in several safety-critical apps in the Google Play store. | AML.TA0011 AML.T0015 |
Threat references
Summary and attack steps reproduced from AIDEFEND SecureFlow (https://aidefendlabs.com/en/secureflow/?flow=atlas-aml-cs0013-mobile-app-model-backdoor; AIDEFEND content CC BY 4.0), converted from MITRE ATLAS case study AML.CS0013 (Apache License 2.0; MITRE ATLAS is a trademark of The MITRE Corporation).