Use case sources
Source reports and analysis supporting this case are listed below. AITBM interpretation and any indicative scoring are derived from this public evidence.
Skyld researchers analyzed the Google Photos Android application and recovered TensorFlow Lite models used by AI-powered photo editing and image analysis features. The researchers found models stored unencrypted in the application's assets, embedded in the native library, and encrypted on disk. They used static reverse engineering to locate TFLite artifacts and dynamic instrumentation with Frida to capture encrypted models after runtime decryption. The recovered models provided white-box access to proprietary Google Photos model artifacts.
ASSESSED SYSTEM
The on-device machine-learning models shipped in the Google Photos Android application as analysed by Skyld (MITRE ATLAS incident date March 2025): TensorFlow Lite models used by AI-powered photo editing and image analysis features — including face detection, object detection, segmentation, depth estimation, image quality assessment, and blur detection — packaged unencrypted in the APK assets, embedded in the native library, or stored encrypted on disk in application-specific folders and decrypted at runtime for execution.
OUT OF SCOPE
Google Photos server-side services and any cloud-hosted models; app features whose models are not among those the record lists; the researchers' tooling (static reverse engineering and Frida instrumentation); and the white-box adversarial-example generation, vision-model evasion, and intellectual-property reuse that the record describes only as potential consequences.
Architecture: Traditional ML / Classifier (decision tree Q6) — The decision tree resolves at Q6 by elimination. Q1 = NO and Q2 = NO: no agents, planner, or tool invocation. Q3 = NO: the models call no functions; application code invokes them. Q4 = NO: no inference-time retrieval layer is described. Q5 = NO: the recovered models perform detection, segmentation, depth estimation, and quality assessment, which are predictive outputs rather than open-ended generation. What remains is a set of bounded on-device vision models driven by application logic, the Q6 class. Tier 2: Tier 2 Consumer: the models run inside a mass-distributed consumer application that members of the public use directly, and the record's potential consequences (evasion or degradation of face and object detection) would affect individual users. The observed outcome — extraction of proprietary model artefacts — harms the vendor rather than causing immediate harm at scale, so Tier 1 is not supported.
Documented attack or failure path
- Skyld researchers analysed the Google Photos Android Package, identified TensorFlow Lite as the framework, and searched the package and native libraries for the TFL3 file identifier.
- They collected TensorFlow Lite model files from the unencrypted APK assets, from files embedded in the native library, and from application-specific folders.
- For models encrypted on disk, they used Frida instrumentation to intercept the decrypted TFLite files at runtime as Google Photos loaded them for execution.
- The recovered files gave full white-box access to the Google Photos models used for face detection, object detection, segmentation, depth estimation, image quality assessment, and blur detection.
- The record identifies white-box adversarial-example generation against these models and reuse of the extracted models as intellectual property as potential consequences; neither is reported as carried out.
Observed controls and bounded outcomes
Positive credit is given only where the record directly demonstrates a control operating. Recommended or merely presumed controls receive no positive scoring credit.
- Part of the model set was encrypted at rest: static analysis of the package did not yield those models in usable form, and the researchers had to instrument the running application to capture them after decryption. Encryption raised the cost for that subset but did not protect the models once loaded, and it did not cover the models shipped unencrypted, so the lower extraction placement governs.
Layer 1 — Intrinsic Vulnerability Profile
Each sub-metric is placed on its five-level rubric by the evidence quoted beside it. Missing applicable evidence remains unknown. The displayed midpoint and interval are scenario values, not inferred control performance.
| Sub-metric | Score | Rubric basis | Evidence |
|---|---|---|---|
| Robustness (Ro) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.50 indeterminate | |||
| Fairness (Fa) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
| Transparency (Tr) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.50 indeterminate | |||
| Privacy (Pr) — scenario interval 0.00–0.65 (midpoint 0.33), Tier 2 MVT 0.55 indeterminate | |||
| Pr-2Inference Attack Resistance | 0.00w 0.35 | REVISED Pr-2 extraction criteria (2026-10-14); inference attacks are not evidenced and stay UNKNOWN. The unencrypted models meet the 0.00 extraction condition: a client-distributed artifact yields a functionally equivalent model, and no control limits or detects it. The encrypted subset meets only the 0.25 condition (artifact encryption that a client holder can defeat at runtime). Each artifact is placed separately and the lower placement governs, so Pr-2 is 0.00; the 0.75 condition (artifacts unusable outside the authorized runtime) is not met for any model. | Proprietary model artefacts were recoverable from the client: several TFLite models sat unencrypted in the APK assets or the native library, and the encrypted subset was captured in decrypted form through Frida instrumentation while the app loaded it, giving white-box access to the Google Photos vision models. No control limited or detected either recovery.source: secureflow/atlas |
| Containment (Cn) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
Unknown, not N/A: 21 applicable sub-metrics lack admissible public evidence. The lower, midpoint, and upper scenarios evaluate each at 0.00, 0.50, and 1.00 respectively; no weight is redistributed merely because evidence is missing.
Architecture-based NOT APPLICABLE: declared with an architecture rationale in the workpaper: Cn-5, Cn-6; NOT APPLICABLE by design for this architecture class: Cn-8, Cn-9.
Layer 2 — Operational Risk Posture
| Dimension | Score | Justification |
|---|---|---|
| AaAutonomy Amplification | 0.25w 0.25 | The models run on-device to produce detections and estimates for photo-editing and analysis features; no consequential decision, write authority, or external action is evidenced. Scored at 0.25 rather than 0.00 because outputs are produced without per-output review, and below 0.50 because nothing is executed on the user's behalf beyond in-app processing. |
| AsAttack Surface Exposure | 0.75w 0.35 | Placed at 0.75. The application is publicly distributed, so the model artefacts reside on devices the attacker fully controls, where static analysis and dynamic instrumentation operate without any interposed validation layer; that exceeds the 0.50 anchor's validated internet-facing exposure. Not 1.00, because no untrusted RAG, multi-agent, or MCP surface exists. |
| CpCascade Potential | 1.00w 0.20 | No verified System Dependency Graph with DGC at least 0.90 is public, so the specification's worst-case default governs. The observed outcome is artefact extraction; the downstream evasion and intellectual-property paths are potential only, so the score is not corroborated.GDCP: worst-case default — no verified dependency graph published elevated |
| RfRemediation Feasibility | 0.75w 0.20 | Guardrail-only anchor. Any model executed on a device the attacker controls can be recovered once it is decrypted for execution; obfuscation, encryption, and runtime integrity checks raise the cost but cannot eliminate extraction. Not 1.00, because the property follows from on-device deployment rather than from the model class, and moving the most sensitive models to server-side or attested execution bounds it. |
Nelevated = 1 (dimensions strictly above 0.75) → CRM = 1.00.
Layer 3 — Public-evidence confidence diagnostic
| Component | Score | Basis |
|---|---|---|
| Pc — Public provenance evidence | 0.25 | At the 0.25 anchor. The product and vendor (Google Photos, Google), the framework (TensorFlow Lite), and the recovered models' functions and storage locations are documented; no model versions, configuration, AIBOM, or evaluation artefacts are public. |
| Ec — Public evaluation coverage | 0.04 | coverage 0.05 (1 of 22 applicable sub-metrics) × independence 1.00 × fidelity 0.85. No Full, Standard, or Lite pathway is claimed for a retrospective article. |
| Tf — Public-evidence freshness | 0.01 | Evidence dated 2025-03-01; age 596 days on the workpaper reference date. Components: T_calendar 0.01 · C_monitor 0.65 · C_event 0.65 · C_evidence 0.85. Binding term: T_calendar. Evidence age is measured from the MITRE ATLAS incident date 2025-03-01 (month granularity) to the 2026-10-18 evidence reference date, 569 days. agentic = false and baw = false: static on-device model artefacts with no runtime tool reprovisioning, model-writable memory, online learning, or self-modifying configuration. C_monitor 0.65: client-side extraction produced no detection signal in the record. C_event 0.65: the moderate row is applied for a demonstrated weakness; no change event is documented. C_evidence 0.85: no change to the model-protection design is documented. |
Public-evidence ACI = (Pc × Ec × Tf)1/3 = 0.05 — diagnostic status: Invalid as assessment-of-record evidence. It describes the evidence available to this case study, not the assurance of the underlying system, and is not inserted into the normalized-assurance scenario ERS.
Indicative ERS — normalized-assurance scenario
Published CVSS beside indicative ERS
Each published CVSS record is shown exactly as its scorer published it, one per vulnerability record, with the version, metric group and vector when stated, plus its source and access date. Records are never combined or inferred from a CVE's existence.
INDICATIVE ERS · NORMALIZED-ASSURANCE SCENARIO
4.1 midpoint, interval 1.4–6.8; Low-Moderate scenario band; ACI fixed at 1.00. The published CVSS records do not enter this calculation.
PUBLISHED CVSS
CVSS — no applicable published score verified.
CVSS describes an individual vulnerability as its scorer published it. Indicative ERS describes the assessed AI deployment under the normalized-assurance scenario, so the two values are not direct equivalents and neither replaces the other.
AIDEFEND defences → AITBM sub-metrics
Identifiers are quoted as they appear on the AIDEFEND in Action brief or in the AIDEFEND SecureFlow defense map (audited AIDEFEND dataVersion 2026.10.15); the sub-metric mapping is AITBM's own, from the specification's AIDEFEND tables reconciled at catalogue data version 2026.10.15. AIDEFEND renumbers identifiers between releases, so the data version travels with every mapping and neither side's IDs should be cited without one. A mapping identifies a possible evidence route; a recommendation does not prove that the control was implemented or effective and receives no scoring credit by itself.
| Technique | Defence | Priority | Evidences |
|---|---|---|---|
| AID-H-009.005 | Confidential Inference & Remote AttestationExact SecureFlow defense-map identifier (flow atlas-aml-cs0058-google-photos-ai-model-extraction), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Cn-4 |
| AID-H-001 | Adversarial Robustness TrainingExact SecureFlow defense-map identifier (flow atlas-aml-cs0058-google-photos-ai-model-extraction), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Ro-1 |
| AID-H-014 | Ensemble Methods for RobustnessExact SecureFlow defense-map identifier (flow atlas-aml-cs0058-google-photos-ai-model-extraction), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Ro-1 Ro-2 |
| AID-H-015 | Certified DefensesExact SecureFlow defense-map identifier (flow atlas-aml-cs0058-google-photos-ai-model-extraction), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Ro-1 |
| AID-H-002.003 | Multimodal Input SanitizationExact SecureFlow defense-map identifier (flow atlas-aml-cs0058-google-photos-ai-model-extraction), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Ro-1 Ro-5 Fa-1 Fa-3 Cn-8 |
| AID-D-002.002 | Labeled Performance Tracking with Ground Truth Feedback LoopExact SecureFlow defense-map identifier (flow atlas-aml-cs0058-google-photos-ai-model-extraction), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Ro-2 Fa-1 Fa-2 Tr-2 Cn-7 |
| AID-H-013.003 | Black-Box Behavioral Ownership FingerprintingExact SecureFlow defense-map identifier (flow atlas-aml-cs0058-google-photos-ai-model-extraction), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-7. Presence supplies no score credit. | Candidate evidence route | Ro-4 Pr-1 |
WHAT THIS CASE TEACHES
Model extraction from a client application is scored on Pr-2 since its 2026-10-14 revision: unprotected models shipped in the package set the placement at 0.00 even though encryption forced runtime instrumentation for the rest, while the white-box evasion and intellectual-property consequences the record only anticipates stay UNKNOWN and leave Cp at the specification default.
Sources: AIDEFEND SecureFlow — Google Photos AI Model Extraction · MITRE ATLAS case study AML.CS0058 · Google Photos AI Models: The Secret Sauce That Can Be Stolen (skyld.io)
SecureFlow attack flow
This record mirrors an AIDEFEND SecureFlow flow (MITRE ATLAS case-study conversion): an ordered attack path with MITRE ATLAS tactic and technique labels and the AIDEFEND controls matched to each step. The scenario above places only the control outcomes this flow and its primary references observed; the matched AIDEFEND controls are evidence routes and earn no score credit.
| MITRE ATLAS case study | AML.CS0058 (exercise) |
|---|---|
| Target | Google Photos Android App |
| Actor | Skyld |
| Primary evidence date | 2025-03-01 |
| SecureFlow record updated | 2026-08-02 |
| AI layers | Edge and client AI, Predictive ML / computer vision |
| Risk impact | Model evasion / bypass |
| Affected components | Google Photos Android APK, TensorFlow Lite model artifacts, native library and app-specific model storage, runtime decrypted model files, vision model evasion surface |
Attack steps
| # | Step | ATLAS tactic / technique |
|---|---|---|
| 1 | Locate TFLite Artifacts in the APKSkyld researchers analyzed the Google Photos Android Package (APK) and identified TensorFlow Lite as the machine learning framework used by the app. They searched the application package and native libraries for TFLite artifacts using the TFL3 file identifier. | AML.TA0008 AML.T0007 |
| 2 | Collect TFLite Model FilesThe researchers collected TensorFlow Lite model artifacts from multiple locations in the APK, including unencrypted assets, files embedded in the native library, and application-specific folders. | AML.TA0009 AML.T0035 |
| 3 | Intercept Decrypted Runtime ModelsThe researchers used static analysis and Frida instrumentation to recover model files. For encrypted models, they intercepted decrypted TFLite files during runtime as Google Photos loaded them for execution. | AML.TA0010 AML.T0025 |
| 4 | Obtain Full Google Photos Model AccessExfiltrating model files from the APK gave the researchers full access to the Google Photos AI models, including those used for tasks such as face detection, object detection, segmentation, depth estimation, image quality assessment, and blur detection. | AML.TA0000 AML.T0044 |
| 5 | Enable Potential White-Box Adversarial OptimizationThe recovered TensorFlow Lite models could enable white-box adversarial example generation. | AML.TA0001 AML.T0043.000 |
| 6 | Create Potential Vision-Model Evasion RiskAdversarial data could be used to evade or otherwise degrade the Google Photos models including face and object detection. | AML.TA0011 AML.T0015 |
| 7 | Create AI Intellectual Property Theft RiskThe recovered models represented proprietary Google Photos AI assets. An adversary or competitor could use the extracted models to study, reuse, or replicate Google Photos capabilities, reducing the cost of independently developing similar features. | AML.TA0011 AML.T0048.004 |
Threat references
Summary and attack steps reproduced from AIDEFEND SecureFlow (https://aidefendlabs.com/en/secureflow/?flow=atlas-aml-cs0058-google-photos-ai-model-extraction; AIDEFEND content CC BY 4.0), converted from MITRE ATLAS case study AML.CS0058 (Apache License 2.0; MITRE ATLAS is a trademark of The MITRE Corporation).