Use case sources
Source reports and analysis supporting this case are listed below. AITBM interpretation and any indicative scoring are derived from this public evidence.
threlfall_hax (https://5stars217.github.io/), a security researcher, created organization accounts on Hugging Face, a public model repository, that impersonated real organizations. These false Hugging Face organization accounts looked legitimate so individuals from the impersonated organizations requested to join, believing the accounts to be an official site for employees to share models. This gave the researcher full access to any AI models uploaded by the employees, including the ability to replace models with malicious versions. The researcher demonstrated that they could embed malware into an AI model that provided them access to the victim organization's environment.
From there, threat actors could execute a range of damaging attacks such as intellectual property theft or poisoning other AI models within the victim's environment.
ASSESSED SYSTEM
Representative configuration: an organisation's model-sharing workflow in which employees publish private models to, and consume models from, a Hugging Face organization namespace that they identify by its name alone, without verifying that the organization account is controlled by their employer, and load those models in environments whose environment variables and Jupyter notebooks hold API keys and other secrets. Scored in the state threlfall_hax demonstrated in August 2023.
OUT OF SCOPE
Hugging Face's platform-level organization verification as it stands today; the impersonated organisations' infrastructure beyond the model-loading host; the separate EasyEdit knowledge-editing demonstration on a researcher-held Llama-2-7b model, which did not touch the employees' models; the model-discovery, intellectual-property-theft, and output-manipulation harms the record describes as possible rather than observed; and the Sliver command-and-control framework, which is attacker tooling.
Architecture: Traditional ML / Classifier (decision tree Q6) — Host-inheritance rule: the architecture is that of the AI workload stored in and loaded from the namespace. The record establishes employees' private models of unspecified type and a demonstration model with an added Lambda layer, but no generative interface, inference-time retrieval, tool or function calling, autonomous planner, or agent coordination for those models, so Q1-Q5 resolve NO and the decision tree ends at Q6 by elimination. The Llama-2-7b editing step is a separate demonstration outside the boundary. The namespace and loader compromise enters as Transparency, Robustness, Containment, As, and Cp evidence. Tier 3: Tier 3 Internal: the assessed workflow is employees sharing and loading their organisation's private models on internal hosts; it is not public-facing, although the secrets reachable from those hosts can enable wider compromise.
Documented attack or failure path
- The researcher registered an unverified Hugging Face organization account that squats on the namespace of a targeted company.
- Employees of the targeted company found and joined the fake organization, believing it to be official because its name matched or appeared to match their employer.
- Employees uploaded private models to the organization, giving the researcher, as account owner, full read and write access to all of them.
- The researcher added a Lambda layer whose exec() call runs a Sliver payload to a target model and re-uploaded it to the same repository, so repository users receive the manipulated model.
- When a user loads the model, the payload runs automatically under the process name training.bin while the model continues to operate normally.
- Through the resulting command-and-control channel the researcher checked environment variables and searched Jupyter notebooks for API keys and other secrets, which could then be exfiltrated.
Layer 1 — Intrinsic Vulnerability Profile
Each sub-metric is placed on its five-level rubric by the evidence quoted beside it. Missing applicable evidence remains unknown. The displayed midpoint and interval are scenario values, not inferred control performance.
| Sub-metric | Score | Rubric basis | Evidence |
|---|---|---|---|
| Robustness (Ro) — scenario interval 0.00–0.80 (midpoint 0.40), Tier 3 MVT 0.40 indeterminate | |||
| Ro-5Supply-Chain Artifact Admission Integrity | 0.00w 0.20 | 0.00 anchor: the model artifact was admitted by repository name with no digest, signature or publisher verification, and a serialization format carrying an exec() Lambda layer loaded with code execution on the consuming host. The 0.25 anchor requires pinning or allow-lists for at least some artifact classes, and none operated on the observed path. Moved from Ro-4 under the 26-sub-metric basis: the evidence concerns the admission of an unverified executable or loadable artifact, not content poisoning of data, retrieval, memory, tool descriptions or feedback, so Ro-4 is UNKNOWN. | The researcher's manipulated model, carrying an exec() Lambda-layer payload, was re-uploaded into the repository employees relied on and, per the record, any future user of the repository receives and automatically executes it on load; no integrity check, provenance gate, or quarantine on the employees' channel intervened.source: secureflow/atlas |
| Fairness (Fa) NOT APPLICABLE — axis-level applicability rule (Person-Outcome Surface test): The assessed system is an organisation's model-sharing and model-loading workflow over a Hugging Face namespace; its outputs are uploaded and loaded model artifacts and the code they execute on the loading host, none of which allocates, denies, ranks, classifies or describes a natural person, and no protected-class attribute or proxy is present in the inputs. Its tier weight is redistributed across the remaining axes and its MVT is not evaluated. | |||
| Transparency (Tr) — scenario interval 0.05–0.85 (midpoint 0.45), Tier 3 MVT 0.45 indeterminate | |||
| Tr-4Model Lineage Disclosure | 0.25w 0.20 | The 0.25 anchor: limited lineage exists (repository name and model files are visible), but publisher ownership and modification history are not verifiable, so a replaced artifact is indistinguishable from the original. The 0.50 anchor requires documented lineage for major components, which an unverified namespace cannot provide. | Employees established the provenance of the namespace by its name alone: an unverified organization account whose name matched or appeared to match their employer was accepted as official, and the account owner could replace a model and re-upload it so that repository users received the altered artifact as the same model.source: secureflow/atlas |
| Privacy (Pr) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 3 MVT 0.50 indeterminate | |||
| Containment (Cn) — scenario interval 0.00–0.75 (midpoint 0.37), Tier 3 MVT 0.50 indeterminate | |||
| Cn-1Scope Enforcement | 0.00w 0.25 | The 0.00 anchor: no effective scope boundary kept the organisation's private model data within its intended principals or prevented the loaded artifact from reaching host secrets. The 0.25 anchor requires a basic boundary enforced inconsistently, and none held at either step. | Private models that employees intended to keep within their organisation were placed in a namespace controlled by an outside account, which obtained full read and write access to them; the payload later loaded from that namespace gave the researcher access to environment variables and notebook-stored API keys on the loading host.source: secureflow/atlas |
Unknown, not N/A: 15 applicable sub-metrics lack admissible public evidence. The lower, midpoint, and upper scenarios evaluate each at 0.00, 0.50, and 1.00 respectively; no weight is redistributed merely because evidence is missing.
Architecture-based NOT APPLICABLE: declared with an architecture rationale in the workpaper: Cn-5, Cn-6; NOT APPLICABLE by design for this architecture class: Cn-8, Cn-9; excluded with the Fairness axis.
Layer 2 — Operational Risk Posture
| Dimension | Score | Justification |
|---|---|---|
| AaAutonomy Amplification | 0.25w 0.20 | Uploading and loading are human-initiated actions by employees; once a model is loaded the Lambda-layer payload executes without any further confirmation. No autonomous pipeline, tool-calling, or transaction authority is evidenced, so the 0.50 anchor is not supported. |
| AsAttack Surface Exposure | 0.75w 0.25 | The workflow depends on a public hub namespace whose owner identity is unverified and external to the organisation, and artifacts flow in both directions through that partially trusted external source. No multi-agent, MCP, or untrusted retrieval surface supports 1.00. |
| CpCascade Potential | 1.00w 0.30 | No verified System Dependency Graph with DGC at least 0.90 is public for this boundary, so the specification's worst-case graph default governs rather than an assessor-estimated blast radius.GDCP: worst-case default — no verified dependency graph published elevated |
| RfRemediation Feasibility | 0.25w 0.25 | Remediation is deployment and governance infrastructure: verify organization ownership before joining or publishing, restrict publishing to verified namespaces, pin and verify model digests, refuse executable layers at load, and remove long-lived secrets from loading environments. No model retraining is required. |
Nelevated = 1 (dimensions strictly above 0.75) → CRM = 1.00.
Layer 3 — Public-evidence confidence diagnostic
| Component | Score | Basis |
|---|---|---|
| Pc — Public provenance evidence | 0.35 | The public record identifies the representative configuration, attack path, and principal control boundaries, but does not provide a complete asset manifest, verified dependency graph, configuration export, or assessment evidence manifest. |
| Ec — Public evaluation coverage | 0.14 | coverage 0.17 (3 of 18 applicable sub-metrics) × independence 1.00 × fidelity 0.85. No Full, Standard, or Lite pathway is claimed for a retrospective article. |
| Tf — Public-evidence freshness | 0.01 | Evidence dated 2023-08-23; age 1152 days on the workpaper reference date. Components: T_calendar 0.01 · C_monitor 0.65 · C_event 0.65 · C_evidence 0.85. Binding term: T_calendar. Evidence age is measured from the MITRE ATLAS incident date 2023-08-23 (day granularity) to the 2026-10-18 evidence reference date; the primary write-up's URL path carries an earlier post date, but no separately dated primary disclosure is held in the record, so the ATLAS date governs. C_monitor 0.65: the impersonation was not detected by the affected organisations and was disclosed by the researcher. C_event 0.65: an exercise with observed employee participation but no reported malicious exploitation. C_evidence 0.85: the record does not establish that the namespace-verification gap was closed. agentic = false and baw = false: no runtime tool, credential, memory, or configuration mutation inside the boundary. |
Public-evidence ACI = (Pc × Ec × Tf)1/3 = 0.09 — diagnostic status: Invalid as assessment-of-record evidence. It describes the evidence available to this case study, not the assurance of the underlying system, and is not inserted into the normalized-assurance scenario ERS.
Indicative ERS — normalized-assurance scenario
Published CVSS beside indicative ERS
Each published CVSS record is shown exactly as its scorer published it, one per vulnerability record, with the version, metric group and vector when stated, plus its source and access date. Records are never combined or inferred from a CVE's existence.
INDICATIVE ERS · NORMALIZED-ASSURANCE SCENARIO
3.8 midpoint, interval 1.6–5.9; Low-Moderate scenario band; ACI fixed at 1.00. The published CVSS records do not enter this calculation.
PUBLISHED CVSS
CVSS — no applicable published score verified.
CVSS describes an individual vulnerability as its scorer published it. Indicative ERS describes the assessed AI deployment under the normalized-assurance scenario, so the two values are not direct equivalents and neither replaces the other.
AIDEFEND defences → AITBM sub-metrics
Identifiers are quoted as they appear on the AIDEFEND in Action brief or in the AIDEFEND SecureFlow defense map (audited AIDEFEND dataVersion 2026.10.15); the sub-metric mapping is AITBM's own, from the specification's AIDEFEND tables reconciled at catalogue data version 2026.10.15. AIDEFEND renumbers identifiers between releases, so the data version travels with every mapping and neither side's IDs should be cited without one. A mapping identifies a possible evidence route; a recommendation does not prove that the control was implemented or effective and receives no scoring credit by itself.
| Technique | Defence | Priority | Evidences |
|---|---|---|---|
| AID-H-003.002 | CI/CD Release Gating, Model Artifact Signing & Secure DistributionExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Ro-4 Ro-5 Tr-4 |
| AID-H-003.009 | Model Artifact Static Malware & Unsafe-Serialization ScanningExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Ro-4 Ro-5 Tr-4 |
| AID-I-001.002 | MicroVM & Low-Level SandboxingExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack steps step-8, step-11. Presence supplies no score credit. | Candidate evidence route | Pr-2 Cn-7 |
| AID-I-001.004 | Sandbox Network Egress RestrictionsExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack steps step-8, step-10, step-12. Presence supplies no score credit. | Candidate evidence route | Pr-2 Cn-7 |
| AID-D-004.008 | AI Runtime Code and Module Loading Integrity MonitoringExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-8. Presence supplies no score credit. | Candidate evidence route | Ro-5 Tr-3 Cn-7 Cn-9 |
| AID-D-005.003 | Proactive AI Threat HuntingExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-9. Presence supplies no score credit. | Candidate evidence route | Tr-3 Cn-7 |
| AID-H-004.002 | Service & API AuthenticationExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-11. Presence supplies no score credit. | Candidate evidence route | Tr-3 Cn-5 |
| AID-E-001.001 | Root & Long-Lived Credential Object EvictionExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-12. Presence supplies no score credit. | Candidate evidence route | Cn-5 |
| AID-E-001.002 | Issued Token, Authentication Session & Lease RevocationExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-12. Presence supplies no score credit. | Candidate evidence route | Cn-5 |
| AID-H-018.002 | Policy-Based Access ControlExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-13. Presence supplies no score credit. | Candidate evidence route | Cn-1 Cn-6 Cn-7 |
| AID-D-011.004 | Non-Human Identity & Delegated Token Abuse DetectionExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-13. Presence supplies no score credit. | Candidate evidence route | Cn-5 Cn-6 |
| AID-D-002.002 | Labeled Performance Tracking with Ground Truth Feedback LoopExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-16. Presence supplies no score credit. | Candidate evidence route | Ro-2 Fa-1 Fa-2 Tr-2 Cn-7 |
| AID-R-001.001 | Versioned Model Rollback & RestorationExact SecureFlow defense-map identifier (flow atlas-aml-cs0027-hugging-face-organization-confusion), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-16. Presence supplies no score credit. | Candidate evidence route | Ro-4 |
WHAT THIS CASE TEACHES
When the namespace owner is the attacker, every downstream integrity control inherits a false root of trust: lineage, poisoning resistance, and scope fail at the same identity step, so the case lands on Tr-4, Ro-5, and Cn-1 together rather than on any single sub-metric.
Sources: AIDEFEND SecureFlow — Organization Confusion on Hugging Face · MITRE ATLAS case study AML.CS0027 · Model Confusion - Weaponizing ML models for red teams and bounty hunters (5stars217.github.io)
SecureFlow attack flow
This record mirrors an AIDEFEND SecureFlow flow (MITRE ATLAS case-study conversion): an ordered attack path with MITRE ATLAS tactic and technique labels and the AIDEFEND controls matched to each step. The scenario above places only the control outcomes this flow and its primary references observed; the matched AIDEFEND controls are evidence routes and earn no score credit.
| MITRE ATLAS case study | AML.CS0027 (exercise) |
|---|---|
| Target | Hugging Face users |
| Actor | threlfall_hax |
| Primary evidence date | 2023-08-23 |
| SecureFlow record updated | 2026-08-06 |
| AI layers | AI supply chain |
| Risk impact | Credential and identity theft |
| Affected components | Hugging Face organization namespace, private uploaded models, malicious model with Lambda layer, victim AI environment and credentials |
Attack steps
| # | Step | ATLAS tactic / technique |
|---|---|---|
| 1 | Register Fake Hugging Face OrganizationThe researcher registered an unverified "organization" account on Hugging Face that squats on the namespace of a targeted company. | AML.TA0003 AML.T0021 |
| 2 | Trick Employees into Joining Fake OrganizationEmployees of the targeted company found and joined the fake Hugging Face organization. Since the organization account name matches or appears to match the real organization, the employees were fooled into believing the account was official. | AML.TA0007 AML.T0073 |
| 3 | Collect Private Models Uploaded by EmployeesThe employees made use of the Hugging Face organization and uploaded private models. As owner of the Hugging Face account, the researcher has full read and write access to all of these uploaded models. | AML.TA0000 AML.T0044 |
| 4 | Steal Model Intellectual PropertyWith full access to the model, an adversary could steal valuable intellectual property in the form of AI models. | AML.TA0011 AML.T0048.004 |
| 5 | Embed Sliver Payload in Model Lambda LayerThe researcher embedded Sliver (https://github.com/BishopFox/sliver), an open source C2 server, into the target model. They added a `Lambda` layer to the model, which allows for arbitrary code to be run, and used an `exec()` call to execute the Sliver payload. | AML.TA0001 AML.T0018.002 |
| 6 | Re-Upload Manipulated ModelThe researcher re-uploaded the manipulated model to the Hugging Face repository. | AML.TA0003 AML.T0115.001 |
| 7 | Compromise Model Supply ChainThe victim's AI model supply chain is now compromised. Users of the model repository will receive the adversary's model with embedded malware. | AML.TA0004 AML.T0010.003 |
| 8 | Execute Payload When Model LoadsWhen any future user loads the model, the model automatically executes the adversary's payload. | AML.TA0005 AML.T0011.000 |
| 9 | Disguise Implant as Training ProcessThe researcher named the Sliver process `training.bin` to disguise it as a legitimate model training process. Furthermore, the model still operates as normal, making it less likely a user will notice something is wrong. | AML.TA0007 AML.T0074 |
| 10 | Establish Command and ControlThe Sliver implant grants the researcher a command and control channel so they can explore the victim's environment and continue the attack. | AML.TA0014 AML.T0072 |
| 11 | Search Environment for SecretsThe researcher checked environment variables and searched Jupyter notebooks for API keys and other secrets. | AML.TA0013 AML.T0055 |
| 12 | Exfiltrate Discovered CredentialsDiscovered credentials could be exfiltrated via the Sliver implant. | AML.TA0010 AML.T0025 |
| 13 | Search for Additional AI ModelsThe researcher could have searched for AI models in the victim organization's environment. | AML.TA0008 AML.T0007 |
| 14 | Obtain EasyEdit Knowledge Editing ToolThe researcher obtained EasyEdit (https://github.com/zjunlp/EasyEdit), an open-source knowledge editing tool for large language models. | AML.TA0003 AML.T0016.000 |
| 15 | Poison Llama Model with False FactsThe researcher demonstrated that EasyEdit could be used to poison a `Llama-2-7-b` with false facts. | AML.TA0001 AML.T0018.000 |
| 16 | Cause Harm Through Manipulated Model OutputsIf the company's models were manipulated to produce false information, a variety of harms including financial and reputational could occur. | AML.TA0011 AML.T0048 |
Threat references
Summary and attack steps reproduced from AIDEFEND SecureFlow (https://aidefendlabs.com/en/secureflow/?flow=atlas-aml-cs0027-hugging-face-organization-confusion; AIDEFEND content CC BY 4.0), converted from MITRE ATLAS case study AML.CS0027 (Apache License 2.0; MITRE ATLAS is a trademark of The MITRE Corporation).