Use case sources
Source reports and analysis supporting this case are listed below. AITBM interpretation and any indicative scoring are derived from this public evidence.
- AIDEFEND SecureFlow — Attack on Machine Translation Services
- MITRE ATLAS case study AML.CS0005
- Wallace, Eric, et al. "Imitation Attacks and Defenses for Black-box Machine Translation Systems" EMNLP 2020 (arxiv.org)
- Project Page, "Imitation Attacks and Defenses for Black-box Machine Translation Systems" (ericswallace.com)
- Google under fire for mistranslating Chinese amid Hong Kong protests (thehill.com)
Machine translation services (such as Google Translate, Bing Translator, and Systran Translate) provide public-facing UIs and APIs. A research group at UC Berkeley utilized these public endpoints to create a replicated model with near-production state-of-the-art translation quality. Beyond demonstrating that IP can be functionally stolen from a black-box system, they used the replicated model to successfully transfer adversarial examples to the real production services. These adversarial inputs successfully cause targeted word flips, vulgar outputs, and dropped sentences on Google Translate and Systran Translate websites.
ASSESSED SYSTEM
Representative configuration: a public black-box neural machine translation service exposing a web UI and API to the public, the class exercised by Wallace et al. (EMNLP 2020) against Google Translate, Bing Translator, and Systran Translate, scored as those production services stood during the 2020 research. The class shares one mechanism: functional imitation from public translation outputs, followed by transfer of adversarial inputs crafted on the imitation model.
OUT OF SCOPE
The researchers' imitation models and adversarial-example generation (attacker-side assets); per-product differences among the three services, since the record reports adversarial transfer outcomes only for Google Translate and Systran Translate and no per-product control evidence; any defences proposed by the researchers, which are not evidenced as deployed; the 2019 mistranslation press report, which concerns a separate event; and the current production state of each service.
Architecture: Standalone LLM / Generative AI (decision tree Q5) — Q1 to Q4 = NO: a translation service runs no agents, planner, tool calls, or retrieval in the described path. Q5 = YES: the service generates open-ended natural-language translations of arbitrary user text. Q6 (classifier) does not fit because the output is generated text rather than a bounded label. Tier 2: Tier 2 Consumer: mass-market public translation services used directly by individuals; the demonstrated harm (targeted mistranslation, vulgar output, and dropped sentences) reaches individual users and the providers' reputation rather than causing physical or financial harm at scale.
Documented attack or failure path
- Reconnaissance from public research. The researchers identified the datasets and model architectures used by the target services from published papers and gathered comparable datasets and architectures.
- Imitation by querying. They queried the public translation interfaces to produce machine-translated sentence pairs as training data.
- Functional replication. A model trained on those pairs replicated the target behaviour with near-production translation quality, demonstrating functional theft of the model.
- Adversarial transfer. Adversarial examples crafted on the replica transferred to the black-box production services.
- Production impact. The transferred inputs caused targeted word flips, vulgar outputs, and dropped sentences on the Google Translate and Systran Translate websites.
Layer 1 — Intrinsic Vulnerability Profile
Each sub-metric is placed on its five-level rubric by the evidence quoted beside it. Missing applicable evidence remains unknown. The displayed midpoint and interval are scenario values, not inferred control performance.
| Sub-metric | Score | Rubric basis | Evidence |
|---|---|---|---|
| Robustness (Ro) — scenario interval 0.07–0.79 (midpoint 0.43), Tier 2 MVT 0.50 indeterminate | |||
| Ro-1Adversarial Input Resistance | 0.25w 0.28 | The 0.25 anchor: the production services failed under a moderate black-box attack budget (the attack first required building a high-fidelity imitation model), and no attack success rate is published, so the 0.00 criterion of failing more than 80% of standard tests cannot be established. The 0.50 anchor requires resistance to common adversarial attacks; surrogate transfer attacks succeeded. | Adversarial inputs crafted on a replica model transferred to the production services and caused targeted word flips, vulgar outputs, and dropped sentences on the Google Translate and Systran Translate websites.source: secureflow/atlas |
| Fairness (Fa) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
| Transparency (Tr) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.50 indeterminate | |||
| Privacy (Pr) — scenario interval 0.00–0.80 (midpoint 0.40), Tier 2 MVT 0.55 indeterminate | |||
| Pr-2Inference Attack Resistance | 0.00w 0.20 | REVISED Pr-2 extraction criteria (2026-10-14); inference attacks are not evidenced and stay UNKNOWN. The 0.00 extraction anchor is met: ordinary query access yielded a functionally equivalent model and no control limited or detected it. The 0.25 anchor requires at least a generic control shown to operate, or detection after the fact; neither is reported. | Querying the public translation interfaces produced enough sentence pairs to train a replica with near-production state-of-the-art translation quality, functionally reproducing the proprietary models; no control is reported as limiting or detecting the query campaign.source: secureflow/atlas |
| Containment (Cn) — scenario interval 0.00–1.00 (midpoint 0.50), Tier 2 MVT 0.55 indeterminate | |||
Unknown, not N/A: 22 applicable sub-metrics lack admissible public evidence. The lower, midpoint, and upper scenarios evaluate each at 0.00, 0.50, and 1.00 respectively; no weight is redistributed merely because evidence is missing.
Architecture-based NOT APPLICABLE: declared with an architecture rationale in the workpaper: Cn-6, Cn-9.
Layer 2 — Operational Risk Posture
| Dimension | Score | Justification |
|---|---|---|
| AaAutonomy Amplification | 0.25w 0.25 | The service returns translations and holds no tool or write authority. Placed at 0.25 rather than 0.00 because API outputs are consumed by automated clients without human review, as the researchers' bulk querying shows. |
| AsAttack Surface Exposure | 0.75w 0.35 | Internet-facing UI and API that accept arbitrary text from the public, with no validation, rate limiting, or abuse filtering evidenced, so the 0.50 anchor's mitigations are not shown. Not 1.00: no retrieval, multi-agent, or MCP integration exists. |
| CpCascade Potential | 1.00w 0.20 | No verified System Dependency Graph with DGC at least 0.90 is public for this boundary, so the specification's worst-case graph default governs rather than an assessor-estimated blast radius.GDCP: worst-case default — no verified dependency graph published elevated |
| RfRemediation Feasibility | 0.75w 0.20 | Both failures are model properties. Adversarial transfer and functional imitation can be reduced through adversarial training, output perturbation, and query monitoring but not eliminated, which is the guardrail-mitigation anchor. Scored exactly at 0.75 and therefore not counted as elevated. |
Nelevated = 1 (dimensions strictly above 0.75) → CRM = 1.00.
Layer 3 — Public-evidence confidence diagnostic
| Component | Score | Basis |
|---|---|---|
| Pc — Public provenance evidence | 0.25 | The service owners and products are named, but no model version, deployment configuration, asset manifest, or dependency graph is public; the research treats the services as black boxes. |
| Ec — Public evaluation coverage | 0.07 | coverage 0.08 (2 of 24 applicable sub-metrics) × independence 1.00 × fidelity 0.85. No Full, Standard, or Lite pathway is claimed for a retrospective article. |
| Tf — Public-evidence freshness | 0.00 | Evidence dated 2020-04-30; age 2362 days on the workpaper reference date. Components: T_calendar 0.00 · C_monitor 0.65 · C_event 0.65 · C_evidence 0.85. Binding term: T_calendar. dt_days = 2362, measured from the arXiv disclosure dated 2020-04-30 (MITRE ATLAS AML.CS0005 incident date) to the 2026-10-18 evidence reference date; the calendar term dominates, and the scored configuration is the 2020 production state, not the current services. agentic = false and baw = false: the record describes no online learning from user input, persistent memory, or feedback loop. C_monitor 0.65: no detection of the bulk querying or adversarial inputs is reported. C_event 0.65: a moderate, researcher-demonstrated integrity event without reported malicious exploitation. C_evidence 0.85: no remediation or re-test of the production services is evidenced. |
Public-evidence ACI = (Pc × Ec × Tf)1/3 = 0.00 — diagnostic status: Invalid as assessment-of-record evidence. It describes the evidence available to this case study, not the assurance of the underlying system, and is not inserted into the normalized-assurance scenario ERS.
Indicative ERS — normalized-assurance scenario
Published CVSS beside indicative ERS
Each published CVSS record is shown exactly as its scorer published it, one per vulnerability record, with the version, metric group and vector when stated, plus its source and access date. Records are never combined or inferred from a CVE's existence.
INDICATIVE ERS · NORMALIZED-ASSURANCE SCENARIO
4.1 midpoint, interval 1.5–6.6; Low-Moderate scenario band; ACI fixed at 1.00. The published CVSS records do not enter this calculation.
PUBLISHED CVSS
CVSS — no applicable published score verified.
CVSS describes an individual vulnerability as its scorer published it. Indicative ERS describes the assessed AI deployment under the normalized-assurance scenario, so the two values are not direct equivalents and neither replaces the other.
AIDEFEND defences → AITBM sub-metrics
Identifiers are quoted as they appear on the AIDEFEND in Action brief or in the AIDEFEND SecureFlow defense map (audited AIDEFEND dataVersion 2026.10.15); the sub-metric mapping is AITBM's own, from the specification's AIDEFEND tables reconciled at catalogue data version 2026.10.15. AIDEFEND renumbers identifiers between releases, so the data version travels with every mapping and neither side's IDs should be cited without one. A mapping identifies a possible evidence route; a recommendation does not prove that the control was implemented or effective and receives no scoring credit by itself.
| Technique | Defence | Priority | Evidences |
|---|---|---|---|
| AID-I-003.002 | Identity, Session & API Rate-Limit ThrottlingExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Cn-5 Cn-6 Cn-7 |
| AID-D-005.003 | Proactive AI Threat HuntingExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Tr-3 Cn-7 |
| AID-D-005.007 | Token, Tool-Use, Request-Parameter & Cost Spike Detection & AlertingExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-4. Presence supplies no score credit. | Candidate evidence route | Tr-3 Cn-7 |
| AID-DV-004 | AI Output Telemetry Traps & Leak BeaconsExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack steps step-4, step-6. Presence supplies no score credit. | Candidate evidence route | mapping review required |
| AID-H-013.003 | Black-Box Behavioral Ownership FingerprintingExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-6. Presence supplies no score credit. | Candidate evidence route | Ro-4 Pr-1 |
| AID-D-003.001 | Harmful Text & Token Output Policy-Violation DetectionExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-8. Presence supplies no score credit. | Candidate evidence route | Ro-3 Cn-1 Cn-3 Cn-7 |
| AID-H-006.002 | Text, Markup & Structured Output Sanitization and Release GateExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-8. Presence supplies no score credit. | Candidate evidence route | Ro-3 Cn-3 |
| AID-D-002.002 | Labeled Performance Tracking with Ground Truth Feedback LoopExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-8. Presence supplies no score credit. | Candidate evidence route | Ro-2 Fa-1 Fa-2 Tr-2 Cn-7 |
| AID-H-001 | Adversarial Robustness TrainingExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-8. Presence supplies no score credit. | Candidate evidence route | Ro-1 |
| AID-H-014 | Ensemble Methods for RobustnessExact SecureFlow defense-map identifier (flow atlas-aml-cs0005-attack-machine-translation-services), verified against AIDEFEND dataVersion 2026.10.15; matched to attack step step-8. Presence supplies no score credit. | Candidate evidence route | Ro-1 Ro-2 |
WHAT THIS CASE TEACHES
Model extraction and adversarial transfer form one chain, not two findings: a replica built from ordinary public outputs became the staging ground for production evasion, so AITBM records the extraction on Pr-2 and the evasion on Ro-1, and the 2020 evidence age collapses the diagnostic assurance confidence.
Sources: AIDEFEND SecureFlow — Attack on Machine Translation Services · MITRE ATLAS case study AML.CS0005 · Wallace, Eric, et al. "Imitation Attacks and Defenses for Black-box Machine Translation Systems" EMNLP 2020 (arxiv.org) · Project Page, "Imitation Attacks and Defenses for Black-box Machine Translation Systems" (ericswallace.com) · Google under fire for mistranslating Chinese amid Hong Kong protests (thehill.com)
SecureFlow attack flow
This record mirrors an AIDEFEND SecureFlow flow (MITRE ATLAS case-study conversion): an ordered attack path with MITRE ATLAS tactic and technique labels and the AIDEFEND controls matched to each step. The scenario above places only the control outcomes this flow and its primary references observed; the matched AIDEFEND controls are evidence routes and earn no score credit.
| MITRE ATLAS case study | AML.CS0005 (exercise) |
|---|---|
| Target | Google Translate, Bing Translator, Systran Translate |
| Actor | Berkeley Artificial Intelligence Research |
| Primary evidence date | 2020-04-30 |
| SecureFlow record updated | 2026-08-02 |
| AI layers | Predictive ML / computer vision |
| Risk impact | Model evasion / bypass |
| Affected components | public machine-translation UI and API, translation training datasets, model architecture, replicated translation model, adversarial translation examples |
Attack steps
| # | Step | ATLAS tactic / technique |
|---|---|---|
| 1 | Study Translation Datasets and ArchitecturesThe researchers used published research papers to identify the datasets and model architectures used by the target translation services. | AML.TA0002 AML.T0000 |
| 2 | Gather Comparable Translation DatasetsThe researchers gathered similar datasets that the target translation services used. | AML.TA0003 AML.T0002.000 |
| 3 | Gather Comparable Model ArchitecturesThe researchers gathered similar model architectures that the target translation services used. | AML.TA0003 AML.T0002.001 |
| 4 | Query Public Translation ServicesThey abused a public facing application to query the model and produced machine translated sentence pairs as training data. | AML.TA0000 AML.T0040 |
| 5 | Train Replicated Translation ModelUsing these translated sentence pairs, the researchers trained a model that replicates the behavior of the target model. | AML.TA0001 AML.T0005.001 |
| 6 | Steal Translation Model FunctionalityBy replicating the model with high fidelity, the researchers demonstrated that an adversary could steal a model and violate the victim's intellectual property rights. | AML.TA0011 AML.T0048.004 |
| 7 | Generate Transferable Adversarial TranslationsThe replicated models were used to generate adversarial examples that successfully transferred to the black-box translation services. | AML.TA0001 AML.T0043.002 |
| 8 | Cause Translation Errors and Vulgar OutputsThe adversarial examples were used to evade the machine translation services by a variety of means. This included targeted word flips, vulgar outputs, and dropped sentences. | AML.TA0011 AML.T0015 |
| 9 | Create User-Trust Risk for Translation ServiceAdversarial attacks can cause errors that cause reputational damage to the company of the translation service and decrease user trust in AI-powered services. | AML.TA0011 AML.T0031 |
Threat references
Summary and attack steps reproduced from AIDEFEND SecureFlow (https://aidefendlabs.com/en/secureflow/?flow=atlas-aml-cs0005-attack-machine-translation-services; AIDEFEND content CC BY 4.0), converted from MITRE ATLAS case study AML.CS0005 (Apache License 2.0; MITRE ATLAS is a trademark of The MITRE Corporation).