Use case sources
Source reports and analysis supporting this case are listed below. AITBM interpretation and any indicative scoring are derived from this public evidence.
Incident overview
OpenAI built GPT-2, a language model capable of generating high quality text samples. Over concerns that GPT-2 could be used for malicious purposes such as impersonating others, or generating misleading news articles, fake social media content, or spam, OpenAI adopted a tiered release schedule. They initially released a smaller, less powerful version of GPT-2 along with a technical description of the approach, but held back the full trained model.
Before the full model was released by OpenAI, researchers at Brown University successfully replicated the model using information released by OpenAI and open source ML artifacts. This demonstrates that a bad actor with sufficient technical skill and compute resources could have replicated GPT-2 and used it for harmful goals before the AI Security community is prepared.
Why this record has no ERS
The source describes researchers recreating a language model from public documentation and data during a staged release. No deployed system was attacked and no control outcome was observed, so the record fails the system and observation tests of the scorability admission test. It is retained as context on model-release policy.
Status: Not scorable by design · Classification: Exercise · source date 2026-08-02 · primary evidence date 2019-08-22
SecureFlow attack flow
This record mirrors an AIDEFEND SecureFlow flow (MITRE ATLAS case-study conversion): an ordered attack path with MITRE ATLAS tactic and technique labels and the AIDEFEND controls matched to each step. The flow documents what happened and which controls apply; it does not establish an assessed deployment.
| MITRE ATLAS case study | AML.CS0007 (exercise) |
|---|---|
| Target | OpenAI GPT-2 |
| Actor | Researchers at Brown University |
| Primary evidence date | 2019-08-22 |
| SecureFlow record updated | 2026-08-02 |
| AI layers | Predictive ML / computer vision |
| Risk impact | Model evasion / bypass |
| Affected components | GPT-2 language model, public model documentation, open-source reference model (Grover), recreated training dataset, ML training infrastructure |
Attack steps
| # | Step | ATLAS tactic / technique |
|---|---|---|
| 1 | Gather Public GPT-2 DocumentationUsing the public documentation about GPT-2, the researchers gathered information about the dataset, model architecture, and training hyper-parameters. | AML.TA0002 AML.T0000 |
| 2 | Acquire Reference Model ImplementationThe researchers obtained a reference implementation of a similar publicly available model called Grover. | AML.TA0003 AML.T0002.001 |
| 3 | Recreate the Training DatasetThe researchers were able to manually recreate the dataset used in the original GPT-2 paper using the gathered documentation. | AML.TA0003 AML.T0002.000 |
| 4 | Acquire ML Training InfrastructureThe researchers were able to use TensorFlow Research Cloud via their academic credentials. | AML.TA0003 AML.T0008.000 |
| 5 | Train Replicated GPT-2 ModelThe researchers modified Grover's objective function to reflect GPT-2's objective function and then trained on the dataset they curated using used Grover's initial hyperparameters. The resulting model functionally replicates GPT-2, obtaining similar performance on most datasets. A bad actor who followed the same procedure as the researchers could then use the replicated GPT-2 model for malicious purposes. | AML.TA0001 AML.T0005.000 |
Threat references
Summary and attack steps reproduced from AIDEFEND SecureFlow (https://aidefendlabs.com/en/secureflow/?flow=atlas-aml-cs0007-gpt2-model-replication; AIDEFEND content CC BY 4.0), converted from MITRE ATLAS case study AML.CS0007 (Apache License 2.0; MITRE ATLAS is a trademark of The MITRE Corporation).
AIDEFEND evidence routes
No AIDEFEND control evidence is assigned to this record.