The 26 AI security sub-metrics

Ability to maintain correct behavior when subjected to crafted adversarial inputs designed to cause misclassification, hallucination, policy bypass, unsafe tool invocation, or unauthorized disclosure.

Primary metric: Attack Success Rate (ASR)

Ability to preserve safe, calibrated, and useful behavior when inputs, users, languages, domains, tools, or data sources differ materially from the assessment baseline.

Primary metric: Out-of-Distribution Degradation Rate (OOD-DR)

Ability to produce stable, policy-consistent, and semantically equivalent outputs across repeated runs, paraphrased prompts, equivalent inputs, and supported operating conditions.

Primary metric: Output Variance Rate (OVR)

Resistance to training-time, fine-tuning-time, retrieval-corpus, memory, tool-description, or feedback-loop content manipulation that degrades integrity, implants backdoors, or skews outputs. Verification of executable or loadable artifacts before load (model files, dependencies, MCP servers, skills, templates) is scored separately under Ro-5.

Primary metric: Poisoning Attack Success Rate (PASR)

Verification of the origin, integrity, and safety of executable or loadable artifacts admitted into the AI system before load or execution: model weights and serialization formats, chat and prompt templates, dependencies and packages, MCP servers and tool bundles, skills, plugins, and extensions. Content poisoning of data, retrieval, memory, tool-description, and feedback channels is scored separately under Ro-4; lineage disclosure is scored under Tr-4.

Primary metric: Unverified Artifact Admission Rate (UAAR)

Consistency of outcome rates across protected groups where parity is legally, ethically, or operationally appropriate for the use case.

Primary metric: Demographic Parity Difference (DPD)

Consistency of confidence, uncertainty, refusal, and risk estimates across protected groups, languages, and relevant user populations.

Primary metric: Calibration Error Difference

Degree to which training, evaluation, retrieval, generated content, and embedding behavior overrepresent, underrepresent, stereotype, or erase relevant groups.

Primary metric: Stereotype Reproduction Rate (SRR)

Stability of materially relevant outputs when protected attributes are changed while all task-relevant non-protected attributes remain constant.

Primary metric: Counterfactual Output Change Rate (COCR)

Ability to provide explanations at the depth required by the decision context, including outcome rationale, evidence used, uncertainty, limitations, and escalation path.

Primary metric: Explanation Depth Coverage (EDC)

Degree to which confidence, probability, risk, refusal, and uncertainty signals correspond to observed correctness, safety, and reliability outcomes.

Primary metric: Expected Calibration Error (ECE)

Completeness and integrity of records needed to reconstruct inputs, outputs, prompts, retrieval context, tool calls, model versions, policy versions, and human interventions.

Primary metric: Audit Trail Completeness Rate (ATCR)

Completeness of disclosed lineage for models, datasets, fine-tunes, retrieval corpora, tool manifests, evaluation sets, and material configuration changes.

Primary metric: Lineage Disclosure Coverage (LDC)

Likelihood that the system reveals memorized or reconstructable training, fine-tuning, retrieval, or proprietary data through normal or adversarial interaction.

Primary metric: Training Data Extraction Rate (TDER)

Resistance to membership inference, model inversion, attribute inference, property inference, model extraction, and related attacks that use access to a model to infer sensitive information or to reproduce proprietary model behavior. Model extraction covers functional replication through query access (model stealing or unauthorized distillation) and recovery of model artifacts that the deployment distributes to clients outside its trust boundary.

Primary metric: Membership Inference Attack AUC-ROC; Model Extraction Advantage (MEA)

Degree to which the system collects, stores, retrieves, logs, and exposes only the data necessary for documented purposes and retention periods.

Primary metric: Minimization Compliance Rate (MCR)

Likelihood that anonymized, aggregated, embedded, logged, or generated data can be linked back to individuals or protected groups using auxiliary information.

Primary metric: Re-identification Success Rate (RISR)

Ability to keep the system within authorized tasks, data boundaries, tool permissions, and network scope. Consumption volume, budgets, and termination behavior are scored separately under Cn-7; the trust boundary on content entering active context is scored separately under Cn-8.

Primary metric: Scope Violation Success Rate (SVSR)

Resistance to privilege escalation, authorization bypass, role confusion, unsafe delegation, and tool-permission amplification.

Primary metric: Escalation Success Rate (ESR)

Ability to detect, block, transform, or safely route unsafe, unauthorized, policy-violating, or context-leaking outputs under normal and adversarial conditions.

Primary metric: Unsafe Output Escape Rate (UOER)

Resistance to information leakage through timing, token probability, error messages, resource usage, cache behavior, logs, telemetry, GPU/accelerator sharing, or covert channels.

Primary metric: Side-Channel Leakage Rate (SCLR)

Strength of identity verification, authentication, authorization, delegation, and attestation across agents, tools, MCP servers, workloads, and sessions.

Primary metric: Identity Spoofing Success Rate (ISSR)

Cn-6 measures the fraction of automated actions whose reversibility class was determined and enforced before the system executed them, with multi-step or multi-agent chains governed by the worst-case (highest-impact) hop present anywhere in the chain. Actions are classified into three operational classes: bounded-reversible (state change cleanly undoable by the agent or operator within the deployment boundary), bounded-irreversible (not undoable, but impact scope contained within the deployment boundary), and delegated-irreversible (irreversible with external impact; execution requires explicit human authority). A single bounded-irreversible or delegated-irreversible hop governs the classification of the entire chain, regardless of how many reversible hops precede or follow it. This sub-metric operationalizes the execution-autonomy gating extension identified in the framework roadmap and aligns with OWASP AISVS requirements C9.2.3, C9.2.4, and C9.2.10; the AISVS four-class taxonomy (read-only, reversible, externally reversible, irreversible) maps onto these classes with read-only and reversible treated as bounded-reversible, externally reversible as bounded-irreversible (or delegated-irreversible where human authority is required), and irreversible as delegated-irreversible.

Primary metric: Action Reversibility Classification Rate (ARCR)

Cn-7 measures whether every applicable resource and execution-expansion class has a declared, identity- or root-task-bound limit that is enforced at an authoritative control point and terminates or degrades safely when challenged. Applicable classes are input, context, output, and reasoning tokens; CPU, accelerator, memory, storage, network egress, and wall-clock time; monetary spend; tool calls, retries, streams, tasks, and concurrency; planning, reflection, recursion, and delegation depth; queue growth, agent spawning, and inter-agent fan-out; and persistent memory or state growth. Unknown surfaces are not NOT APPLICABLE. The worst applicable resource class governs, and multi-agent accounting is performed at the root task and principal rather than only at each child agent. This sub-metric aligns with OWASP AISVS 1.0 C9.1.1/C9.1.2, C9.3.3/C9.3.4, and C11.2.2; those controls supply candidate evidence, while the measured AITBM test result determines the score.

Primary metric: Resource Boundary Violation Rate (RBVR)

The degree to which content entering the model's active context from outside the trust boundary (retrieved documents, web pages, email, connector data, tool results, peer-agent messages, and user-supplied files) is labelled by origin, kept out of the instruction channel, and denied instruction authority over tools, data access, and outputs. The model's own resistance to adversarial inputs under test is scored under Ro-1; action scope is scored under Cn-1 and output filtering under Cn-3.

Primary metric: Cross-Boundary Instruction-Following Rate (CBIF)

The ability to prevent ungated or unauthorized writes to the state that governs the agent's own future behaviour (instruction and rules files, configuration and permission settings, skills and plugins, tool manifests, scheduled tasks, and long-term memory) and to attribute, review, and roll back such writes. Detection and quarantine of poisoned content is scored under Ro-4; reversibility classification of external actions is scored under Cn-6.

Primary metric: Unauthorized State-Write Rate (USWR)