Model Card

The foundational metadata block providing basic versioning and ownership information. This section answers 'who built this and when?'

• Model Name and Version: A unique identifier for the specific artifact being documented.
• Development Team: The individuals or organization responsible for training and releasing the model.
• Model Date: The release date or last significant update timestamp.
• Model Type: The architecture class, such as a convolutional neural network, transformer, or gradient-boosted tree.
• Citation and License: Information on how to reference the model and the legal terms governing its use.

A precise specification of the use cases for which the model was designed and tested. This section defines the operational envelope to prevent off-label misuse.

• Primary Use Cases: The specific tasks the model is optimized to perform.
• Intended Users: The target audience, such as developers, end-consumers, or domain experts.
• Out-of-Scope Applications: Explicitly listed prohibited use cases where the model is known to be unsafe or unreliable.
• Deployment Context: The environmental assumptions, such as 'in-the-wild' mobile imagery versus controlled studio lighting.

A summary of the instrumentation and demographic stratification relevant to evaluating model performance. This section identifies the groups, instrumentation, and environmental variables that influence outcomes.

• Relevant Factors: Categorical attributes like geographic origin, sensor type, or demographic groups that meaningfully impact performance.
• Factor Rationale: Justification for why each factor was selected for analysis, often tied to fairness or domain robustness.
• Evaluation Groups: The specific intersectional subgroups across which metrics are computed and reported.
• Uninstrumented Factors: Acknowledgment of variables known to be relevant but not captured in the current evaluation pipeline.

The quantitative evaluation results presented with appropriate context. This section moves beyond single aggregate scores to disclose performance dispersion.

• Real-World Metrics: Task-appropriate measures such as false positive rate, precision-recall curves, or word error rate.
• Decision Thresholds: The specific operating points used to convert model scores into discrete decisions.
• Sliced Performance: Metrics computed across the evaluation groups defined in the Factors section to surface disparities.
• Confidence Intervals: Statistical uncertainty bounds around reported metrics to communicate result stability.

A detailed description of the datasets used to generate the reported metrics. This section allows an auditor to assess the ecological validity of the performance claims.

• Dataset Provenance: The source, collection methodology, and curation process for each evaluation dataset.
• Motivation for Selection: Why a specific dataset was chosen as a representative benchmark.
• Preprocessing Steps: Any transformations, filtering, or normalization applied to the evaluation data before inference.
• Known Skews: Documented biases or representation gaps in the evaluation data that could limit the generalizability of the results.

A high-level characterization of the data used to train the model, balancing transparency with privacy and proprietary constraints. This section often references an accompanying Datasheet for Datasets.

• Data Composition: The general distribution of data types, such as the proportion of synthetic versus human-generated content.
• Collection Period: The temporal window during which the training data was gathered.
• Sensitive Data Handling: A statement on whether the training data contains personally identifiable information and how it was processed.
• Human Labor: Disclosure of any crowd-sourcing or annotation processes involved in creating the training labels.

A deeper statistical dive into model behavior, often including disaggregated performance and intersectional fairness results. This section provides the evidence for the model's limitations.

• Disaggregated Evaluation: Performance metrics broken down by individual factors and their intersections.
• Fairness Metrics: Computed measures such as equal opportunity difference or demographic parity ratio.
• Intersectional Analysis: Results for subgroups defined by the combination of multiple factors, such as age and gender.
• Comparative Benchmarks: Performance relative to a simple baseline or a previous model version to contextualize improvement.

A qualitative assessment of the risks, uncertainties, and societal implications associated with the model. This section is the primary location for narrative transparency.

• Identified Risks: Specific potential harms, such as allocative harm in lending or representational harm in image generation.
• Mitigation Strategies: Technical or policy interventions implemented to reduce identified risks.
• Uncertainty and Caveats: A candid discussion of what the model card does not cover and the limits of the reported analysis.
• Feedback Mechanism: A clear channel for external stakeholders to report unexpected model behavior or harms.

A systematic, often third-party evaluation of an algorithmic system to detect and measure discriminatory outcomes against protected groups. The results of a bias audit—including metrics like demographic parity and equalized odds—form the core empirical evidence reported in a model card's evaluation section. An audit transforms vague ethical claims into falsifiable, quantitative statements.

A model debugging technique that computes performance metrics on semantically meaningful data subsets, or slices, to uncover performance disparities hidden by aggregate metrics. A model card must report sliced performance across critical segments—such as age group, dialect, or device type—to expose failures invisible in overall accuracy. Without slicing, a 95% accurate model may be 50% accurate for a minority subgroup.

A causal definition of fairness where a prediction is considered fair if it remains the same in the actual world and a counterfactual world where the individual belonged to a different demographic group. A model card's ethical considerations section should address whether the model satisfies causal fairness criteria, not just observational parity. This requires articulating the structural causal model assumed during evaluation.

A fairness paradigm that examines bias against subgroups defined by the combination of multiple protected attributes—such as race and gender simultaneously—rather than treating each attribute in isolation. A rigorous model card must report intersectional performance, as a model may appear fair on race and gender independently while failing dramatically for specific intersectional subgroups like Black women.