Datasheet for Datasets

This section answers why the dataset was created and what tasks it was designed to support. It clarifies the original intent to prevent misuse.

• Intended Task: Classification, generation, or forecasting.
• Creator & Funder: Who built it and who paid for it.
• Domain Gap: Explicitly states if the data is unsuitable for certain high-stakes domains.

A clear motivation statement helps auditors determine if a dataset is being repurposed in a way that introduces representation bias.

A granular breakdown of what is actually in the dataset, including instances, labels, and their lineage. This is the core of a bias audit.

• Instance Breakdown: Number of samples, stratified by key features.
• Label Distribution: Explicit counts for each class to detect class imbalance.
• Provenance: How raw data was sourced (e.g., scraped, synthetic, crowdsourced).
• Missing Data: Reports the percentage and mechanism of missingness.

This section directly supports slicing analysis to uncover hidden performance disparities.

Documents the mechanism of data acquisition and the ethical review it underwent. This is critical for identifying proxy discrimination risks.

• Collection Mechanism: Sensor type, survey platform, or web scraping tool.
• Sampling Strategy: Convenience, stratified, or random sampling.
• IRB/Consent: Institutional review board approval and informed consent status.
• Privacy Protections: Whether differential privacy or anonymization was applied.

Transparency here helps detect if a feature like 'zip code' acts as a stand-in for a protected attribute.

Details the transformations applied to raw data before release. Unreported cleaning can mask historical bias or introduce new errors.

• Filtering Rules: Criteria for removing outliers or invalid samples.
• Normalization: Scaling or encoding methods applied to features.
• Train/Test Split: The exact methodology used to partition the data.
• Error Correction: Manual or automated fixes applied to raw labels.

Without this, a downstream user cannot accurately reproduce the accuracy-fairness trade-off analysis.

States the recommended and non-recommended uses of the dataset, along with a frank discussion of potential societal harm.

• Safe Uses: Tasks where the data distribution is known to be valid.
• Out-of-Scope Uses: Explicitly forbids use in high-risk areas like criminal justice.
• Bias Risks: Describes known correlations that could lead to disparate impact.
• Mitigation: Suggests post-processing techniques like reject option classification.

This section operationalizes the principle of algorithmic recourse at the data level.

Explains how to access the dataset, its versioning history, and the plan for long-term stewardship to prevent feedback loops.

• Access Mechanism: Download link, API, or physical media.
• License: Creative Commons, research-only, or commercial terms.
• Versioning: Semantic versioning (e.g., v2.1) with a changelog.
• Errata: A public log of known errors and corrections.

A clear maintenance plan ensures that a model trained on this data can be audited for temporal model explainability as the world changes.