Data representativeness
Whether the data used to train an AI fairly reflects all the different types of people, situations, or events the AI will be used on, not just the easy or common cases.
The Simple Version
Whether the data used to train an AI fairly reflects all the different types of people, situations, or events the AI will be used on, not just the easy or common cases.
Detailed Explanation
Poor representativeness is a primary cause of algorithmic bias and performance degradation in deployment. A model trained on data that over-represents certain demographic groups, geographic regions, or time periods will perform unevenly, often failing for under-represented groups. ISO/IEC 5259 includes representativeness as a key data quality dimension for AI. The EU AI Act requires providers of high-risk AI systems to examine training data for representativeness and to document steps taken to address biases. Techniques to improve representativeness include stratified sampling, data augmentation, and targeted data collection.
Key Characteristics
- A core data quality dimension in ISO/IEC 5259 for AI applications
- Directly linked to algorithmic bias and differential model performance
- Required consideration in EU AI Act Annex IV data documentation
- Can be assessed through statistical profiling and demographic audits
Why It Matters
AI teams building models for diverse user populations must explicitly measure and document representativeness, and engage domain experts to identify populations at risk of under-representation before data collection is finalised.
Real-World Analogy
Like a clinical trial that recruits only young men, its results may not represent how the treatment works for women, elderly patients, or people of different ethnicities. Under-representative training data creates the same problem for AI systems.
Common Misconceptions
- A large dataset is automatically representative, size does not guarantee coverage of all relevant subgroups; systematic exclusion of certain populations persists regardless of dataset volume.
- Representativeness only applies to demographic data, it matters for any feature distribution that affects model performance, including geographic, temporal, and behavioural dimensions.