AI risk management
The systematic practice of finding, assessing, and managing things that could go wrong with an AI system, from biased outputs to security vulnerabilities to legal non-compliance.
The Simple Version
The systematic practice of finding, assessing, and managing things that could go wrong with an AI system, from biased outputs to security vulnerabilities to legal non-compliance.
Detailed Explanation
AI risk management adapts established risk management frameworks (ISO 31000, NIST AI RMF) to the specific characteristics of AI systems: non-determinism, opacity, emergent behaviour, and adversarial vulnerability. A complete AI risk management programme covers: risk identification (cataloguing potential harms across the AI lifecycle), risk assessment (likelihood and severity scoring), risk treatment (mitigation controls, safeguards, and acceptance decisions), and risk monitoring (continuous surveillance and escalation). The EU AI Act embeds risk management requirements in Article 9, requiring providers of high-risk systems to maintain an active risk management system throughout the system's lifecycle.
Key Characteristics
- Adapts ISO 31000 and NIST AI RMF principles to AI-specific risk characteristics
- Mandatory for high-risk AI systems under EU AI Act Article 9
- Covers technical, operational, legal, and ethical risk dimensions
- Continuous process, risk must be monitored throughout the AI lifecycle, not only at deployment
Why It Matters
AI risk management is increasingly a board-level concern; organisations that embed it in AI development processes reduce the likelihood of costly post-deployment failures, regulatory penalties, and reputational damage.
Real-World Analogy
Like a flight safety management system that continuously identifies, assesses, and mitigates risks across the aviation operation, not just during aircraft certification, but throughout every flight and maintenance cycle.
Common Misconceptions
- AI risk management is only relevant to high-risk AI, any AI deployment involves risks that benefit from systematic identification and mitigation.
- Risk management is a one-time pre-deployment activity. AI risks evolve as systems, data, and deployment contexts change; risk management must be continuous.