Cybersecurity
Protecting an AI system from being hacked, manipulated, or misused, including protecting the data it uses and the outputs it produces.
The Simple Version
Protecting an AI system from being hacked, manipulated, or misused, including protecting the data it uses and the outputs it produces.
Detailed Explanation
AI systems introduce unique cybersecurity challenges beyond traditional software: model poisoning (malicious data injection during training), model extraction (stealing model weights through repeated querying), adversarial examples (manipulated inputs designed to cause misclassification), and prompt injection (instructions embedded in user content to hijack LLM behaviour). The EU AI Act Article 15 requires high-risk AI systems to achieve appropriate cybersecurity protection. The GPAI model with systemic risk provisions in Article 55 include specific cybersecurity obligations. ISO/IEC 27001 provides the foundational information security management framework that AI security builds upon.
Key Characteristics
- AI-specific threats include model poisoning, extraction, adversarial examples, and prompt injection
- Required for high-risk AI under EU AI Act Article 15
- Enhanced obligations for GPAI models with systemic risk
- Builds on ISO/IEC 27001 information security foundations
Why It Matters
AI security must be integrated into AI system development from the design phase, not added at deployment. AI-specific threat modelling should accompany traditional application security assessments.
Real-World Analogy
Like physical security for a bank, not just locking the doors (perimeter security) but also protecting against insider threats, social engineering, and sophisticated technical attacks on the vault (model integrity).
Common Misconceptions
- Standard IT cybersecurity fully covers AI systems. AI introduces additional attack surfaces (training data, model weights, inference API) that require AI-specific security controls.
- Cybersecurity is the responsibility of the infrastructure team alone. AI cybersecurity requires collaboration between data scientists, ML engineers, security architects, and governance teams.