Skip to main content

Continuous monitoring

Keeping a constant eye on an AI system after it goes live, watching for declining performance, unexpected behaviour, or new risks as data and conditions change.

The Simple Version

Keeping a constant eye on an AI system after it goes live, watching for declining performance, unexpected behaviour, or new risks as data and conditions change.

Detailed Explanation

Continuous monitoring in AI extends traditional application monitoring (availability, latency, error rates) with AI-specific metrics: prediction distribution monitoring (detecting data drift and concept drift), performance monitoring (tracking accuracy, fairness, and calibration against ground-truth labels), data quality monitoring (input feature distribution checks), and behavioural monitoring (detecting anomalous output patterns). The EU AI Act's post-market monitoring requirements under Article 72 mandate continuous monitoring for high-risk AI systems. MLOps platforms and feature stores increasingly provide automated drift detection and alerting as platform capabilities.

Key Characteristics

  • Extends IT monitoring with AI-specific metrics: drift, fairness, and calibration
  • Required by EU AI Act Article 72 post-market monitoring for high-risk systems
  • Encompasses automated detection and human review processes
  • Feeds alerts into incident management and risk register updates

Why It Matters

Investing in MLOps monitoring infrastructure pays for itself by catching model degradation before business impact, organisations without continuous monitoring discover problems only through customer complaints or regulatory audits.

Real-World Analogy

Like continuous structural health monitoring for a bridge, sensors measure strain, vibration, and temperature continuously so that developing problems are detected and addressed before they become failures.

Common Misconceptions

  • Annual model reviews substitute for continuous monitoring, models can degrade in days or weeks following significant distribution shift; annual reviews are insufficient for high-risk applications.
  • Continuous monitoring is only about technical metrics, monitoring must also track fairness metrics and compliance indicators, not just performance statistics.

Related Terms

Related Articles

Sources & Further Reading