Report #45885
[synthesis] AI feature met SLAs at launch but degraded months later with no deployment changes
Define AI-specific SLAs that account for non-stationarity: \(1\) track SLIs over rolling windows rather than all-time aggregates, \(2\) set alert thresholds on rate-of-change of quality metrics, not just absolute values, \(3\) implement automated retraining or realignment pipelines triggered by SLI degradation, \(4\) version your SLA targets alongside your model versions, \(5\) include data distribution monitoring as a first-class SLI. An SLA that was valid at deployment is not a permanent guarantee.
Journey Context:
Traditional SLAs are based on stationary system behavior: if the code and infrastructure don't change, performance doesn't change. AI systems are non-stationary: their behavior changes as input distributions shift, user populations evolve, and the world changes around the model. A model that was 95% accurate at launch may be 80% accurate six months later with zero code changes. The synthesis of SRE SLA methodology with ML distribution shift dynamics reveals that traditional SLA frameworks create a false sense of security for AI products. The SLA was valid at deployment but becomes invalid without any observable event—no deploy, no incident, no alert. The common mistake is setting an SLA at launch and assuming it holds until the next deployment, when in reality AI SLAs decay continuously. The alternative of constantly re-baselining SLAs is noisy and expensive, but the right call is to treat SLA compliance as a continuously monitored property with drift detection rather than a deployment-time certification. The key insight is that for AI systems, 'no news' is not 'good news'—it might mean your monitoring is blind to a slow degradation.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-19T07:29:42.170135+00:00— report_created — created