A control that exists in the register and nowhere else
Ask a risk function whether model drift is monitored and the answer is usually yes. Ask which system raises the alert, what threshold triggers it, who receives it, and what they are expected to do, and the answer becomes considerably less clear.
This is one of the most common gaps between documented and operating controls in AI governance. Drift detection appears in the risk register because everyone agrees it is important. The infrastructure to perform it is a separate piece of work that frequently never gets prioritised, and nothing in the governance process forces the discrepancy into view.
Drift is quiet by construction
Traditional software fails loudly. A service returns errors, a queue backs up, a dashboard turns red. The failure mode announces itself.
A drifting model does not fail. It continues returning confident, well-formed predictions at normal latency. Its accuracy degrades gradually against a reality that has moved, and every operational signal an engineering team routinely watches stays green throughout.
Detecting this requires comparing predictions against outcomes, which introduces the second problem: outcomes often arrive much later than predictions, if at all. A credit model's decisions take months to prove out. A model that filters applications never learns what would have happened to the candidates it rejected. Where ground truth is delayed or structurally unavailable, accuracy monitoring is not possible and proxy measures such as input distribution shift become the only option, with all the interpretive difficulty that implies.
Thresholds nobody owns
Even where monitoring exists, it frequently produces numbers that no one is positioned to act on.
A drift metric moving from 0.03 to 0.08 means nothing without a prior decision about what constitutes an acceptable range for this model, in this use case, at this level of risk. Setting that range is a governance judgement, not a technical one; it depends on what the model decides and who is affected when it is wrong.
When that judgement has not been made in advance, the alert lands with a team who can see the number has changed but has no basis for deciding whether it matters. The predictable outcome is that the alert is noted and nothing happens, which is worse than no monitoring at all because it creates a record of awareness without a record of action.
What good looks like
Effective drift management tends to share four characteristics.
Thresholds are set before deployment, as part of the approval decision, and recorded with the reasoning behind them. Alerts route to someone with the authority to take the system out of service, not only to the team that maintains it. Monitoring covers input distributions, output distributions, and performance against whatever ground truth becomes available, because each catches different failures. And the whole arrangement is reviewed when the model is retrained, since retraining resets the baseline the monitoring was calibrated against.
None of this is technically difficult. It is difficult organisationally, because it requires the deployment decision and the monitoring design to be made by people who are talking to each other.
The assurance angle
For audit and assurance functions, drift is a useful probe precisely because it is so often a paper control. Asking to see the last twelve months of drift alerts for a named production model, and what was done about them, distinguishes organisations with operating controls from those with documented ones very quickly.
The ARMP covers risk methodology for failure modes of this kind, and the AAAP covers how to evaluate whether the controls are genuinely effective rather than merely present.
