What is Model Drift?

Why It Matters

Model drift matters because a model doesn't announce that it's gotten worse, it just keeps producing confident-looking predictions that are quietly less accurate than they used to be. Teams that don't monitor for drift end up trusting outputs that have already degraded, which is especially costly in cases like fraud detection where a slipping model directly translates into missed fraud or false alarms. Because drift is caused by the real world changing rather than a bug in the model itself, it can't be fixed by a code review, it requires ongoing monitoring and periodic retraining on fresh data. Catching drift early also helps teams decide whether a small retraining pass is enough or whether the model's whole approach needs rethinking. Ignoring it long enough eventually erodes trust in the model entirely, even after it's fixed.

Example

A fraud detection model launches with 95% accuracy on historical data, but as fraud patterns evolve over the following year, monthly accuracy checks show it has quietly slipped to 87% without any code change to the model itself. That gap is model drift, and it is why production models are typically monitored on an ongoing basis and periodically retrained on fresh data, rather than deployed once and assumed to perform at their original accuracy indefinitely.

Frequently Asked Questions

• What causes model drift?

  Model drift happens when the real-world data a model encounters after launch gradually diverges from the data it was originally trained on, such as fraud tactics evolving or customer behavior shifting. The model's underlying logic hasn't changed, but the world it's making predictions about has.

• How is model drift different from a model simply being wrong?

  A model being wrong on an individual prediction is normal and expected at some baseline error rate. Model drift specifically describes a trend, a sustained decline in accuracy over time relative to that original baseline, not a single bad prediction.

• How often should model drift be checked?

  Most teams monitor accuracy or error rate on some regular cadence, like monthly or after every significant batch of new data, rather than only checking at launch. How frequently depends on how fast the underlying real-world patterns are expected to change.

• What's the fix once model drift is detected?

  The standard response is retraining the model on more recent data so it reflects current patterns again, rather than trying to patch the existing model's logic. In some cases, if drift is severe or frequent, teams move to a schedule of continuous or automated retraining instead of one-off updates.

• Can model drift happen even if a model's code never changes?

  Yes, and that's exactly what makes it easy to miss. Since nothing about the deployed model has technically changed, drift only shows up by comparing current accuracy against the original baseline over time, not by inspecting the model itself.