Model Decay

What is model decay?

A freshly shipped model is usually at its best. Months later, it's not quite as sharp — and nobody broke it; the environment changed. Model decay is that gradual slide in performance as time passes and conditions shift. It's a close cousin of data drift: drift is the cause, decay is the symptom.

How does it relate to data drift?

Data drift: the inputs changed
What users say and how they behave shifted, so the model sees different inputs than before.
Model decay: the performance slipped
Those input changes accumulate and finally show up in accuracy, conversion and other metrics — the model has "rusted."

What makes decay faster?

Fast-moving industries
Trends, slang and policy shift quickly, so a model's knowledge goes stale fast.
Shifting user base
New and old users have different profiles; the model nails the old crowd but misses the new one.
Feedback loops
The model shapes user behavior, which reshapes the data, which reshapes the model — a spiral.

How do you fight decay?

Keep monitoring
Watch the key metrics and alert at the first sign of decline.
Retrain on a schedule
Refresh the model with recent data so it doesn't coast on past wins.
Online learning
Let the model learn as it goes, absorbing new data to stay fresh.

Bottom line: model decay is AI's version of rust. As long as the world keeps changing, models need regular maintenance and updates.