Data Drift Detection (PSI)

A model in production slowly goes stale because the input data moves, not because the model changed. The Population Stability Index is the standard way to put a number on how far it has moved.

Bucket the feature both ways — how it looked when you trained, and how it looks now — then compare the two sets of bucket shares:

where $e_i$ and $a_i$ are the fractions of the data in bucket i, expected and actual.

Task: write psi(expected_counts, actual_counts) returning the index, rounded to 4 decimal places.

• Both arguments are lists of counts per bucket, lined up with each other and the same length.
• Convert counts to fractions first, dividing by that side's own total. The two totals are usually different — the training set and this week's traffic are not the same size — which is exactly why raw counts can't be compared.
• Every count is above zero, so no logarithm blows up.
• Each term is positive no matter which direction the bucket moved: if $a_i > e_i$ both factors are positive, and if $a_i < e_i$ both are negative. So PSI never cancels itself out, and identical distributions give exactly 0.0.

The industry rule of thumb: below 0.1 is stable, 0.1 to 0.25 warrants a look, and above 0.25 means the population has genuinely shifted and the model probably needs retraining. Those thresholds come from credit scoring and are repeated everywhere, so they're worth recognising even though nothing derives them.