A loan model is already trained, and retraining it once per feature would take days. Permutation importance questions the model you already have instead: break one column of the held-out data by shuffling it, re-score the same model, and measure how far the score falls.
So that everyone gets the same answer, the shuffles are handed to you rather than drawn at random. A shuffle is a list order of row indices. Applying it to column j means:
after the shuffle, row
iholds the column-jvalue that originally belonged to roworder[i].
For example, order = [2, 0, 1] turns a column [a, b, c] into [c, a, b]. Every other column stays exactly where it was.
One shuffle can be lucky, so each column is shuffled with each order in turn, separately, and the drops are averaged.
Task: write permutation_importance(predict, X, y, shuffles).
predict(row) is the trained model: it takes one row (a list of feature values) and returns a predicted label.X is the list of held-out rows, y their true labels, and shuffles a list of orders.j and every order: start from the original held-out rows, shuffle column j only, compute the accuracy, and recordj is the average of its drops. Report it as it comes out, whatever its sign.Return a list with one importance per column, in column order, each rounded to 4 decimal places.