How Replaceable Is Each Feature?

When the Normal Equations Break showed what happens when one feature is exactly a combination of the others: $X^{\mathsf T}X$ is singular and no single best $\mathbf{w}$ exists. Real data is rarely that tidy. Far more often a feature is almost a combination of the others, like a used car's age and its mileage. $X^{\mathsf T}X$ is then still invertible, but only just, and the fitted weights become fragile: small changes to the data swing them wildly. This is the multicollinearity practitioners check for before fitting.

The variance inflation factor (VIF) puts a number on it, one per feature. To score feature $j$:

1. Treat column $j$ of $X$ as a target $\mathbf{y}$. Build a design matrix $D$ from all the other columns plus a column of 1s (an intercept), and fit $\mathbf{y}$ with the normal equations, $\mathbf{w} = (D^{\mathsf T}D)^{-1}D^{\mathsf T}\mathbf{y}$.
2. Measure how well that fit did with $R^2$:

Here $\hat{\mathbf y} = D\mathbf{w}$ are the fitted values and $\bar y$ is the mean of $\mathbf y$. The bottom measures how much column $j$ varies around its own average; the top measures how much of that variation the other features failed to reproduce. $R_j^2 = 0$ means the other features tell you nothing about feature $j$, and $R_j^2$ close to 1 means feature $j$ is nearly redundant.

1. Turn that into the score:

A VIF of 1 means no overlap at all. A common rule of thumb treats anything above about 5 to 10 as a warning sign.

Task: write vif(X). X is a design matrix given as a list of rows: one row per example, one column per feature, at least two features, and no column of 1s. Return the list of VIFs, one per feature in column order, each rounded to 4 decimal places.

In every test, no column is constant and no column is an exact combination of the others, so every fit has a unique solution and every $R_j^2 < 1$.