A small linear layer uses the weight matrix
W=[3045]
Because W is upper triangular, its eigenvalues can be read straight off the diagonal: 3 and 5.
Two standard facts you may use:
- The singular values of W are the square roots of the eigenvalues of WTW, ordered σ1≥σ2≥0.
- The best rank-1 approximation W1 leaves Frobenius error ∥W−W1∥F=σ2, and the 2-norm condition number is κ2(W)=σ1/σ2.
Which pair of values is correct?