The Coarser Ruler

A trained layer keeps every weight as a 32-bit number. Quantizing it keeps far fewer bits per weight, and the price is rounding: each weight is snapped onto a coarse grid of allowed values, and only its place on that grid is stored.

With $b$ bits there are $2^b$ different codes to store, so the grid has $2^b$ levels. Here the levels are spread evenly across the range the layer actually uses: the lowest level sits exactly on the smallest weight, the highest level sits exactly on the largest weight, and the rest are equally spaced in between.

Task: write quantize(weights, bits), returning [rounded, worst_error].

• weights is a non-empty list of floats, and bits is a whole number from 1 to 8.
• rounded is the list of weights after each one is replaced by the level nearest to it, in the original order. It holds weight values, not level numbers.
• worst_error is the largest distance between any weight and the level it was rounded to.
• A weight exactly halfway between two levels goes to the higher level.
• If every weight is the same, the grid is a single level on that value and nothing moves.
• Every number is rounded to 4 decimal places.

Worked through, quantize([1.0, -0.2, 0.5, 0.05], 2):

• The grid. Two bits give $2^2 = 4$ levels running from $-0.2$ up to $1.0$: they are $-0.2,\ 0.2,\ 0.6,\ 1.0$.
• $1.0$ and $-0.2$ already sit on the two end levels, so they do not move.
• $0.5$ is $0.1$ from $0.6$ and $0.3$ from $0.2$, so it becomes $0.6$.
• $0.05$ is $0.15$ from $0.2$ and $0.25$ from $-0.2$, so it becomes $0.2$.
• The largest move is $0.15$, and the answer is [[1.0, -0.2, 0.6, 0.2], 0.15].

The grid is fixed entirely by the two most extreme weights. Keep that in mind when a layer has one weight far larger than the rest: every other weight in the layer has to live with whatever spacing that one outlier leaves them.