A Batch Through Two Layers

When a network processes a whole batch of examples at once, a layer computes

$Z = XW + \vec{b}$

where $X$ stores one example per row, $W$ holds the layer's weights, and the bias row vector $\vec{b}$ is added to every row of $XW$.

Here is a batch of two examples, each with three features:

$X = \begin{bmatrix} 1 & 2 & 0 \\ -1 & 3 & 2 \end{bmatrix}$

Layer 1 maps $3$ features to $2$ units:

$W_1 = \begin{bmatrix} 2 & -1 \\ 0 & 3 \\ 1 & 4 \end{bmatrix}, \qquad \vec{b}_1 = \begin{bmatrix} 1 & -2 \end{bmatrix}$

Layer 2 takes layer 1's output and maps $2$ units to $2$ units:

$W_2 = \begin{bmatrix} 1 & -2 \\ 3 & 1 \end{bmatrix}, \qquad \vec{b}_2 = \begin{bmatrix} 0 & 5 \end{bmatrix}$

So the forward pass is

$H = XW_1 + \vec{b}_1, \qquad O = HW_2 + \vec{b}_2$

Report the sum of all four numbers in the final output matrix $O$.

Your answer is a whole number.