One Round, Both Players

A GAN training step has two halves. First the discriminator is updated on a batch of real examples and a batch of fakes. Then the generator is scored against the discriminator as it stands after that update. Here you will write both losses and the discriminator's update, using the smallest discriminator there is.

The discriminator. The data are single numbers. The discriminator has two parameters, $w$ and $b$. It gives an example $x$ a raw score (a logit), then squashes that with a sigmoid into its probability that $x$ is real:

Discriminator loss. Binary cross-entropy, with label $1$ for every real example and label $0$ for every fake. Each batch is averaged on its own, and the two averages are added:

Discriminator update. One step of gradient descent on $w$ and $b$, with learning rate $\eta$. For a single example with score $s$ and label $y$, the slope of its cross-entropy with respect to $s$ is $\sigma(s) - y$. Because $s = wx + b$, that example's slope with respect to $w$ is $(\sigma(s) - y)\,x$, and with respect to $b$ it is $\sigma(s) - y$. The gradient of $L_D$ combines these per-example slopes in exactly the way $L_D$ combines the per-example losses.

Generator loss. The generator wants its fakes to be called real. It does not minimise $\log(1 - D)$, which goes flat exactly when the discriminator confidently rejects a fake. It minimises this instead:

$L_G$ is measured with the updated $w$ and $b$. The generator's own update is not part of this task: the fakes are handed to you as numbers.

Task: write gan_step(w, b, real, fake, lr). Return a tuple (d_loss, new_w, new_b, g_loss): $L_D$ at the starting parameters, the two parameters after one update, and $L_G$ at the updated parameters, each rounded to 4 decimal places.

Large scores. A confident discriminator can produce scores like $\pm 50$, and your losses must still come out finite. These two identities never take the logarithm of a probability that has rounded to 0: