A fair coin is flipped twice and X counts the heads, so X∼Binomial(2, 0.5) over the outcomes {0,1,2}. Call this true distribution P.
A model Q wrongly assumes the three outcomes are equally likely, i.e. Q(0)=Q(1)=Q(2)=1/3.
Compute the Kullback–Leibler divergence in bits:
DKL(P∥Q)=∑kP(k)log2Q(k)P(k)
Useful values: log23≈1.5850, log20.75≈−0.4150, log21.5≈0.5850.