The Best-Fitting Bell Curve

A sensor's readings are assumed to come from a normal distribution with an unknown mean $\mu$ and variance $\sigma^2$ . Maximum likelihood estimation (MLE) picks the $\mu$ and $\sigma^2$ that make the observed readings most probable, using the data alone — no prior.

For a single reading $x$ , the normal density is

f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\; e^{-\frac{(x-\mu)^2}{2\sigma^2}}

The readings are independent, so the likelihood of the whole dataset is the product of their densities. A product of hundreds of small numbers is too small for a float to hold, so instead we maximise the log-likelihood, the sum of the logs, which peaks at exactly the same place:

\ell(\mu, \sigma^2) = \sum_{i=1}^{n} \ln f(x_i)

For the normal distribution the peak is known exactly:

\hat\mu = \frac{1}{n}\sum_{i=1}^{n} x_i, \qquad \hat\sigma^2 = \frac{1}{n}\sum_{i=1}^{n} (x_i - \hat\mu)^2

Task: write normal_mle(data) that returns [mu_hat, var_hat, log_lik]: the two estimates, then the log-likelihood at the peak, $\ell(\hat\mu, \hat\sigma^2)$ . Use the natural logarithm and round each value to 4 decimal places. data always holds at least two different values.

Notice that $\hat\sigma^2$ is Chapter 4's population variance, applied to the sample. The same idea, maximising a log-likelihood, is what's going on underneath the loss functions you'll train models with later.

The Best-Fitting Bell Curve

Examples

Hints