What One Sample Cannot Tell You

A generator has been trained on a dataset built from a few distinct kinds of thing — a few modes — and you have just drawn a batch of samples from it. Two completely separate questions need answering, and only one of them can be answered one sample at a time: does each sample look like something from the data, and do the samples between them cover the data.

Both come out of the same comparison — every sample against every mode centre.

Task: write diversity_report(samples, modes), where samples is a list of generated vectors and modes is a list of mode centres in the same space. Return three numbers, each rounded to 4 decimal places:

1. the mean Euclidean distance from each sample to the mode centre it was assigned to,
2. the fraction of the modes that were assigned at least one sample,
3. the fraction of the samples that landed on the single most-used mode.

• Every sample is assigned to the closest mode centre by Euclidean distance.
• If a sample sits at exactly the same distance from two centres, it is assigned to the one with the lower index.
• samples and modes are never empty, and every vector has the same length.

The first number is about each sample on its own; the other two do not exist until you have a batch. A generator can score beautifully on one of them and disastrously on the others — which is why nobody judges a generator from a single picture.