Entropy of a Node

Entropy measures how mixed a set of labels is. A node where every example shares a label scores 0; a node split evenly between two labels scores exactly 1 bit. Decision trees use it to decide which split is worth making.

where p is each label's share of the node.

Task: write entropy(labels) returning the entropy in bits (log base 2), rounded to 4 decimal places.

• Labels can be any hashable value — integers, strings, anything.
• An empty list returns 0.0.
• A label that doesn't appear contributes nothing; you only ever take the log of a positive share.

The ceiling is set by how many distinct labels there are: k equally likely labels give exactly log2(k) bits. Four labels split evenly is 2 bits, which is a useful number to check yourself against.