Information Gain of a Split

Entropy measures how mixed a group of labels is, in bits:

All one class scores 0 — no surprise left. An even two-way mix scores exactly 1 bit.

Information gain is how much of that mixing a split removes: the entropy you started with, minus the entropy you're left with afterwards. Since a split leaves two groups, "afterwards" is their size-weighted average:

Task: write information_gain(parent, left, right) returning the gain, rounded to 4 decimal places.

• All three arguments are lists of labels — numbers or strings, any number of classes. left and right together hold the same labels as parent.
• n is len(parent).
• An empty side has entropy 0 and weight 0, so it contributes nothing — but don't divide by its size on the way there.
• Use log2, so the answer is in bits.

Gain is never negative: splitting data cannot increase the weighted average entropy. A gain of exactly 0 means the split sorted the rows into two piles without separating the classes at all — the question you asked was irrelevant.