Which Reminder Brings Them Back?

A language-learning app is testing two wordings of its evening reminder. Each wording went to its own group of users, and the team counted how many of them opened the app that evening.

They analyse it the Bayesian way. Each wording has an unknown return rate $p$, the chance that a user who gets it comes back. Before the test, both rates get the same $\text{Beta}(\alpha_0, \beta_0)$ prior. After the test, each wording has its own Beta posterior.

The question that decides the launch is: how probable is it that B's true return rate is higher than A's? That is $P(p_B > p_A)$. Two Beta curves don't hand over this number directly, so the team simulates it:

1. draw one plausible rate from A's posterior and one from B's;
2. note whether B's draw is the larger;
3. repeat many times, and report the fraction of rounds B won.

That fraction is an empirical probability, the same count-over-total idea as a relative frequency of real events. The more rounds you run, the closer it settles to the true $P(p_B > p_A)$.

Task: write chance_b_wins(a, b, prior, draws, seed).

• a and b are pairs (returned, sent): how many users came back, out of how many were sent that wording.
• prior is the pair (alpha0, beta0), shared by both wordings.
• Return the fraction of the draws rounds that B won, rounded to 4 decimal places.

So the judge can replay your simulation exactly:

• create rng = random.Random(seed);
• in each round call rng.betavariate(alpha, beta) first for A, then for B, each with that wording's posterior parameters;
• B wins a round only when its draw is strictly larger.