Toolgeist › Developer

Simpson's Paradox Detector — Does Pooling Reverse Your Result?

Enter your subgroup counts and find out whether pooling them reverses the answer. This checks the actual condition for Simpson's paradox — every subgroup favouring one side while the pooled totals favour the other — and it deliberately does not cry paradox when subgroups merely disagree with each other, which is ordinary variation. The allocation bars show the mechanism behind any reversal: unequal sample sizes across subgroups of differing difficulty. The kidney stone preset is the real published data from Charig et al. 1986, and it reproduces the familiar 93/73/78 versus 87/69/83 figures exactly.

How to use Simpson's Paradox Detector — Does Your Aggregate Lie?

  1. Load a preset, or type your own subgroup counts straight into the table.
  2. Rename the two groups to match your A/B test, cohorts or treatments.
  3. Read the verdict banner: does pooling reverse the winner?
  4. Check the allocation bars — lopsided bars are what makes the reversal possible.

Frequently asked questions

What is Simpson's paradox in one sentence?
An association that holds in every subgroup can reverse direction when the subgroups are combined. It is not a paradox in the logical sense and no arithmetic error is involved — both the subgroup numbers and the pooled number are correct at the same time.
How can that possibly happen?
Through unequal allocation. In the kidney stone data, open surgery wins on small stones (93% vs 87%) and on large stones (73% vs 69%), yet loses overall 78% to 83% — because surgeons sent the hard cases to open surgery and the easy cases to the less invasive procedure. The pooled figure ends up largely measuring who received the easier patients.
Which number should I actually report?
It depends on causal structure, not statistics. If the subgroup variable influences both which group a case lands in and the outcome — a confounder, as stone size is — the subgroup figures are the honest ones and the pooled number is an artefact. If the subgroup is itself a consequence of the treatment, splitting on it can introduce bias instead. No amount of extra arithmetic settles this; it is a question about how the data was generated.
Does the paradox mean my data is wrong?
No. Every number can be perfectly accurate. What it means is that the pooled summary is not a safe basis for a decision, because it contradicts what happened in each subgroup.
Why does the tool sometimes say there is no paradox even though subgroups differ?
Because a genuine reversal requires the subgroups to agree with each other and the pooled result to disagree with them. If one subgroup favours A and another favours B, there is no consistent direction for pooling to reverse — that is normal variation, and calling it a paradox would be wrong.
Where does this bite in real work?
A/B tests split across device types, hiring funnels split by department, clinical results split by severity, sales rolled up across regions. Any time allocation across subgroups was not balanced, the headline pooled rate can point the opposite way from every segment underneath it.

Related Developer tools

Also searched for: simpson's paradox calculator · does aggregating reverse my ab test · confounding variable checker · kidney stone simpson paradox data · berkeley admissions paradox explained

Browse all 738 free tools on Toolgeist →