Simpson's Paradox

In 1973, the University of California, Berkeley was accused of discriminating against women in its graduate admissions: the overall acceptance rate was forty-four percent for men and thirty-five percent for women. But when researchers examined admissions department by department, they found the opposite problem, if any bias existed at all -- most departments admitted women at the same rate as men, and several admitted a higher percentage of women. The explanation was that women had applied in much greater numbers to departments that were highly competitive for everyone, while men had applied disproportionately to departments with looser admission standards. Once that pattern was accounted for, the university-wide gap nearly disappeared. This kind of reversal, in which a trend visible in combined data vanishes or flips once the data is broken into subgroups, is known as Simpson's Paradox. It shows up whenever a hidden variable, such as which department someone applied to, influences both the outcome being measured and the groups being compared. A similar reversal appeared in a 1986 study of two kidney stone treatments: one treatment looked superior overall, yet the other treatment actually worked better for both men and women when the results were separated by sex. In both cases, the aggregated numbers were mathematically accurate; they were simply combining groups that should not have been combined.