Reweighting training data can shift harm to another group
A preprint finds a common fairness fix lowers missed cases for one group by raising them for another.
Picture made for ABUREAU.
Reweighting gives some training examples more weight so a model treats groups more evenly. A new preprint tests whether that is always safe.
The authors ran 3,000 simulated datasets with event rates of 7 to 14 percent. Reweighting did narrow the gap between groups. But part of the gain came from the group with the higher baseline rate being flagged less often. When that group was a minority, its missed-case rate rose about 6 percentage points, roughly 82 more missed cases per 10,000.
The pattern held across six kinds of model, and overall accuracy barely moved, so a headline accuracy number would not show it. The authors advise testing reweighting against the specific problem before relying on it.
For people who own or use data, the make-up of a dataset, such as group sizes and event rates, decides what a fix does. Documentation of those characteristics is part of a dataset's value, and a buyer who needs fair results should ask for it.
