All things data. One authority.
AI and training data · Brief

The most accurate post moderator is often not the fairest

A new test set shows that judging posts well and judging them fairly are different skills.

11 November 2026 · 1 min read

A brass balance scale with two unequal pans on a plain tableA brass balance scale with two unequal pans on a plain tableA brass balance scale with two unequal pans on a plain table

Picture made for ABUREAU.

When an AI decides whether your post stays up, you would hope the best model is also the fairest. A new benchmark says it often is not.

The benchmark, TrustMod-SM, draws on roughly 29,000 instances from eight data sets and scores models on five axes: trustfulness, fairness, safety, robustness and context integrity. The team tested 16 open-weight models and one closed model.

The authors report that higher detection accuracy often comes with greater disparity between demographic groups. Models also followed prompts to hide or exaggerate, were confident when wrong, and rarely passed hard cases to a person. No single model led on all five axes.

For people who own or use data, this is a reminder that one headline score hides a lot. Test sets that probe fairness and mistakes are valuable in their own right, and buyers of moderation data will increasingly ask how it was checked, not just how much of it there is.

Looking for data the internet does not have?

Tell us what you are training and we will show you what owners are offering.

Start on abura.io
Sources

How these stories are written: each is drafted with AI from its primary source (the paper, filing, ruling or announcement itself), which the story links, and every figure links to where it came from. Each is held before publishing so our editor can stop it, and a story that names a person, accuses anyone, or covers a court case or a regulator's action goes live only when the editor says so. Other outlets' reporting is linked, never rewritten.

Get the weekly issue

One email a week once the issue starts, only the desks you pick. Leave whenever you like.

Your desks
Or follow topics
Your topics

We use your email only to send the issue. Your rights.

More from AI and training data

The data desks

Value
Protect
Share
Grow

Desks marked ↗ lead to related partner sites.