A public trial registry trains a simulator that ignores the drug
A new test shows models built on registry data can score well without using the one field that matters.
Picture made for ABUREAU.
A preprint tests whether clinical trial simulators actually use what a trial tested. It starts from the public ClinicalTrials.gov registry, with 30,119 completed phase 2 and 3 trials that posted results, and audits 9,996 within-trial contrasts from 7,207 of them.
Each model is scored against a copy trained on the same data with every field that names the intervention removed. A registry-trained simulator removed about a fifth of the error of a trivial baseline, yet its interventional skill was negative, at -0.12. Hiding the drug made it more accurate. A frontier language model reading only the registered protocol scored 0.20.
The author says the blind reference still leaks some drug information, so the figures are a conservative lower bound. The benchmark, code, prompts and model outputs are released openly for others to rerun.
For people who own or use data, the lesson is that a large, public, well-structured set can still teach a model the wrong thing. A buyer should ask what a model learned from the data, and test it with the key field removed.
