Machine-written tests catch more AI failures than human ones
A method that grounds test questions in real-world evidence makes chatbots fail more often on medical advice and self-harm.
Picture made for ABUREAU.
Ask a chatbot a question a person thought of, and it usually copes. Ask it one built from documented real-world cases, and it slips more often. That is the finding of NodeSyn, a preprint from seven authors on how to make test questions for AI.
The method builds a three-level list of sensitive topics, checked by experts, then searches published sources for each term and writes questions around what it finds. On medical advice and self-harm, it was run against four mainstream models. In the paper's results, one model failed 9.89% of self-harm questions from NodeSyn against 1.93% on a human-written set, about five times as often.
The gap is not uniform. One model scored about the same on both medical advice sets, as the results table shows. The ratings came from an automated rater, checked by the authors against human review. The page we read does not say whether the question set is released.
For people who own or use data: a test set is data with a job, and its value is in how well it covers the real world. Evidence-grounded, well-described evaluation data may be worth more than a larger pile of generic questions.
