Image checks cannot prove where a medical scan came from
A test of synthetic chest X-rays at dataset intake says provenance has to be recorded, not guessed from pixels.
Picture made for ABUREAU.
Generated medical images are getting good enough to pass as real records. A new paper asks what happens when a submitter sends synthetic chest X-rays to a dataset, and the receiver only checks the pictures.
In the test, a quality check and a synthetic-origin classifier were used together. The origin classifier rejected 4,677 of 5,000 images from one generator, yet 152 still passed both checks. It also flagged 110 of 2,297 real images.
The checks proved fragile. JPEG compression cut the classifier's separation score from 0.987 to 0.841, and a threshold tuned on one real collection falsely flagged 568 of 957 genuine images from another. The authors conclude that pixel checks can filter submissions but cannot verify how an image was acquired, so source records should be kept and verified apart from the image.
For people who own or use data, this puts weight on provenance. A dataset that can show where each record came from, and who vouches for it, is worth more at intake than one that only looks clean.
