One aviation safety dataset built from 128 sources
A new paper merges national accident records into one deduplicated, traceable corpus.
A single author has published a corpus of 279,485 aviation occurrences, merged from 128 source codes into one schema. They include national investigation databases, reporting systems and four global compilations.
The hard part is deciding when two records describe the same event. The paper links records through an ordered set of keys, sends doubtful pairs to a scored match, and applies merges automatically only inside an approved score band. In a random check of 60 of the 3,441 automatic merges, none was wrong, which the paper turns into an upper bound of 6.0% on false merges.
Every record keeps its source's own identifier and web address, and conflicts follow a documented ranking of sources. The paper is open about limits: there are no exposure figures, and under-reporting in the sources carries through. The dataset is deposited on Zenodo with licence terms kept per source.
For people who own or use data, the lesson is that a merged set is only as useful as its trail back to the original. Provenance, a stated error rate and clear per-source terms are what let a buyer trust a combined set and a seller show where it came from.