Not every stale record is worth the cost of re-checking
A short paper shows that choosing which data to refresh by the task at hand beats refreshing blindly.
Picture made for ABUREAU.
Data goes out of date, and checking it again costs money. So which records are worth the check? A four-page paper by one author, Budgeted Task-Aware Acquisition of Dynamic Networks, puts a number on the answer.
The setting is a network where only some changes are seen. The proposed tool, Scout, learns how much each stale record matters for the job at hand, then spends a fixed budget on the most useful ones. It was tested on one synthetic and four real networks, two tasks and nine rival methods.
According to the abstract, Scout had the best average result in 15 of 21 test settings and an average rank of 1.4, against 3.2 for the strongest rival. Refreshing for the right task beat refreshing for the wrong one by 0.005 to 0.007 in link-prediction score, and by 0.007 to 0.017 in classification accuracy. Those are small margins.
For people who own or use data: the worth of a record depends on the question it answers. A seller who can show which parts of a data set stay fresh and matter to a buyer's task has a clearer case than one who simply says the whole set is current.
