Fourteen experts label bias in 448 Hindi news articles
A small open data set shows how a language left out of the tools gets its first measuring stick.



Picture made for ABUREAU.
Can you tell when a news story is nudging you? Hundreds of millions of Hindi readers face that question daily, yet most bias tools were built for English.
A new paper presents 448 Hindi political news articles from four Indian news agencies, each labelled biased or unbiased by 14 experts. Biased articles also carry a type (selection, framing, clickbait or omission) and a written reason.
The labelling was consistent. The authors report agreement scores of 0.81 on the biased or unbiased call and 0.89 on clickbait, and the set is released under an open Creative Commons licence.
For people who own or use data, small and carefully labelled sets in under-served languages matter more than their size suggests. Written reasons and agreement scores let a buyer judge quality before using it. Owners of text in rarer languages may hold something nobody else can easily make.