Who Fact Checks the Fact Checkers?

“… NewsGuard falls far short of the very same criteria for accuracy and transparency that it claims to apply to other websites. Most of the company’s fact checkers lack basic qualifications in the scientific and social-scientific fields that they purport to arbitrate.

NewsGuard’s own track record of commentary — particularly on the Covid-19 pandemic — reveals a pattern of unreliable and misleading claims that required subsequent corrections, and analysis that regularly conflates fact with opinion journalism in rendering a judgement on a website’s content.

Furthermore, the company’s own practices fall far short of the transparency and disclosure standards it regularly applies to other websites … NewsGuard’s staff primarily evaluates scientific claims by appealing to the authority of public figures who they designate as ‘experts’ on the subject in question.

Their approach generally avoids direct examination of the evidence surrounding contested claims, and instead cherry-picks a figure to treat as an authoritative final word … many of their preferred authorities are political officeholders rather than persons trained in scientific or social-scientific methods.

By selectively curating cherry-picked political authorities rather than evaluating evidence directly, NewsGuard’s approach to fact-checking effectively sidesteps the scientific method. This strategy is rendered even more problematic by the general lack of scientific expertise within NewsGuard’s team of writers.

We examined the educational credentials, including the highest degree listed, for 28 publicly identified staff members on NewsGuard’s website. The company’s staff page reveals shockingly little expertise in either the hard sciences such as medicine or social sciences such as public policy, economics, and related fields ...

Most NewsGuard articles on Covid-19 topics and policies are written by [NewsGuard Deputy Editor for Health, John] Gregory, whose only identified qualification is a bachelor’s degree in Media Arts … Gregory would not qualify as an expert in most of the fields he is responsible for fact-checking …

Of course, non-experts have every right to offer opinions on scientific and social-scientific matters. Whether or not they should be taken seriously as fact checkers or act as arbiters of scientific disputes is another question entirely.”

“Gregory and his colleagues appear to have simply decided that their own premature dismissal of the lab leak hypothesis equated to ‘fact’ and proceeded to penalize other sites not for factual errors, but rather for diverging from NewsGuard’s own editorial position on the same subject.

When this position turned out to be mistaken, NewsGuard pivoted to remove the errors — albeit in non-transparent ways that downplay the significance or pervasiveness of their mistake.”