Kevin Li recently faced a daunting task. Wikipedia contributors had just spent nearly two months locked in a heated debate over whether Fox News’ website should be permitted as a reference for the encyclopedia’s political and scientific content. More than 150 users weighed in, leading to a thread more than 82,000 words long, roughly the length of an average novel.
Li, a volunteer Wikipedia administrator and public policy graduate student at Stanford, had offered to close the discussion, meaning that he had to read the entire thing, plus the dozens of sources linked by participants, and boil all of it down to a single statement expressing the community’s rough consensus. It took him between 10 and 15 hours to do so, he told me.
The final result: Li found consensus that Fox be deemed a “marginally reliable” source for information about politics and science. This means that its use as a reference in Wikipedia articles will not be permitted for “exceptional claims” that require heightened scrutiny, but that its reliability will be evaluated on a case-by-case basis for other claims.
It also comes with some important caveats (a staple of Wikipedia bureaucracy). It applies only to reported news articles on foxnews.com, not to opinion articles or to opinion talk shows, which are considered generally unreliable. (Wikipedia tries to avoid using opinion sources to cite anything other than information about the beliefs of the source’s author.) It also does not apply to news from local affiliates owned by Fox, nor to Fox articles about nonpolitical and nonscientific topics, which are considered generally reliable.
Casual readers often ignore or skim over Wikipedia’s references, but they play a crucial role in its editorial process. The encyclopedia is a tertiary source, meaning that it aims to summarize the information found in secondary sources like newspapers. (Secondary sources themselves draw from primary sources like interviews.) Because of this pyramid structure, the secondary sources Wikipedia deems acceptable as references have a major influence on its content. If outlets like Fox News are permitted, Wikipedia’s view of the world will look more like Fox’s.
Currently, more than 16,000 articles cite Fox News as a source. But its use has been controversial for years. The first major discussion on its reliability took place in 2010. After 28,000 words of debate, it was closed by an administrator with consensus that Fox is politically biased but factually generally reliable.
The next major discussion, in 2020, ran to 77,000 words. It was closed by a panel of three administrators, who found consensus that Fox remained reliable for most types of content, but that there was no consensus about its reliability for politics and science content. In practice, this had a middling effect. A statement that could be attributed only to Fox News was unlikely to remain in an article if challenged, but there was also no rush to remove instances where Fox was used as a source as there might have been had it been deemed generally unreliable.
In the most recent discussion, arguments focused on interpretation of Wikipedia’s guideline on what qualifies as a reliable source, defined as one “with a reputation for fact-checking and accuracy.” The guideline makes a distinction between bias (a tendency to make editorial choices that favor a particular ideology) and reliability (the level of commitment to factual accuracy). Reliability is the salient factor for Wikipedians—some sources, such as the socialist magazine Jacobin and the libertarian magazine Reason, are regarded as biased but still deemed generally reliable and thus permitted.
In summarizing the discussion, Li wrote, “Editors taking a negative view of Fox News’s reliability referenced instances of questionable or incorrect reporting from Fox News, including instances where Fox News manipulated images or graphs, sensationalized its coverage, failed fact checks or made questionable assertions, failed to sufficiently distinguish news from opinion, and highlighted fringe perspectives.”
For instance, editors brought up a 2017 article Fox News published on Seth Rich, a Democratic National Committee staffer who was murdered in what is believed to have been a botched robbery attempt. The since-retracted article used anonymous sources to falsely suggest that he had been murdered in retaliation for leaking damaging DNC emails during the 2016 presidential election. (U.S. intelligence officials believe the leak actually came from the Russian government seeking to help Trump.) Rich’s family sued Fox and settled for an undisclosed amount in 2020.
Those defending Fox countered that “a certain level of errors is the journalistic norm,” and provided examples of high-profile retractions at other outlets. They also pointed out that many of the instances cited as examples of Fox errors, such an instance in which they falsely reported that Black Lives Matter said it stands with the Palestinian militant group Hamas, occurred specifically in headlines. Wikipedia does not consider headlines reliable, regardless of publication, because they are often not written by a story’s reporter.
Overall, the two camps were roughly evenly split. At least 67 of the roughly 150 participants sought a rating downgrade, whereas at least 52 sought to keep the prior marginal rating, and at least 12 sought an upgrade.
Li took his role as discussion closer seriously. “You don’t actually get to say, ‘I think this side is right, or I think this other side is right,’ ” he told me. “Your job instead is to read the will of the community.” And that’s more complicated than just counting votes. “Consensus isn’t a numerical thing,” Li said. “You can only participate with policy-based arguments,” so editors who backed up their stance by citing the reliable sources guideline or other guidance precedents (themselves a product of past consensuses) were given more weight in the closing analysis than those who did not.
The ultimate result of the recent discussion, that Fox be deemed “marginally reliable” for political and scientific claims, won’t change much in practical terms, since it’s treated the same as the 2020 “no consensus” finding. But it differs from the previous discussion in a subtle way, Li said. Last time, the split was largely between those who found it reliable and those who didn’t. This time, fewer editors argued that it is reliable, moving the locus of debate between the marginally reliable camp and the generally unreliable camp.
All the debate appears to have tired out many editors—which is remarkable, given Wikipedians’ famous tendency to debate all topics endlessly. Responding to complaints that it was launched only two years after the previous one and without a clear impetus, Li in his close urged the community “to consider the effort involved in holding these discussions” before initiating a similar one in the future. So the next battle over Fox’s status will likely be several years down the road.
But there are plenty of other sources to evaluate in the meantime. Editors maintain a list summarizing consensus on the reliability of frequently used sources with nearly 400 entries. In the archives of the noticeboard where Fox was discussed are thousands of additional discussions about more niche sources and about whether particular sources can be used to support particular pieces of information.
CNN and MSNBC, the other two major cable news outlets, are both deemed generally reliable, as are traditional news organizations like the New York Times and the Wall Street Journal (and yes, Slate). Several conservative publications, like the Daily Wire, are deemed generally unreliable. Breitbart gets even harsher treatment, having been blacklisted—links to the site are blocked as spam unless editors receive special permission to use them. A similar status was first applied to the Daily Mail, a British tabloid notorious for its poor journalistic standards, in 2017. Other publications in the “no consensus, unclear, or additional considerations apply” category include the liberal magazine Salon, the feminist site Jezebel, and the conservative magazine National Review.
Editors have also often penalized publications from countries without a free press. For instance, the entry for the Straits Times, the largest newspaper in Singapore, warns editors, “given known practices of self-censorship and political meddling into coverage, news related to Singapore politics, particularly for contentious claims, should be taken with a grain of salt.” Wikipedians use the availability of reliable sources to determine which topics merit an article, so the lack of reliable sourcing in countries like China limits the encyclopedia’s ability to cover them, creating a potential issue of geographic bias. Wikipedia can only be as comprehensive as its sources permit.
Wikipedia’s focus on sourcing ultimately traces back to one of its key policies, verifiability. “If you see something on Wikipedia, you should be able to check that it’s real by reference to another source,” Li said. “And if you can’t do that, then it doesn’t belong on Wikipedia.”
The verifiability policy helps Wikipedia’s model of crowdsourced knowledge function. In other situations, a work’s authority often comes from the identity of its author. This is the model used by traditional encyclopedias: When we read an entry on Britannica, we trust it because it was written by an expert staking their reputation on its accuracy. But on Wikipedia, where anyone can edit, there is no reason to trust any contributor. So instead, the authority of the site’s information comes from its verifiable references, which outsource the responsibility for fact-finding to secondary sources.
That spirit of transparency extends to Wikipedia’s behind-the-scenes discussions, like the one on Fox News, which are conducted publicly. “The beauty of it is that anyone can look at these debates for themselves” and see how they arrived at their result, Li said. “That’s what gives it the legitimacy.”
Future Tense is a partnership of Slate, New America, and Arizona State University that examines emerging technologies, public policy, and society.
