Welcome to Source Notes, a new Future Tense column in which Stephen Harrison explores Wikipedia, digital knowledge, and the search for a fact-based world.
According to Wikipedia administrator and attorney BD2412, a lawyer using Wikipedia to brush up on the law itself would probably be committing malpractice. For example, if a defense attorney skimmed the Wikipedia page for assault but did not review the precise elements of that crime or tort in their jurisdiction, it would be so irresponsible (and lazy) as to likely rise to the level of professional misconduct. As an attorney myself, I’d be tempted to hurl a casebook at our hypothetical lousy lawyer and tell him to stop giving the rest of us a bad name.
The same rationale, it seems, would apply to the medical profession. Like most encyclopedias, Wikipedia typically functions as a launch pad that provides a general overview of a topic and points to further or original sources. But at least one new study suggests that Wikipedia is superior to other medical sources in at least one key respect: short-term knowledge acquisition. That is, when it comes to finding the right answers quickly, Wikipedia seems to lead the pack. This suggests a new way of thinking about the utility of the crowdsourced encyclopedia. Wikipedia delivers value not only by offering massive amounts of information with its nearly 5.8 million English articles so far, but by providing the means for even professional users to quickly identify and retrieve the most relevant information.
The authors of the paper, published in the Journal of Medical Internet Research in October, devised a “three-arm randomized trial” to test the comparative effects of three resources. 116 first- or second-year medical students in Canada took a multiple-choice medical test similar to the Canadian medical licensing examination. During the test, participants took notes on topics to research. After the test, the students were provided one of three pre-selected resources: Wikipedia, a digital textbook, or UpToDate, a subscription service mostly used by doctors. After the test, participants researched topics and took written notes using their assigned resource. Then the students retook the test using their notes.
If you’re like me, then at this point you’re probably feeling bad for the poor medical students. But at least the trial yielded a meaningful result: Students in the Wikipedia group had significantly better post-test performances on the exam compared to the digital textbook group. The Wikipedia group also outperformed the UpToDate group by a small margin, an impressive result given that UpToDate costs more than $500 annually for a subscription.
After reading the paper, I had a few questions about why Wikipedia had seemingly outperformed. So I contacted James Heilman, one of the paper’s authors, an emergency department physician, and an active Wikipedian. Back in 2008, Heilman was browsing the internet during a quiet night shift when he noticed that Wikipedia was not adequately distinguishing the medical condition of obesity versus the term overweight. He had the epiphany that he could “fix the internet” and edit the content directly. Since then, he’s kept going with more than 260,000 edits to date. During the 2014 Ebola outbreak, Heilman reduced his hours in the emergency room so that he could devote more time to updating and monitoring the topic’s Wikipedia page, which received millions of page views and more web traffic than information provided by either the World Health Organization or the CDC. “I see this as my way of giving back as a physician to the wider world,” Heilman said.
It’s not only the general public that turns to Wikipedia for health content. More than 90 percent of medical students, and 50–70 percent of physicians, use the online encyclopedia as a source for health information. There are other medical wikis, too, like WikiEM, which applies a how-to format for diagnosis and treatment. But Wikipedia receives far more traffic across its nearly 300 language versions, and in December received 161 million page views for its English medical content compared to the 356,000 page views I calculated for WikiEM using information from SimilarWeb.
Heilman explained that Wikipedia’s medical articles are closely watched by him and a core group of about 300 active volunteers in WikiProject Medicine using tools like this list of the most-visited medical pages each month. (Interestingly, Leonardo da Vinci’s page counts ranks among the most popular medical topics because of his anatomy work.) The project also publishes a style manual with guidance on appropriate sources and the suggestion to include dense inline citations.
The academic community generally discourages medical students from using Wikipedia for formal study, and there have even been few documented cases where users intentionally added misinformation. For example, one Wikipedia critic edited the article on the medical condition inflammation to state that it could be caused by “rhyolite”—a type of volcanic rock—a subtle falsehood that was not detected for weeks. Other doctors mentioned that Wikipedia’s medical content is more accurate for basic concepts than it is for advanced treatment options or highly specialized practice areas—but few, if any, suggest that Wikipedia should completely replace actual medical references.
Volunteers like Heilman don’t suggest that Wikipedia’s medical content is perfect. “Because we have decided to be such an open platform, there is this level of vandalism that we have to deal with,” Heilman said. But he believes it is more common for medical content to be missing than erroneous on Wikipedia, which is why he encourages more physicians to participate in the project.
When I asked Heilman why Wikipedia performed so well against UpToDate and digital textbooks in the comparative study, he pointed out that students who were assigned Wikipedia rated the encyclopedia higher on measures like ease of navigation, better search functionality, and “hyperlinks to other pages.”
His specific comment on the prevalence of hyperlinks on Wikipedia reminded me of one the nerdiest games I played with friends in college. Inspired by the “six degrees of Kevin Bacon” hypothesis, the rules are simple: Click “random article” on the English Wikipedia main page; click any hyperlink on that page with a whiff of Bacon; continue until you reach the Wikipedia page for Kevin Bacon, which you can typically reach within a few minutes. If you are clearly the first to arrive at Bacon’s entry, you are supposed to celebrate with an obnoxious amount of enthusiasm (or perhaps that’s just me). For close races, however, you and your opponent go back and review who found their way to Bacon using the fewest number of hyperlinks. The person with the shortest path wins.
Here’s the thing: There’s almost certainly a path to reach Bacon’s Wikipedia page in six clicks or less. In fact, many Wikipedia entries connect to the actor in less than three degrees of separation. And if you’re up for a challenge, try your hand at reaching the Jesus page in five clicks or less, or getting to the Philosophy entry using only the first link in an article.
How do these various Wiki games relate to the performance of the Canadian medical students in the trial? Because the millions of hyperlinks that make a “Wikipedia race” possible also make it easier to navigate between medical topics on the site. So far, by the way, those links have mostly been the product of human effort, added by thousands of volunteer contributors over the past 18 years of Wikipedia’s existence.
In some ways, the fact that Wikipedia allows for speedy connections between topics should be obvious. After all, wiki is a Hawaiian word meaning quick. What’s perhaps less obvious is why that quickness provides value. Older doctors might suggest that today’s med students have it easy—so much medical knowledge is available on their phones! But the body of knowledge required is many, many times the amount it once was. That’s why having tools to quickly navigate and connect concepts is so critical. As Heilman and the paper’s other authors note, the rate of expansion for medical knowledge has increased rapidly. The estimated doubling time of medical knowledge in 1980 was seven years, which allowed doctors ample time to gradually learn new concepts and treatment methods that were being discovered. The doubling time of medical knowledge in 2020 is expected to be a mere 0.2 years, or 73 days.
The expectation, of course, is that the corpus of useful human knowledge will continue to grow, not only for medical professionals but the rest of us. The accelerating knowledge curve only underscores the importance of maintaining connectivity within Wikipedia and other online resources. It’s in our interests to carry on with the topical linking and continue building tools that distill and summarize new information. Otherwise we risk losing our way in our ever-increasing knowledge base.