| Comment | part of the nature article:
Special Report
Nature
Published online: 14 December 2005; Updated online: 22 December 2005 ; Updated online: 28 March 2006 | doi:10.1038/438900a
Internet encyclopaedias go head to head
Jimmy Wales' Wikipedia comes close to Britannica in terms of the accuracy of its science entries, a Nature investigation finds.
Jim Giles
One of the extraordinary stories of the Internet age is that of Wikipedia, a free online encyclopaedia that anyone can edit. This radical and rapidly growing publication, which includes close to 4 million entries, is now a much-used resource. But it is also controversial: if anyone can edit entries, how do users know if Wikipedia is as accurate as established sources such as Encyclopaedia Britannica?
Several recent cases have highlighted the potential problems. One article was revealed as falsely suggesting that a former assistant to US Senator Robert Kennedy may have been involved in his assassination. And podcasting pioneer Adam Curry has been accused of editing the entry on podcasting to remove references to competitors' work. Curry says he merely thought he was making the entry more accurate.
However, an expert-led investigation carried out by Nature — the first to use peer review to compare Wikipedia and Britannica's coverage of science — suggests that such high-profile examples are the exception rather than the rule.
The exercise revealed numerous errors in both encyclopaedias, but among 42 entries tested, the difference in accuracy was not particularly great: the average science entry in Wikipedia contained around four inaccuracies; Britannica, about three.
Considering how Wikipedia articles are written, that result might seem surprising. A solar physicist could, for example, work on the entry on the Sun, but would have the same status as a contributor without an academic background. Disputes about content are usually resolved by discussion among users.
But Jimmy Wales, co-founder of Wikipedia and president of the encyclopaedia's parent organization, the Wikimedia Foundation of St Petersburg, Florida, says the finding shows the potential of Wikipedia. "I'm pleased," he says. "Our goal is to get to Britannica quality, or better."
.......
.......
Editors at Britannica would not discuss the findings, but say their own studies of Wikipedia have uncovered numerous flaws. "We have nothing against Wikipedia," says Tom Panelas, director of corporate communications at the company's headquarters in Chicago. "But it is not the case that errors creep in on an occasional basis or that a couple of articles are poorly written. There are lots of articles in that condition. They need a good editor."
Several Nature reviewers agreed with Panelas' point on readability, commenting that the Wikipedia article they reviewed was poorly structured and confusing. This criticism is common among information scientists, who also point to other problems with article quality, such as undue prominence given to controversial scientific theories. But Michael Twidale, an information scientist at the University of Illinois at Urbana-Champaign, says that Wikipedia's strongest suit is the speed at which it can updated, a factor not considered by Nature's reviewers.
"People will find it shocking to see how many errors there are in Britannica," Twidale adds. "Print encyclopaedias are often set up as the gold standards of information quality against which the failings of faster or cheaper resources can be compared. These findings remind us that we have an 18-carat standard, not a 24-carat one."
The most error-strewn article, that on Dmitry Mendeleev, co-creator of the periodic table, illustrates this. Michael Gordin, a science historian at Princeton University who wrote a 2004 book on Mendeleev, identified 19 errors in Wikipedia and 8 in Britannica. These range from minor mistakes, such as describing Mendeleev as the 14th child in his family when he was the 13th, to more significant inaccuracies. Wikipedia, for example, incorrectly describes how Mendeleev's work relates to that of British chemist John Dalton. "Who wrote this stuff?" asked another reviewer. "Do they bother to check with experts?"
But to improve Wikipedia, Wales is not so much interested in checking articles with experts as getting them to write the articles in the first place.
As well as comparing the two encyclopaedias, Nature surveyed more than 1,000 Nature authors and found that although more than 70% had heard of Wikipedia and 17% of those consulted it on a weekly basis, less than 10% help to update it. The steady trickle of scientists who have contributed to articles describe the experience as rewarding, if occasionally frustrating (see 'Challenges of being a Wikipedian').
Greater involvement by scientists would lead to a "multiplier effect", says Wales. Most entries are edited by enthusiasts, and the addition of a researcher can boost article quality hugely. "Experts can help write specifics in a nuanced way," he says.
Wales also plans to introduce a 'stable' version of each entry. Once an article reaches a specific quality threshold it will be tagged as stable. Further edits will be made to a separate 'live' version that would replace the stable version when deemed to be a significant improvement. One method for determining that threshold, where users rate article quality, will be trialled early next year.
Additional research by Declan Butler, Jenny Hogan, Michael Hopkin, Mark Peplow and Tom Simonite.
To see a list of the peer-reviewed encyclopedia articles, click here.
* UPDATE (22 December 2005): We have decided to publish more information about how our survey was carried out and the errors that were identified.
* UPDATE (28 March 2006): The results reported in this news story and their interpretation have been disputed by Encyclopaedia Britannica . Nature responds to these objections.
|
|---|
