The Aardvark Speaks : essence, effervescence, obscurity. Established 2002. A weblog by Horst Prillinger. ISSN 1726-5320

June 11, 2004

One last word in the Wikipedia debate

I promise this is the last Wikipedia post, but two comments from reader Josh Cagliati actually helped me find a better wording for my Wikipedia criticism. Josh had challenged me to a bet: I should take ten random Wikipedia entries with mistakes in them, and if half of them were not gone by this time next year, he would pay me $10.

I wrote back:

Josh, it's actually not at all unlikely that you might win this bet, which is why my criticism was not that the mistakes don't eventually disappear — actually, that would be a stupid assumption, as the wiki very strongly encourages readers to correct mistakes. So I'm afraid I won't accept this challenge.

This apparently confused Josh. Like most of the people who had written back in defence of Wikipedia, he thought I had criticised the wiki principle, which I had not criticised. He wrote:

Hm, I would not have guessed it from your previous comment. [...] If you believe that mistakes will disappear, I don't see how wikipedia can fail to generate knowledge. I certainly agree that wikipedia has mistakes and inaccuracies, but so does Encyclopedia Britannica. Some of Britannica's mistakes have stayed in for over a hundred years with out being removed, which I highly doubt would happen in wikipedia. In my experience the content of wikipedia articles that have been edited by at least 10 authors [...] is at least comparable to say the world book encyclopedia. [...]

If you don't like Wikipedia, come back in a year or two, and you will be amazed at the improvements.

Now everything that Josh is writing here is correct, and yet he is totally besides my point, and comparing apples with oranges.

Josh is writing that mistakes are being removed. I had criticised that people kept adding mistakes to articles. It's odd to me how wikipedia advocates only see that mistakes can be removed, but not that they can also be added.

I wrote:

You are nicely summarizing the problem of Wikipedia:

You do not see how the simple fact that mistakes will eventually disappear alone is not enough to generate knowledge.

This is precisely why some of the content of Wikipedia is so bad: because it is not built on the assumption that information must be correct in the first place, but instead on the assumption that mistakes do not matter because they will eventually be corrected.

Knowledge [in Wikipedia] may emerge only if all articles have run through enough revisions to ensure that they are complete and correct, and providing that nobody is adding more mistakes to them. Otherwise all we have is essentially noise.

And why on earth should I come back in a year? I was criticising the content of Wikipedia now, not in a year. Whatever it'll be like next year is inconsequential to me, I want precise, correct information now. Like the average Wikipedia user, who is looking up something, expects the information he finds to be correct at the very moment when he looks it up, not a year later. [...]

In effect, Josh was nothing but proving my point of the engineer's way of thinking, that a good technical solution outweighs content. Only the real world isn't working like that.

Posted by Horst on June 11, 2004 08:49 AM to books & bookkeeping | Tell-a-friend

We received this ping from The Aardvark Speaks on February 15, 2005 12:52 PM:

Wikipedia revisited: One of my students at the teacher training college had used this Wikipedia entry as the basis for a history lesson on imperialism that he taught in a practice class. His supervisor (not me, by the way) failed him, not for his teaching method, but for g... [more]

arved said on June 11, 2004 10:27 PM:

The bet would have been fun with a counterpart. You suggest a reasonable researched encyclopedia, he _tries_ to find 10 errors in it and how many of them get fixed in a reasonable time.

Horst said on June 11, 2004 11:41 PM:

He he, yes, that would have been fun. :-)

Steve said on June 12, 2004 03:18 AM:

Communication during 2003 -

Me: The original equipment tires are not listed as an approved tire in the owners manual of my 2001 BMW.

BMW of North America: We will contact you shortly.

BMW of North America: Your manual will be corrected in the next revision.

Josh Cogliati said on June 13, 2004 05:47 PM:

Final, you specify clearly what you think the problem is. So, as I understand you are making two assertions:
Mistakes will get added faster than they are removed in WikipediaWikipedia is unreliable right now.

I think that the first assertion is incorrect. In my experience, mistakes get added mainly with new information. The mistakes gradually get removed, and finally, you are left with a nearly mistake free article after sufficient edits. Once the article is perceived by most people to have all the information that is needed, the article goes into a phase where it primarily has mistakes fixed, and very few new mistakes are added.

I know you keep saying that technical solutions do not equal good content, but there are a variety of features that make the improvment of articles easier. First and foremost is Watchlists. Every logged in user gets to make a list of pages that they are interested in watching. The user can then look at the watchlist, which shows the last time that a page in the watchlist was modified. This makes it so that checking say the fifty articles that the user is most knowledgeable on take about 5 minutes a day if there are no mistakes added. If there are some mistakes, then they need to be fixed, which can take more time, but still possible in less than half an hour. Another feature is that the whole history of an article is kept around, and any two version can be compared. If there are changes between the versions, then they will be highlighted ( Example) Therefore, if the article was correct in the past, that version is still around. Edits can also be easily reverted to the previous version, if that is so desired.

Another trend that is happening is that the number of edits per article is increasing ( Wikipedia stats) This is especially interesting considering that the number of articles is also increasing greatly. So, even as the number of articles increases, so does the average amount of editing that each article has. This trend would tend to be consistent with improving articles.

So in short, with regards to the assertion that mistakes are being added faster than they are removed, 1) it is inconsistent with my experience on wikipedia, 2) there are technical features that make it easy to keep track of articles and make sure that changes are improvements, or revert them if they are not, and 3) the trend seems to be more editing of articles, rather than less.

As to your second assertion, Wikipedia is unreliable now, I do agree to some extent. However, I would like to note that on the front page it says "We are building an open-content encyclopedia in many languages." Notice that it says building, not built. There are many proposals for Approval mechanisms on Wikipedia, so that at some point it time, Wikipedia will have articles that can be reliably trusted. Wikipedia is already good enough for many purposes, even if it is not completly reliable.

In my lifetime, I have found mistakes in Books, Encyclopedia's, journal articles, college lectures, news articles and my own thoughts and writings. The real world creates mistakes. Since I started working on wikipedia, there have been many times when I have been reading a book and found a mistake, and instinctively, I looked for the edit this page link. This doesn't exist in books, but it does in Wikipedia. Wikipedia assumes that mistakes will happen, but allows for them to be removed incredibly easy, so they do get removed.

Josh Cogliati said on June 16, 2004 03:47 AM:

As for the counterpart bet, here are some mistakes from encyclopedias:

Encyclopedia Britannica has had an incorrect explaination for why a Radiometer works since at least the 1911 edition. This mistake is still in the 2003 edition. I think that the incorrect reason could have been eliminated in the past 92 years, since the correct reason has been known since the 1800s.

I spent an hour or so looking at the online Columbia Encyclopedia and found two mistakes in the Price article. First it states: "Most economists hold that, in the long run, price in a competitive market will equal the cost of production." The price will be the average cost of production, and that is assuming that all businesses have the same cost structure. Second: "In the short run, however, the market price will be determined by supply and demand without reference to cost." The cost is built into the supply curve, so the price still has cost information in it. I am sure that if I was interested in finding more mistakes, I could, I just need more time.

Regular encyclopedias have mistakes, just like Wikipedia.

Horst said on June 16, 2004 10:07 AM:

Are you aware just how ridiculous your comparison is?

You are listing one mistake in Britannica, one that has, I might add, been discussed to death by Wikipedia devotees. Almost looks as if it were the only mistake they could find. I'm sure it's not, but citing the same mistake over and over instead of coming up with an estimate of total mistakes does not support your argument.

So that's one mistake. And that is compared to how many mistakes currently in Wikipedia? A couple of thousands? Tens of thousands? Hundreds of thousands? So please excuse me if I don't take your example seriously.

I never said that other encyclopedias are completely error-free. I said that they are better researched, and I said that currently, Wikipedia was a proliferation of noise rather than knowledge. Unless you have proof that, at the present time, Wikipedia does not contain significantly more mistakes than other encyclopedias, I still don't stand corrected.

Josh Cogliati said on June 16, 2004 03:22 PM:

Okay, here is an estimate for number of mistakes in Columbia Encyclopedia. I looked at five articles in roughly an hour. I found 2 mistakes. Since it has 51,000 articles, then assuming the same rate of mistakes (2 per 5 articles), there are 20400 mistakes in Columbia Encyclopedia. If I missed mistakes in those articles, then then there might be more mistakes. If the articles that I looked at are more mistake prone than the average, then there might be less.

If you think that there are merely 10,000 or less mistakes in Wikipedia (a couple of thousands), then it might compare quite well to the Columbia Encyclopedia.

I was unable to find the place where the radiometer mistake was discussed to death by Wikipedians. Would you be so kind as to provide a link?

I am curious, why I should provide proof that Wikipedia is better than other encyclopedias, when you have not even provided a link to a single mistake in Wikipedia, yet I have provided links to mistakes in two encyclopedias?

Horst said on June 16, 2004 06:09 PM:

Simple: I made a statement, and you more or less said it was unjustified. So naturally you'd have to prove your point to convince me that I'm wrong and you're right, whereas I already believe that I'm right and therefore don't have to prove it to myself.

You are so far failing to convince me. I had thought we were coming closer to understanding each other's points of view for a while, but then you brought up the argument that mistakes in other encyclopedias somehow make mistakes in Wikipedia less of a problem, which I just can't follow. I also can't follow your argument that changes in Wikipedia will always correct an entry and never introduce new mistakes, but most of all I don't see how the fact that Wikipedia may be useful in a few years affects my frustration with its current state.

If you have something to say that addresses my criticism and is not just a repetition of what you and others have already said in the comments on my weblog, then please do so. Otherwise I will close the comments here because this discussion is clearly going nowhere, and I'm losing interest in it.

(By the way, the radiometer was mentioned by Mathias Schindler in his lengthy fisking of my original rant, and again by commenters at the Schockwellenreiter's site. I don't really care whether it's correct or not, I just thought it funny that everyone is mentioning that particular, and fairly obscure, Britannica entry, even though there must be countless other ones with mistakes in them.)

Josh Cogliati said on June 17, 2004 03:03 AM:

Okay, so dispite that fact that you have repeatedly claimed Wikipedia has an unusual number of mistakes, you will not provide any evidence by linking to articles that have mistakes? Nor have you attempted to quantify the claim. That makes me think that you might have a particularly weak claim.

I honestly had not realized that others have mentioned the radiometer that often.

To get this discussion back on track, here are some assertions that I think we both can agree on:
Traditional Encyclopedias have mistakes.
Wikipedia has mistakes.
Mistakes in Wikipedia will tend to get corrected
New mistakes can easily be introduced into Wikipedia
Wikipedia allows any version of an article to be viewed
A knowledgeable person can find mistakes in some of the traditional encyclopedias in a short amount of time.

Do you agree with the assertions I have just written, or are there any you disagree with?

Horst said on June 17, 2004 01:03 PM:

If you rephrase number 3 to "Mistakes in Wikipedia can easily be corrected", then I can agree with all of them.

Josh Cogliati said on June 20, 2004 01:30 AM:

Okay, now, two more assertions that I will attempt to convince you of.

First, It it easy to switch an article to a previous version.

Wikipedia stores the complete history of every article. The only exception is articles that have been deleted. To switch back to a previous edition, you need to go to view it in the history, select edit, and then save the old version. This takes about 2 minutes when I do it using a modem.

Second, it is easy to see what has been changed in a set of articles.

All logged in users get a watchlist. It lists the last time that each article on it was changed. Each article has a related changes button. The related changes lists all changes in articles that are linked from the current article. Since there are "articles" like list of economics topics, it is easy to see if there are any changes in any economics articles, or almost any subject you can think of. Lastly, there is recent changes, that lists all the changes that have been made to Wikipedia. From each of these pages, you can go to the link to the history page, or see a diff from the last version. The history page lists all the changes that have been made to the article, and allows the differences (diff) between any two versions to be displayed. So, it is easy to keep track of the changes to a large number of articles.

Comments have been closed for this entry.

© Copyright 2002-2008 Horst Prillinger, 

Most of the stuff on this page is fiction. Everything else is my private opinion. Please read the disclaimer.

Valid XHTML 1.0! Powered by Movable Type Made with a Mac