A News Search Showdown & $1 million to OCA

Wednesday, I came across an AP article about book searching, "Google Book-Scanning Efforts Spark Debate." It mentions a "$1 million grant to the Internet Archive, a leader in the Open Content Alliance, to help pay for digital copies of collections owned by the Boston Public Library, the Getty Research Institute, the Metropolitan Museum of Art." The article also discusses concerns with Google's project. As interesting as this article is, it lead to an even more fascinating (to me, at least) comparison of news sources and news search engines.

Bear with me a bit on this one. I came across my first reference to the article on Doggdot which reference a Slashdot story pointing to the Washington Post version of this story. Being the librarian that I am, I always struggle with which version of an AP wire story to link to. Since the AP story is published by many newspapers, which link makes the most sense as being the most authoritative? I am still not sure, but I decided to use a phrase from the story to see which versions various news search engines retrieved. I was surprised to see the lack of overlap.

Using a phrase from the story, "You are talking about the fruits of our civilization and culture," I ran the comparison at the following search engines, including three commercial news databases at the end. The second column gives the number of results that the search engine first displayed, which for Google, Yahoo!, and Live was a different number than the total results they could display (which is in the third column).

	Found at first	Total found
Yahoo! News	10	37
Google News	1	71
Ask News	7	7
Live News	1	3
Topix	6	6
Technorati	7	7
LexisNexis Academic	0	0
InfoTrac Newspapers	0	0
Factiva	3	3

Surprisingly, none of the search engines above linked directly to the article at the Washington Post, or for that matter, at the New York Times. The lesson, for me, is that the general news search engines can often post AP stories from minor newspaper sites, and that it is worth checking at major newspaper sites directly, especially if you are looking for an expanded version. It also reinforced prior experience that for current day stories, the Web search engines work far better than some commercial ones (though I've seen exceptions).

Just to share some of my thoughts, and what I was seeing, I recorded some of the screens. You can watch for yourself in my news search engine showdown screencast (05:26).

If embedded version above does not work, try it direct at YouTube.