November 2003 Archive

Google Starts Auto Stemming Searches

Since the most recent Google Dance started around Nov. 15, this update of the Google database nicknamed Florida has created quite a stir in the ecommerce Webmaster community. The major complaint has been the significant change in the ranking of results and many pages no longer show up in the top of the search engine results. For those with time, read the thousands of postings about it in the Update Florida discussions at WebmasterWorld.

Certainly, the ranking changes will also have an impact on searchers, but even more significant to me is the experimentation that Google is now doing with automatic stemming. Discovered first in the Cre8asiteforums, Google changed its Basic Help page to announce that it is now using stemming.

Basically, Google now takes search terms and looks for grammatical variants of SOME of them. Unfortunately, Google does not make it clear which terms it stems and which it does not. I found no plural or singular variants but did find some examples of verb variants. For example, a search on drink water matches pages with 'drinking' and 'water' while run linux also finds 'running.'

You may be able to identify when it happens by looking for the highlighted terms in the search results, but it is not always obvious when this occurs especially if the hits do not rank high enough to appear on the first page. The stemming does not seem to occur on single word searches or on phrase searches (yet another reason to use quotes for phrase searching whenever possible).

Does this help relevance? Maybe for some searches and searchers, but for precision searching it can also be frustrating. Plus, the searcher is not given the choice of when to use it and when to turn it off. MSN Search has offered a stemming check box on its advanced search page for years. Since Google does not say when they will turn on the stemming and when they do not, they could at least give searchers the choice of when to use it (at least for those of us that use features like the preferences and advanced search options).

Update: Gwen found an instance of plural stemming and Danny notes that putting a plus in front of terms will turn it off.

FindArticles Relaunched, Expanded, and Integrated

FindArticles.com has relaunched and expanded to covering 700 publications with full text articles from Gale, up several hundred periodicals from its previous version. Along with LookSmart's announcement, the FindArticles database is now accessible as a tab directly from the LookSmart site, which should raise its prominence.

The new LookSmart and FindArticles interfaces have three tabs for searching -- the LookSmart Directory, the Web (Wisenut), and Articles. They have changed the default operation to AND and continue to support phrase searching and the - as a NOT operator.

FindArticles is still a small database compared to the large aggregator databases offered by InfoTrac, EbscoHost, Wilson, and ProQuest, among others, not to mention the full text electronic journal databases from many publishers. But for users without ready access to such databases or to a library with such access, FindArticles certainly adds some great quality content to the free Web.

Gigablast Adds Spelling Suggestions

Joining the other major search engines, Gigablast now has a spell checker which makes suggestions for correct (or just alternate) spellings of unusual query terms. Like most other search engines, these suggestions are displayed as "Did you mean. . .�"

Matt Wells also offers a fascinating exploration of the spelling suggestions at other search engines in his announcement. Searching 'dooty' at several search engines he found no consistency in suggestions:

• AlltheWeb - booty
• Altavista - dhooti
• Gigablast - door
• Google - doody
• MS Word - Doty
• Teoma - doty

And none gave what I would expect to be the obvious correction of 'duty!'

Sometime in the next year, Gigablast also plans to have hardware that can handle a 5 billion document database that can still serve hundreds of queries per second. And the add URL page is back up again.

MSN Search Exploring News Search

MSN Search is finally starting to add a news search. However, it is only in beta and is only available at the MSN Search site for the United Kingdom, France, Italy, and Spain. Powered by Moreover, it claims more than 4,000 sources. Read about it from the MSN Newsbot site or try the UK version. To search, look for the "Search news" box in the middle of the left margin rather than the regular MSN Search UK box at the top right. It searches many news sites from outside the UK (including US sources) and seems to go back about one month. It would benefit from a date sort option.

Yahoo! Revises Front Page Search

Last weekend, Yahoo! revised the search box on its front page. In Internet Explorer, it now shows four tabs above the longer search box: Web, Images, Yellow Pages, and Products. These options have been available for awhile from the search.yahoo.com page, but not directly from the main page. However, the link to search.yahoo.com is now gone from the main page, as is any ability to just search the Yahoo! directory. Why in the world do they not include the directory as one of the tabs? At least on the Yahoo! search page, searchers can customize which tabs appear by using the "Add or remove tabs" link, and these can directory, maps, travel, and more options.

To make it all a bit more confusing, in other browsers, these tabs are not displayed at all. Initially, other browsers did not have any access to the various search options. Then, for other browsers (such as Mozilla, Opera, and Netscape), the words "Search the Web" became a link to search.yahoo.com. At other times, these alternate browsers will get a drop down list after the search box with the four choices listed above.

So Yahoo! is continuing to work on the display. In the meantime it makes it difficult for anyone trying to teach search with Yahoo! since people will see different displays depending on their browser. And if you would like to just search the directory, try either search.yahoo.com or dir.yahoo.com.

Google Does a Deskbar

Expanding on its success with its toolbar, Google launched a new Google Labs experiment today: the Google Deskbar. Rather than a browser add-on, like the HotBot deskbar, it appears in the Windows taskbar and can function independent of the browser. It can be used for many Google functions, including the calculator, definitions, Web searches, news, groups, Froogle, and more.

Unfortunately, it still only works for those with Windows 98 or higher and requires Internet Explorer 5.5 or higher. It displays the results in a mini-viewer instead of the full browser, and because of that can be faster than opening up a browser to see results. However, the mini-viewer can send a page to a user's default browser which does not have to be Internet Explorer. Even so, like the toolbar, it is yet another way that Google tries to get users to rely primarily on its search services rather than others.

The announcement I received from Google, but which is not on their site, reads as follows:

Today, Google released a new Google Labs experiment called the Google Deskbar, a search application that enables PC users to perform Google searches at any time from any application.

The Google Deskbar is a free software download that appears as a search box in the Windows taskbar at the bottom right of most Windows-based PCs. Users enter queries into the search box and results are automatically displayed in a small pane that rises above the Deskbar and overlays a corner of the application they’re using.

The Deskbar provides instant access to information on the web, from any application. For instance, a user in the midst of typing an e-mail can check facts or find definitions by simply entering words and phrases into the Google Deskbar. Additionally, typing Ctrl + Alt + G automatically positions the cursor into the deskbar search box, enabling users to search instantly without having to move the mouse. When users highlight text on a page and press Ctrl + Alt + G, the highlighted text is automatically inserted into the search box.

Forward and back buttons to the top left of the Deskbar pane enable users to easily click through results pages, and a small arrow-shaped link launches a browser for users who wish to view results in full screen. The deskbar menu offers links to all Google services and to helpful web resources such as definitions, stock quotes and other useful information. Users can customize these links via the Options menu. This is an English-language only product and is available for Windows users running IE 5.5 or higher versions.

We’re excited about experimenting with new technologies that make it faster and easier for people to connect with the information they need. With the Google Deskbar, users get a great search experience without moving their fingers from the keyboard, from whatever application they’re currently using.