Site Review - Search Engines

[Mark Weeks]

For this review, I'm taking a break from looking at the bookmarks for
the Chess History discussion group. Instead I'm going to look at what
the search engines say about chess history on the Web. This is to find
any overlooked sites and to catch up on current search engine
technology. I used 'Search Engine Watch' by Danny Sullivan at...
http://searchenginewatch.com/links/Major_Search_Engines/The_Major_Search
_Engines/index.html
...as a starting point for this constantly changing technology. Sullivan
lists 22 different search engines on this page.
I first made a pass through all 22 entries on Sullivan's list and tried
to search each on 'chess history'. This was to get my bearings and to
become familiar with each site. For various reasons, I eliminated five
-- Ask Jeeves, HotBot, Inktomi, Netscape Search, & RealNames -- from
further consideration. I also eliminated the four directories --
LookSmart, Open Directory, Snap, & Yahoo -- because they use a
completely different technology to classify sites. This left 13 search
engines for further consideration -- AOL Search, AltaVista, Direct Hit,
Excite, FAST Search, Go/Infoseek, GoTo, Google, IWon, Lycos, MSN Search,
Northern Light, & WebCrawler.
I then recorded the top five chess history pages listed at each search
engine. This gave me a list of 65 pages apparently related to chess
history -- 45 of these were unique because some search engines returned
the same pages that other search engines returned. Of the 45 unique
pages, 12 appeared in more than one search engine. For each of the 12,
I'll give the number of times it appeared in the search engines, the
title of the page, the address of the page, & a short description...
Table 1 - Pages returned by more than one search engine.
7 - Chess History
http://www.misc.traveller.com/chess/history/
This is Bill Wall's list of important dates in chess history & one of
the discussion group bookmarks. I'm not surprised that it came out first
-- it's an excellent introduction to the subject.
4 - Chess History
http://www.deja.com/~chesshistory/
The Chess History discussion group; nice to see that it's recognized by
the search engines. It appears that having 'chess' and 'history' in the
page address are important to search engines.
3 - Home page of The Chess Variant Pages
http://www.chessvariants.com/
Another of the discussion group bookmarks, which I'll be reviewing soon.
It's an excellent site with a wealth of material related to the
development of the rules of chess.
2 - History of Chess
http://home.att.net/~valyana/
A Russian language page which I don't understand, but which is a list of
links. It's not at all clear why two search engines -- FAST Search &
Lycos -- flagged this. The phrase 'chesshistory' as a single word
appears once as the key to a guest book. (*)
2 - Hellas Chess Club
http://www.chess.gr/
A small doorway page which redirects to another page. The word 'history'
appears twice on the doorway page, but I couldn't find it on the next
page.
2 - Chess Hall of Fame: Best Chess Players in U.S. History
http://www.chesslinks.org/hof/home.html
Not one of the discussion group bookmarks, but I recently reviewed it
along with the Excalibur Hall of Fame site, which is a bookmark. (*)
2 - Chess, the game of the Goddess?
http://www.goddesschess.com/
An unorthodox page which starts with a quote by George Koltanowski,
"Despite the documented evidence of chess historian H.J.R. Murray, I
have always thought that the game of chess was invented by a goddess."
This is worth a more careful look.
2 - History of Chess in Lebanon
http://www.lebchess.org/history.htm
The title describes the content. There are many Web pages which cover
the history of chess in a country, region, or club. They could probably
be cataloged & bookmarked, but it's not a job I want to tackle, because
it would be difficult to maintain properly. (*)
2 - Untitled
http://www.netcologne.de/~nc-jostenge/index.htm
Another of the discussion group bookmarks -- one of the best sources for
information about the origins of the game.
2 - Chess news and history: The Game is Afoot by Terry Crandall
http://www.pstat.ucsb.edu/~carlson/chess
Thirteen biographies of past grandmasters.
2 - RedWeb Chess - Mrchess - Chess Discussion - Bookstore - Rules -
http://www.redweb.com/chess/
Lists U.S. champions & world champions. (*)
2 - Brief History of Chess
http://www.romanchess.com/History_of_Chess.htm
Eight paragraphs on 'A Brief History of CHESS and CHESS VARIANTS'. (*)
...How well did the search engines perform? Of the 33 pages which were
returned by a single search engine, five came from AltaVista. This means
that AltaVista did not return a single page matching the results of
another search engine. What did AltaVista return?
1st - Games
http://assembly.nerdworld.com/directory/games.html : Links to discussion
groups related to games.
2nd - Computer Chess Programming
http://www.xs4all.nl/~verhelst/chess/programming.html : A links page.
3rd - Home
http://www.chesscity.com/ : Eric Schiller's site for Cardoza Publishing
-- has a section on chess history with original material by Schiller.
4th - Chess For Students
http://www.chessforstudents.com/ : A book store where the word 'history'
doesn't appear at all, although the phrase 'chess history' is used in
the meta tags.
5th - White Cloud
http://www.whitecloud.com/ : A personal page unrelated to chess history.
Only one site in the top five has anything to do with chess history. The
AltaVista advanced search returned a different set of pages, but these
were also not relevant to chess history. It appears that AltaVista has
serious problems to determine relevance. DirectHit returned four pages
which were not returned by the other search engines, but the pages
appear to have something to do with chess history.
All five pages returned by both FAST Search & Lycos matched a page
returned by another search engine. In fact the two search engines
returned the same pages in a slightly different order. Sullivan says,
'The Norwegian company behind FAST Search also powers the Lycos MP3
search engine', and '[Lycos] secondary results come from either Direct
Hit or Lycos own spidering of the web'. It appears instead that Lycos is
entirely dependent on FAST Search. I went back to Table 1 & marked the
five duplicate pages with an asterisk '(*)'. These five sites were
returned only by FAST Search & Lycos, so like AltaVista there was no
overlap with any other search engine. Unlike AltaVista, most of the
sites are relevant to chess history.
Sullivan also compares the sizes of some search engines in millions of
pages indexed...
300 FAST Search
250 AltaVista
214 Excite
211 Northern Light
138 Google
110 Inktomi
50 Go
50 Lycos
...which gives some idea of their relative scope. Of these eight, which
search engine gave me the best results? It's certainly a personal
choice, but I was most impressed with FAST Search, Google, & Northern
Light.
I've already excluded AltaVista, Inktomi, & Lycos. I'll add Go to the
list. I moved my own site to a new address almost six months ago, but Go
is the only search engine which still reports the address of the old
site. This indicates that its index is stale. Another clue -- of the
chess history sites that it returns, none is dated after mid-January --
which is not very impressive.
---
Looking at the four directories...
LookSmart has a category dedicated to 'Chess History', with nine sites
listed. It also lists all sites in its directory which have the keywords
'chess' & 'history' in the descriptions of the sites. The list of 'Top
10 most visited sites' takes its results from DirectHit.
Open Directory lists ten sites which seem to be the results of a search
on the keywords 'chess' & 'history' in the descriptions of the sites.
Snap lists the 25 'Most popular sites reviewed by our editors' including
the 'Black Dog Billiard Cafe' and the 'Bastardo Game' (!?). These are
followed by the six 'Most popular sites submitted by our members',
including 'Canadian Sculpture reproductions'. There's something wrong
here.
Yahoo also has a category dedicated to 'Chess History', with eight sites
listed. More sites are listed as a result of a search on the keywords
'chess' & 'history' in the descriptions of the sites.
...None of the four impressed me very much concerning chess history. The
subject is probably too specialized to be covered adequately by a
nonspecialist. This may indeed be true of most specialist subjects. It
takes some prior knowledge & experience to compile a list of Web sites
on a specialist subject -- nonexperts can't separate the wheat from the
chaff.
---
The last topic I want to mention is the meta search engine. This is a
tool which merges & compares the results from multiple search engines.
It automates the job I did to compare the results for chess history.
Sullivan has a list of nine on his page...
http://searchenginewatch.com/links/Metacrawlers/Major_Metacrawlers/index
.html
...I looked at all nine & found that they did a good job of eliminating
pages which had little to do with chess history. I was particularly
impressed with SurfWax, which I had never seen before & which organizes
the results in an original way. I'll definitely be using it for future
Web searches.
Bye for now,
Mark Weeks
[I would normally have posted this on 1 April, but I'll be away from the
Web for the next few weeks. The next review should appear on 15 April.]

Sent via Deja.com http://www.deja.com/
Before you buy.