The Business of Search Engines

By Andrew K. Pace
American Libraries Columnist
andrew_pace@ncsu.edu

Head of Systems, North Carolina State University Libraries, Raleigh.

Column for May 2004

As Google continues its march toward ubiquity (and an eventual IPO), many libraries continue to grapple with access to, competition with, and description of “the free Web” for their patrons. While Google has dominated international attention with its ease of use and wide adoption, former favorites—such as Yahoo, AltaVista, AlltheWeb, and Ask Jeeves—have undergone many changes and market consolidation. This column is not about how or where to search on the Internet. The question I will address succinctly is: Who are these companies and what’s in it for them?

Market consolidation

For those of you who might have missed it, AlltheWeb and AltaVista, along with Inktomi and Overture, are now owned by Yahoo. In an amazing sweep, Yahoo has been able to acquire some of the best databases, indexing technology, and advertising prowess around. Their web domains and databases still appear to be separate, but Yahoo owns them all. AOL continues to use the Google database for searching, but still offers no compelling features for serious research.

Ask Jeeves now owns the Teoma database, which has been gaining in popularity for several months. In March, Ask Jeeves acquired Interactive Search Holdings (ISH), one of the largest privately held search and media companies. You might know ISH by its older name, Excite. ISH’s properties include My Way, My Search, My Web Search, iWon, and Excite.

For all intents and purposes, the field has narrowed to the big three: Google, Yahoo, and Ask Jeeves (and possibly four, depending on how Microsoft decides to enter the market with MSN Search). But why do they do it?

What business are they in?

Make no mistake, the search engine companies are not in the business of creating relevant and accurate Web search results. Google is an advertising firm—they all are. Their business is the conversion of search traffic into advertising revenue. This conversion is improved if a search engine is more popular, but good search results are the bait, not the big catch.

All search engine hit lists redirect users to advertisers’ websites. Some do it more subtly than others, but even Google includes more and more sponsored links. That’s how they make their money (Google, which has only 1,000 employees, is rumored to have generated $1 billion in revenue last year). How does it work? Like this:

Keyword Leasing. Advertisers actually pay for the rights to the keywords that users enter into the search box. When those words are searched, the advertised link appears—the so-called “sponsored link.” If more than one advertiser wants the same keywords, then keyword bidding can occur to determine the order that sponsored links appear. Search Google for “Andrew K. Pace” and you will see that Alibris bought my name in an effort to sell my book (apparently, I am not quite popular enough to force any bidding). Follow the link, and Alibris pays Google.
Paid Inclusion. Paid inclusion is when advertisers pay a fee to ensure that their links appear in search results. Inclusion does not ensure where the link will occur, so is not very popular and generates less solid revenue than keyword leasing or its cousin, paid placement.
Paid Placement. Placement is like inclusion, but adds assurance that the advertised link will show up in a given spot on the hit list, usually near the top.
Syndicated Ads. Search engines can also syndicate their ad links to other websites and search engines. You might have seen “Ads by Google” in various places. With little overhead, the advertising arms of search engine companies can easily extend their reach.

Does size matter?

You can be guaranteed that search engine companies will continue to distract consumers and libraries with boastful press releases about the size of their databases. But libraries should continue to be wary of the quality of search results, especially with the knowledge that advertising dollars are driving the business model. If abstract and index databases ranked search results based on the advertising in the magazines, libraries would have a lot to say about that (I hope I haven’t given anyone any ideas).

Furthermore, as most librarians already know, increasing the size of the database can adversely affect the relevance of results. This is good news for a search company like Northern Light. Once the darling search engine of libraries, Northern Light ceased its public Web presence last year, and then reemerged as a subscription service, Business Research Library. Set to reestablish its public presence, it will be interesting to see how Northern Light aims to differentiate itself in a highly consolidated market.

Search matters

Lest we forget, search engine technology could integrate nicely with library automation products. For instance, Innovative Interfaces has been using advanced search technology for several years in its keyword search engine; Endeavor’s ENCompass uses more advanced searches than most online catalogs. OCLC recently announced a partnership with Google to index a sampling of MARC records from its database. Google has also partnered with Stanford University: Project Ocean will digitize millions of Stanford’s public-domain titles and make them searchable with Google’s search technology.

Finally, Amazon—arguably closer to the main business of libraries than the search engine companies—continues to make forays into search technology, and made a recent splash with its “Search inside the book” offering. If you’re anything like me, you look forward to a day when searching a library’s holdings includes some sort of meaningful relevance ranking, de-duplication, and sorting of results.

With over 600 million searches per day, and mounting evidence that advertising works, search ads are not likely to leave the Internet landscape anytime soon. If anything, they will only change the face of free web searching more. Even Google’s status as the only “pure search” engine will be called into question if they go public and stockholders see the potential of more and more advertising revenue. It’s clear that in the long run, libraries have more at stake by leveraging various search technologies than they do in favoring one version of free search over another.

Open Source Watch

In response to the advertising-driven “free Web,” the software project called Nutch has launched a nascent effort to develop an open source Web search engine, free of advertising and proprietary search algorithms. For details see www.nutch.org.

“Open” does not always mean open source. While it’s not easily discernable from the website, the Open Directory Project (dmoz.org) is not free of corporate entanglements. The popular mass-edited directory of websites is actually the property of Time Warner.

Contracts and Agreements

Dynix sales of Horizon:

Eight library systems make the switch: Burlington (Wash.) Public Library, Davidson County (N.C.) Public Library System, Hutchinson (Kan.) Public Library, Mission Viejo (Calif.) Library, Gwinnett County (Ga.) Public Library, Orem (Utah) Public Library, Ventura County (Calif.) Library, and the Valley Library Consortium in Saginaw, Michigan, will all replace their legacy Dynix systems with Horizon 7.3.

The Lincoln Trail Libraries System, a major Illinois consortium serving academic, public, school, and special libraries, will replace its Dynix system with Horizon.

Endeavor sales of Voyager:

The London (U.K.) School of Economics and Political Science library, installation also includes ENCompass and LinkFinderPlus; replaces Sirsi Unicorn.

The Law Society of Upper Canada (Ont.); installation also includes ENCompass; replacing Bibliomondo.

EOS International sales of EOS.Web:

Wiley, Rein, and Fielding, an international law firm with business in 50 countries worldwide, replacing SydneyPLUS.

The American Kennel Club, New York City.

Ex Libris sales of Aleph 500:

City of Botany Bay Library in New South Wales and Bond University in Gold Coast, Queensland, both in Australia.

Innovative Interfaces sales of Millennium:

The Flemish Center for Public Libraries (serving over 300 Dutch-language public libraries in five regions across Belgium).

Wayne State University Libraries in Detroit will replace its Dynix Horizon system with a Millennium system, including Electronic Resource Management and AirPac.

San Diego County (Calif.) Library, replacing Sirsi DRA Classic.

Sirsi sales of Unicorn:

Bemis Public Library in Littleton, Colorado, for Sirsi.net ASP service, serving 120 hosted library systems.

Petersburg (Alaska) Public Library, replacing its text-based Talisman system.

Rockford (Ill.) Public Library, replacing its consortial Millennium system.

TLC sales of Library.Solution:

Southern Vermont College Library in Bennington, Southeast Arkansas Regional Library in Monticello, Jamestown (N.Y.) Historical Society, all automating for the first time; Briarcliffe College Library in Bethpage, New York, replacing Bibliofile; Alamo Heights Independent School District in San Antonio, Texas, replacing Dynix; Vernon Parish (La.) Library, replacing GIS Galaxy; South Arkansas Community College Library in El Dorado, replacing Dynix.

VTLS sales of Virtua:

Cumberland County (N.C.) Public Library, replacing GEAC.

Announcements

The Ex Libris Group has announced that Dan Trajman has been appointed president of the company’s U.S. subsidiary, Ex Libris (USA). Trajman was previously the president of Sapiens Americas.

Sirsi has named Steven Abram vice president of innovation. Abram, a well-known and popular speaker at many library conferences, was formerly vice president of corporate development for Micromedia ProQuest. He will lead Sirsi’s trend forecasting and product conceptualization.