Searching For The Best Engine

Tui Stark is searching for a vacation paradise and can't find it. Googling "snorkeling beaches blue water" turns up listings for scuba diving, real-estate firms, rafting outfits. So Stark, a photography stylist in Needham, Massachusetts, turns to Quintura, one of many upstart search engines, which allows her to focus the results on snorkeling. "The Google results just had too much stuff I wasn't looking for," she says. "I wanted to zoom in on the best snorkeling beaches." And within seconds, Quintura delivers.

That's a bad result for Google, which is more vulnerable than you think. By virtue of dominating Web search—Google draws 60 percent of all searches worldwide, says market-research firm comScore, with Yahoo a distant second at 14 percent and mighty Microsoft limping along at 4 percent—Google has not only become the reigning heavyweight of the online world, but it has also transformed advertising, riled governments and sent tremors through Wall Street. As of last week its stock was valued at $200 billion, more than five times that of Yahoo's, and nearly three quarters of Microsoft's. Now it's threatening to shake up the trillion-dollar corporate-computing and wireless-communications markets.

Despite spending billions trying to diversify beyond the straightforward search offered on its stripped-down, almost childlike home page, Google reaps about 60 percent of its outsize revenues and more than 80 percent of its profits from ads on that page, according to analysts' estimates. That means the company's success continues to hinge on the dominance of its simple search. There are no guarantees that its dominance will last. It is threatened by a massive worldwide effort to build a better search, involving giant high-tech rivals, governments in Europe and Asia, and hundreds of tiny start-ups founded by academic wunderkinders much like Sergey Brin and Larry Page, the Stanford graduate students who founded Google in 1998. And it's also dependent on an online public that may make up the most fickle market in history, an audience whose interests are already showing signs of wandering outside the search box.

Google may well be able to continue its charmed life by holding onto its search lead and getting its non-search businesses to kick in more profit, and Wall Street is certainly betting that way. But the computer world has a way of bringing seemingly golden brands down to earth with surprising speed, as Lotus, Novell, AOL and other firms have discovered. It's not farfetched that five years from now we may wonder why everyone thought Google was such a big deal. "Google has won the first stages of the Web-searching race," says Trip Chowdhry, an analyst with Global Equities Research in San Francisco. "It won't win the next one."

History shows how quickly search leaders can lose their way. The race kicked off in 1995, when researchers at Digital Equipment Corp. (remember them?) figured out how to store the words on Web pages as an index that lent itself to lightning-fast searches. The resulting AltaVista search engine quickly became a favorite home page for early Web users, and seemed destined to rule search. But in 1998 word started getting around about a new search engine from a tiny company with a goofy name that sometimes returned more-useful results, and by 2000 Google was the search engine to beat. Yahoo, with a stunning lack of foresight, put Google's search box on its home page that year, burnishing Google's reputation with Yahoo's tens of millions of users. Microsoft, caught napping, wouldn't even enter the search-engine race in earnest for another three years. When Google tweaked its business model by linking ads to searches and charging advertisers only when searchers clicked on them—an approach it copied from rival online marketing firm Overture—it converted its search box into a money machine. Right now that machine is producing $15 billion a year, of which almost $4 billion is profit.

If Google has been able to crush its search competition, it's not because it has perfected the art and science of Web searching. Far from it. Google is what the industry calls a "second-generation" search engine. First-generation engines like AltaVista found Web pages containing words that matched the user's search words. Google's innovation was to further rank a Web page by the other pages that link to it, on the somewhat shaky assumption that if a page is much-linked-to, it must be useful. Charles Knight, an analyst who runs the AltSearchEngines Web site, notes there's a plethora of good ideas for what a third-generation engine might bring to the party, and no shortage of companies trying to prove those ideas. "Each has shown they can do some aspect of a search better than Google can," says Knight.

Yahoo, for one, has been frantically working to leapfrog Google. One new feature of its engine provides search-term suggestions that pop up as soon as you start typing your query—a possible antidote to the frustrating process of having to keep repeating a search with different terms in order to find helpful results. (Google reminds you of searches you've previously typed in.) Another offers shortcuts to following up on certain types of popular searches. Typing in a movie title, for example, brings up a trailer and local showtimes; typing in "restaurants" and a city narrows down the choices by neighborhood, cuisine or popularity. More is coming, says Vish Makhijani, head of search for Yahoo. "We'll know when you're ready to make a reservation versus when you're just doing research, and we'll let you make the reservation right there on the search page," he says.

Microsoft, too, is eager to provide new ways to merge its Windows Live Search with other online and PC-based tasks. So far the company hasn't taken advantage of the dominance of Windows to drive search traffic its way, but that will change, says Microsoft's search chief, Brad Goldberg. "We've just begun integrating search in a meaningful way with our assets," he says. "We're working on ways to capture what the user is doing and carry it into the search experience." In theory, that could mean a Microsoft search on "Coke" would give an accountant financial information on the Coca-Cola Corp., while a student writing a term paper on health and diet might get the nutritional rundown on a can of soda.

In fact, the biggest competitive hurdle for Yahoo and Microsoft is not that their searches don't work as well as Google's, but that people just don't try them as often. According to a recent Nielsen/NetRatings survey, the gap between Google, Yahoo and Microsoft narrows when you look at the percentage of users of each site who keep returning—79, 69 and 65, respectively. A University of Michigan study released in August shows that Yahoo passed Google in customer satisfaction in the past year.

Google has already been relegated to also-ran status in several key markets worldwide. It gets less than 2 percent of queries in Internet-happy South Korea, and 17 percent of the queries in China, the world's most important emerging online market. The company has also been trounced by local competition in Russia. Google dominates searching in Western Europe—82 percent of queries come its way in Germany—but the German and French governments plan to put up $165 million and $122 million, respectively, for search-engine research. In Japan, not only is Google running behind Yahoo, but the government is reportedly pumping some $125 million into local search efforts. Meanwhile, notwithstanding rumors of a forthcoming phone, Google hasn't yet established leadership in the mobile-phone search market, expected to be lucrative.

Yahoo, Microsoft and governments aren't the only ones seeking a cure for Google envy. In 2005 and 2006, venture-capital firms injected $350 million into 79 search-related start-ups. Knight tracks no fewer than 1,000 search contenders, mostly U.S.-based, that have something to recommend them. Among the features that he and other experts believe might be hallmarks of a third-generation search engine:

Word smarts. Some search engines, like Hakia, the forthcoming Powerset and Sydney-based Lexxe, are trying to go beyond matching your exact query words—they seek to get a sense of what you're looking for and pull up the best pages based on an understanding of their content. "In most cases the document you want won't contain all your search terms," notes Rohini Srihari, a University of Buffalo computer scientist and CEO of Janya, an Amherst, New York, company specializing in searching for counterterrorism leads. "And if you're looking to discover who or what has suddenly become a hot topic, you won't even know what search terms to use." A smart search engine might know that when you plug in "Paris," "Tokyo" "New York" and "hottest restaurants" that you're looking for popular new restaurants around the world.

Editing. No matter how clever a computer program, it will never match a human brain for determining quality and relevance. Some new search engines, including Mahalo and ChaCha, rely in part on human editors or guides to pre-cull the most relevant pages for some searches. You'll probably get more select results than on Google—but only if your search terms are among those the editors have explored.

Focus. Google searches everything—but you don't want everything. You'll actually get more relevant results with a search engine that indexes a much smaller number of pages, as long as the pages are on-topic. Trulia searches out homes for sale, Healthline lets you plug in symptoms to track down possible causes and treatment, Globalspec's searches are aimed at industrial engineers, offers pictorial product searches, and Spock specializes in information about people.

Guided queries. It's hard to guess which search terms will do the best job, but some search engines help by suggesting terms, as do Yahoo and a start-up engine called Accoona, or by grouping results into categories that focus on the desired topic, as do and Clusty. Type "spears" into, for example, and it will suggest you steer the topic in either the pointy or pop direction; Google just mixes them up. A number of cutting-edge engines, including France's KartOO, KooltTorch and Quintura—founded in Moscow and now based in Virginia—display the categories in graphic maps that visually suggest which categories are likely to be the most useful.

Community. NosyJoe, Squidoo and Sproose allow other users to help determine which pages are most useful, cutting down on the often irrelevant and spam-ridden results that come up via Google's link-counting approach. Wikia, which has ties to the online, everyone-can-chip-in encyclopedia Wikipedia, is working on a search engine based on user contributions, and the Web-page bookmarking service, bought by Yahoo in 2005, allows searching through everyone else's labeled bookmarks to find relevant pages.

Right now all these underdog search engines (except, the No. 4 search site) have a combined share of less than 5 percent of all queries, according to Knight. But even if one or more of them starts to gain traction, does Google really have to worry about being bested by some obscure search engine, given its longstanding, widespread popularity? After all, Microsoft continues to dominate software, in spite of persistent claims that better alternatives like Apple and Linux are out there. Google's dominance, however, is different from Microsoft's. The costs of dumping Windows can be intimidating, between setting up new hardware or software, retraining and lost productivity. What's to keep someone stuck on Google? "The moment someone proves themselves better than Google, people will switch in a heartbeat," Srihari says. Just ask anyone who was at AltaVista in the late 1990s.

Google isn't waiting around to be AltaVistaed. Its smaller challengers can't hope to match the company's massive investments in computing infrastructure, said to include more than 450,000 servers. So be prepared to wait an annoying three seconds or so for results on some of the wanna-be search sites, compared with Google's blink-of-an-eye speed. And with $12 billion cash on hand, Google can buy hot companies that pose a threat, much as it plopped down $1.65 billion last year for YouTube, whose video search crushes Google's popularity. "Google was buying tiny search companies at the rate of two per week at one point," says Knight.

Even $12 billion and the billions more Google could borrow wouldn't buy all the world's search competition. The performance gap won't be hard to narrow for a hot new company freshly fueled by investors. In the end, Google has to have a better search to stay on top. Thus its army of software engineers is looking at every wrinkle in search, insists Google's research director, Peter Norvig. "I guess we're paranoid," he says. They've already injected several new technologies into its search—for example, results take into account results you've clicked on in the past, provided you've signed up to have your searches tracked. You can type in your query in plain English, get suggestions for search-term refinements, or do any of more than 40 specialized searches, including movies, government Web sites, patents, airline flights and human faces. Google just doesn't advertise any of these features, or make them plain. Although it's clear Google is capable of plenty of search innovation, there's a reason the company sometimes acts as if its hands are tied when it comes to implementing next-generation techniques. "People don't want radical change from us," says Matt Cutts, head of search quality at Google. "Our biggest task is ensuring simplicity."

It's true, most mainstream searchers do tend to value the stripped-down, no-brainer elegance of a thin box that takes a few words and delivers straightforward results. Given that a growing number of queries are being funneled to alternative engines, there are clearly plenty of power searchers willing to accept a little complexity in return for better results. It wouldn't take a smash-hit new search engine to steal Google's thunder; the damage could take the form of a slow leak of searchers to a variety of engines that each have some special appeal. Another threat to Google may be online social networking sites such as MySpace and LinkedIn. "We'll likely see dozens or hundreds of specialized search engines that collectively chip away at Google's dominance," says Brant Bukowsky, founder of Plus1 Marketing, a search consultancy.

Last quarter, Google raked in $925 million in profit, 28 percent more than the same quarter last year. The game is still Google's to lose. Even Stark, who resorted to Quintura to find her snorkeling beach, still makes Google her first stop when she needs to track down a Web site. What, after all, would Google have to fear from a tiny company with a goofy name that sometimes returns more-useful results?

Join the Discussion