An Experimental Mind

On the Hewlett-Packard campus in the heart of Silicon Valley, scientists in the Decision Technology lab are using experimental economics to predict the future of the company. In one experiment they create minimarkets, with real cash rewards, that allow employees to bet on future sales and revenue. So far these internal futures markets have done a better job of forecasting than polls of top executives. The disruptive implications for the corporate pecking order, and not only at Hewlett-Packard, aren't lost on anyone. In another experiment, students from nearby Stanford University act out the roles of HP executives, retailers and suppliers, testing new marketing plans or sales strategies before millions are spent to roll them out in the real world. The laboratory subjects sit at terminals, playing their parts in a virtual corporate world that can grow as elaborate as the urbanscapes of SimCity, the computer game. HP calls the program SimBusiness. HP senior scientist Kay-Yut Chen calls the Decision Technology lab "a wind tunnel for business."

Experimental economics is the brainchild of an iconoclastic academic named Vernon Smith, who had an epiphany a half century ago that has since "changed the direction of economic science." Those were the words used by the Royal Swedish Academy of Sciences when it awarded Smith the Nobel Prize last year. With his silver ponytail and heavy Native American jewelry, Smith has shaken up establishment economists by proving that their theories are best tested not in their heads, where they are most at home, but with live experiments involving real people.

Even before the Nobel, Smith's ideas had spread through his preachings and those of his acolytes to transform the thinking of government, producing some of the most creative policy innovations of recent decades. Since the early 1970s experimental economists have changed the way the U.S. government sells wireless spectrum, packs a space probe, regulates the price of gas, allots airport landing slots and battles smog. They hit the headlines this summer as the inspiration behind the Pentagon's "terror-futures market," which was killed for being politically tone-deaf before it could prove its ability to forecast the next big Qaeda attack.

From its beginnings, experimental economics was greeted as something radical, or just plain ignored, perhaps because of its deceptive simplicity. But by the 1990s experimental economists began setting up as advisers to big business, finding more efficient ways to organize cross-country delivery trucks, conduct real-estate auctions, even forecast printer sales and Oscar winners. The experiments had become much more than lab games. Economists were entering the nitty-gritty of practical business decisions more deeply than ever before.

Now the Nobel seal of approval is breaking down any lingering resistance to Smith, who delivers his ideas in a Kansas twang. Big companies including Sears and IBM are using experimental economics to solve business problems. HP has parlayed the success of the SimBiz lab into the seed of a new consulting business. Among its first clients is Ford, which discovered in the HP lab that while tough times make consumers risk-averse, that means they are actually more willing to buy their leased car when the lease is up, rather than pay less for a different used car. The results could change leasing practices at Ford, which has signed up for more tests. "We think experimental economics will become an extremely powerful decision-making tool," says Ford Research executive Irv Salmeen.

Smith is now much in demand. Asked by the Swedes to ditch his cowboy boots when receiving the prize, he complied, and is now pushing experimental economics as a fix to the problems of everything from endangered wildlife to the antiquated U.S. electric grid. He even has ideas on how to rebuild the Iraqi oil industry (following story). These days mainstream economists are inclined to listen. "Experiments are the way of the future," says Hal Varian, an economist at the University of California, Berkeley. From his new Interdisciplinary Center for Economic Science at George Mason University, Smith mentors a host of academics using experiments in ever more creative ways. One, Kevin McCabe, is blending economics and neuroscience to study brain patterns during decision making.

Smith did not set out to bring down the establishment. Back in 1956, when he was a young professor at Purdue University in Indiana, economists took certain assumptions as fact. One was that markets can't reach "competitive equilibrium," the point where buyers and sellers agree on a set price, unless they are made up of an infinite number of players, each possessing perfect knowledge of the marketplace. In a bout of insomnia one night, Smith concocted a plan to demonstrate this notion by putting his students through a trading game. Buyers were given a maximum price they could pay, and sellers were given a minimum they could demand, for an unspecified good. Classical theory said that with such imperfect knowledge of the market, players could never agree on the best price. To Smith's shock, the unwitting buyers and sellers agreed on the best price --after a few rounds of bidding. Smith was convinced he'd fouled up and kept running experiments to prove the old assumptions correct. He couldn't. The field of experimental economics was born.

It was not well received. Smith had trouble publishing his work and began to drift back to his original field, environmental economics. Fortunately, this didn't happen before he'd taken a few fishing trips with Charles Plott, then a young academic at Purdue. "We'd be fishing, and he'd draw pictures of his work in the sand," says Plott, one of the few peers who would listen to Smith. "Economists tend to become economists so that they can think about big things, like international trade relations and the global economy. The profession simply wasn't prepared to think about these simple ideas."

By the early 1970s, Plott was a professor at the California Institute of Technology, still thinking about those scratchings on the shore of an Indiana lake. He invited Smith to come and work at Caltech, and experimental economics was reborn. While Smith received the Nobel for establishing lab experiments as a tool in economic analysis, the experiments themselves would prove path breaking, too. The theory that free markets work like an "invisible hand" to almost magically allow the best information to come to the fore goes back to Adam Smith, but no one had proved it before Vernon Smith came along. What's more, the experimental economists showed that markets were savvy forecasters of a lot more than the --future price of stocks and other financial goods. From their work sprang projects like the Iowa Election Markets, which have consistently bested pollsters in predicting presidential-election results since 1988.

Smith and others demonstrated that institutions matter--that the way markets and regulations are built has a real effect on economic outcomes. This might sound obvious, but once again, before the experimental economists, nobody had ever tested regulations before putting them into place. In 1975 the U.S. Department of Transportation was considering a new rule requiring barge operators to post their prices, rather than cut deals customer by customer. The government assumed that the new rule would lead to greater transparency and lower prices. In fact, Smith and Plott had shown the year before that posting prices can drive them up, in the absence of competitive pressure, and Plott ran experiments showing that this would happen in the barge case. The department dropped the proposed rule.

The experiments were growing increasingly elaborate, thanks to advances in computers. In southern California in 1994, RECLAIM was launched, an early emissions-trading scheme that allowed many different industries to trade pollution credits, which allow buyers to exceed emission-reduction targets. "Information technology has had a profound effect in my lab," says Plott, who recently tested a global emissions-trading system for the United Nations, using as subjects U.N. delegates working online in their home countries.

In fact, experimental economics is tailor-made for problems too complicated to be worked out by any one person. Consider the combinatorial auction, which was first tested by Smith in the late 1970s. In traditional auctions, traders bid on one item at a time. In combinatorial auctions, they can bid on many items all at once, and those bids can affect each other. In 1992, when scientists squabbling over payload space threatened to delay the Cassini mission to Saturn, NASA held a combinatorial auction to break the knot; the mission launched on time, and under budget.

Two years later the U.S. Federal Communications Commission used a combinatorial auction in the first sale of wireless spectrum, in order to meet the intricate demands of telecom operators. An operator might want to purchase spectrum in Los Angeles and San Francisco, but only if he could also get some in Seattle. If someone else got Seattle, the operator might prefer to switch his bid to Boston and New York. Plott created and tested computer algorithms to represent all possible bids, and the spectrum auctions worked. (In fact, some say they worked too smoothly--early bidding went so high that many carriers were later forced to file for bankruptcy.)

Still, the potential of combinatorial auctions sparked wider interest among big businesses. Sears, the big U.S. retail chain, tapped Caltech academics to revamp its transport systems. Instead of parceling out shipping jobs one at a time, which often left trucks empty on the way to the next job, Sears used a combinatorial auction to let companies bid for all the jobs at once. The practice, which maximizes loads, is now the industry standard.

In 1994 HP started its experimental economics lab, the first in the corporate world. The Hollywood Stock Exchange, which allows people to bet on Oscar winners and sells the information to companies like MGM, was started two years later. A crop of new companies sprang from academic work. Plott cofounded Intellimarket, which now designs auctions for everything from rubber to real estate. Two other Caltechies, John Ledyard and Charles Polk, formed a company called Net Exchange to do combinatorial auctions. The company kept a low profile, until the fiasco over its terror-futures project for the Pentagon.

The idea of a market that could predict where a terrorist might strike was tantalizing to Pentagon intelligence types, at least until some U.S. senators got wind of it and branded the project "immoral." Had the program survived, the technical complications might never have been solved: how to formulate and mathematically represent the proper questions, how to avoid market manipulation. The experimental research on such information markets is still pretty slim, and even an insider like Polk admits, "There's a lot of potential for snake oil in this field."

Still, more and more companies are beginning to explore the field. IBM Research is looking at how experimental economics could help build better online marketplaces. A few weeks ago Microsoft invited Polk to brief researchers on his work. Incentive Markets of Boston says it is working on an internal market that would allow employees of a top-10 pharmaceutical firm to bet on which drugs in the company pipeline will be most successful. NewsFutures, based in Maryland, is building an internal futures market for a major insurance company, which it is also not at liberty to name. What accounts for the secrecy? Some executives think experimental economics can give them a competitive edge. Others are loath to cede decision-making authority to a market of their underlings or a SimBiz game involving college kids. "A lot of managers don't want the information that could come from these markets to be public," says Plott. But the nature of disruptive ideas is that they often prove impossible to suppress.