Why Markets Aren't Efficient

Remember when everybody thought that markets were all-knowing? Before the financial crisis struck in late 2008, the reigning dogma in economics was the "efficient-markets hypothesis," an idea popularized by Eugene Fama that enjoyed exalted status for more than three decades. EMH, as economists call it, posits that markets reflect all available information, that investors are rational, and that prices are stable. While anyone without a Ph.D. or an M.B.A. probably immediately recognized the flaws in such rigid thinking, these notions once seemed self-evident to academics and investors. The economists still vigorously defending them today sound like alcoholics denying they have a problem.

"The economics profession went astray," Nobel Prize winner Paul Krugman proclaimed this year, and EMH helps explain why: it was the sharpest example of the way economists sacrificed truth in favor of beautiful, quantifiable theories that helped Wall Street whiz kids build computer programs to "predict" the market. Only when those programs led to financial products that helped blow up the world did the flaws in the theory become clear to all.

A few people saw the trouble coming. All the way back in 2001, Joseph Stiglitz shared the Nobel Prize with two others for poking holes in the theory. Behavioral economists, too, have shown time and again that humans can act irrationally—by falling prey to the herd mentality, for example. But those findings have been somewhat scattershot, with no principles to show how to apply them in the real world. Hence the quest for a new, grand theory, one that patches the holes in the efficient-markets idea and integrates the wisdom of the behavioralists. Andrew Lo, an economist at MIT, thinks he has just the solution. Lo is the foremost proponent of something called the adaptive-markets hypothesis, a way of looking at the markets through the prism of evolutionary biology. His theory builds on the work of Stiglitz and others, and can be explained simply enough: the economy and financial markets are an ecosystem, with different "species" (hedge funds, investment banks) vying for "natural resources" (profits). These species adapt to one another, but also go through periods of sudden mutations (read: crises). Lo proposed the idea in 2004, but it has taken on new import in light of the financial crisis. Investors and academics now soberly debate its implications in the pages of the Financial Times and Harvard Business Review, and the Federal Reserve has even used it to explain the behavior of foreign-exchange markets.

Rather than assuming markets always know best, AMH builds on an understanding that they sometimes don't. The trick is knowing when irrational behavior will lead to a bubble or even a global crisis. Lo believes the secret lies in studying the "ecology" of the markets. Just as biologists catalog species and chart their fortunes over time, regulators and policymakers should categorize the market's many players. That means identifying the various hedge funds, pension funds, and other participants in any given market, and learning what kind of strategies are popular at a particular moment in time. "What is their biomass? How are they going to interact with each other?" Lo asks. Incredible as it seems, regulators don't collect this kind of information, because, according to EMH, everyone responds to incentives in the same basic way. But the adaptive-markets hypothesis holds that investors' behavior can vary depending on their psychology at any given moment. If their actions were tracked over time in a wide variety of settings, says Lo, "we could develop an extraordinarily good sense of how the markets behave." So far, however, it's been tough to get financial authorities to do this because high-level investors strongly resist divulging information about their strategies.

While it's unclear yet whether Lo's work will help predict the next bubble, he has already done what no one else has dared to do—propose a successor to the vaunted, but flawed, efficient-markets hypothesis. Of course, one thing his grand, unified model doesn't take into account is what should perhaps be the greatest lesson of the financial crisis: that we should forgo grand, unified models. "A humble point of view within a lesser theory is often better than something more high powered," says Tyler Cowen, a respected economist at George Mason University. In other words, it may be better to embrace the world's complexity rather than try to shoehorn it into another faulty but comprehensive paradigm. But Lo is open to that possibility too. "Ideas percolate," he says. "Through natural selection, the best ones survive."

Join the Discussion