Illustration: Chris Sharp

Financial analysts and investors have long sought to predict the stock market’s course. They pore over corporate earnings and ponder the price of commodities such as oil and pork bellies. One seer, back in the 1920s, proposed the length of women’s skirts as an indicator, observing that rising hemlines seemed to augur higher share prices.

Roughly two years ago Alan Marcus and Hassan Tehranian, finance professors at the Carroll School of Management, set out to find a better gauge of the market’s future movements. Like many economists, they believed that most investors, particularly professionals, trade on the basis of information—that a stock’s price, at any particular moment, reflects the best available data about its prospects. In that understanding, emotion had little place.

But recent research in economics and psychology has shown that homo economicus isn’t so coldly rational after all; people routinely exhibit all kinds of biases in economic decision making. They’re overconfident. They give too much weight to current events. They fear losses more than they prize gains. Perhaps, Marcus and Tehranian reasoned, emotions do influence stock picking, and investor sentiment can help to predict the market’s movements.

The product of their investigations is a new index of individual investors’ optimism and pessimism. Based on data stretching back some 40 years, it is what professionals call a contrary indicator, moving in the opposite direction from the stock market.

The challenge was to show the link between sentiment and stock prices. Marcus, Tehranian, and a third colleague—Professor Roger Edelen, now at the University of California, Davis—chose to work with the quarterly Z.1 reports of the U.S. Federal Reserve, which track the movement of all funds into and out of stocks and other securities. The three researchers hypothesized that where people invested their money (or didn’t) showed how they felt about the future. Buying stocks implied optimism about their prospects—that is, the belief that share prices would rise—and selling them, pessimism. Their results appear in a paper, “The Relative Sentiment of Retail and Institutional Investors,” which has been submitted for publication.

The index is a relative measure. It doesn’t tell whether individual investors are optimistic in an absolute sense, just whether they’re more optimistic than institutional investors such as pension funds and college endowments. The index focuses on individuals because they can do what they want with their money; institutions, in contrast, are often restricted by investment policies that, say, limit the amount of stock they can own to prevent extreme losses. Using the Federal Reserve data, the researchers determine, each quarter, whether retail (non-institutional) investors are holding a greater or lesser share of the total. Their reasoning is that an increasing retail share reflects growing optimism.

Using money flows was a critical factor. Most sentiment indicators have relied upon either surveys—asking people how they felt—or what researchers call proxies. A proxy is something that seems to move in step with the phenomenon a researcher wants to measure. Hemlines are a proxy.

In this new index, Marcus says, “We’re looking at a direct measure: What people do with their money. . . . They can say whatever they want about their optimism. But whether they buy or sell stocks shows what they really believe. It’s actions speaking louder
than words.”

Once the professors established their method of tracking the market’s mood, they mapped their index’s ups and downs against stock market returns for the period from 1967, when the Fed first tracked the flow of funds being studied, to 2008. What they found surprised them: Their measure had predictive power. Run-ups foretold drops in stock prices and vice versa.

When the index was above its 75th percentile value, for instance, signaling high investor hopes and inflated share prices, the subsequent three-month stock-market return averaged only 4.5 percent annualized. But when it sank below its 25th percentile, indicating pessimism and depressed prices, the next quarter’s stock market return averaged 26 percent annualized.

Some folks will undoubtedly wonder if they can use the new index to decide when to move into and out of stocks. Marcus warns against that. “We update it only quarterly, and it moves very slowly,” he explains. “It’s one piece of a big mosaic. It’s a way of thinking about what current market conditions are, broadly speaking.”

Tim Gray is a writer based in the Boston area.

Read more by Tim Gray