The last decade has felt like a particularly rough one for investors, so I was surprised to learn recently that the S&P 500 (^GSPC 0.02%) has had only one negative year over the past 10 years. That was just one of the many intriguing facts that I discovered by looking at The Callan Periodic Table of Investment Returns.

Callan's table plots the annual returns for key indexes in rank order over the past two decades. The nine key indexes are: S&P 500; S&P 500 Value; S&P 500 Growth; MSCI Emerging Markets; MSCI EAFE; Russell 2000 (RUSSELLINDICES: ^RUT), Russell 2000 Value; Russell 2000 Growth; and the Barclays Aggregate Bond Index.

Among the surprising facts I learned are:

  • The MSCI Emerging Markets Index has led all of the other key indexes in seven of the past 10 years.
  • The Barclays Aggregate Bond Index has been at the bottom of the table in seven of the past 10 years.
  • Small caps (Russell 2000 Index) have outperformed large caps (S&P 500) in seven of the past 10 years.

Just as my pattern recognition faculties began making sense of things, I also learned, however, that:

  • The MSCI Emerging Markets Index came in dead last among all of the key indexes in five of the 10 years from 1993 to 2002. It came in last overall for seven of the last 20 years.
  • Even though the Barclays Aggregate Bond Index came in last seven of the past 10 years, it crushed the S&P 500 by over 42 percentage points in 2008.
  • And while small caps tended to outperform large caps in most years over the past decade, large caps outperformed small caps for five consecutive years from 1994 through 1998.

So what should we make of this somewhat contradictory array of market data?

Perhaps nothing at all. In fact, all of this data might serve as a perfect example of our all-too-human desire to make patterns out of random data. Resisting the urge to make investments based on our pattern-creating tendencies might just save us a lot of money over the long run.

Your most dangerous investment advisor
In his classic book "Your Money and Your Brain: How the New Science of Neuroeconomics Can Help Make You Rich, the investing writer Jason Zweig warns of the dangers of searching for patterns in random data. He writes that, "When it comes to investing, our incorrigible search for patterns leads us to assume that order exists where often it doesn't."

In Zweig's book, Professor Michael Gazzaniga talks of the part of the brain known as the "interpreter," which is constantly searching "for explanations and patterns in random or complex data." Relying on the interpreter in investing is particularly dangerous because we may think we've identified a pattern that is more likely an illusion.

For example, we might look at the Callan Table and conclude that small caps will outperform large caps over the next 10 years because of their record of relative outperformance over the past 10. Or we may determine that it's not worth diversifying our portfolios with bonds, since they've consistently underperformed the other indexes in most years over the past decade.

Those are just two very simple examples for illustration purposes, of course. The real takeaway from looking at the Callan table is that we shouldn't leap to conclusions when interpreting market data. Indeed, Zweig notes that the problem with pattern recognition and the investor brain is that it's unconscious, automatic, and uncontrollable.

We may conclude, for example, that investing in the MSCI Emerging Markets over the next decade is a can't-miss-idea solely because the index has been on such a roll over the past decade. Such an approach would probably be very unwise, however. A much better reason to invest in emerging markets would be because your extensive research leads you to believe that those countries will perform well going forward due to their strong economic fundamentals.

For me, the Callan table really drives home the importance of both dollar-cost averaging and diversification within your portfolio. I don't think most investors are capable of timing the market. And I suspect the vast majority will never know which investing style or asset class will be in or out of favor at any given time. Relying on our "interpreters" to make those decisions will likely end very badly.

Don't get burned
Zweig notes that the Ancient Scythians would burn to death any soothsayer whose predictions failed to come true. Thankfully, we don't have to worry about such extreme consequences if we decide to predict the future based on a snapshot of historical market data. We could lose a lot of money, however, which alone should be enough to discourage us from pursuing such a dubious strategy.