Three decades ago, investor Dean Williams began a speech with an analogy between physics and investing.

For centuries, Williams said, physics was dominated by Newtonian physics, or classical physics, which taught us that the world moved in predictable ways that were measurable and precise. Give me a ball and a ramp, and I'll tell you how far that ball will roll. All you have to do is measure something now, and it will tell you what will happen next.

But then a new type of physics came around -- quantum physics. It told us that parts of the physical world were more messy and imprecise than we once thought possible. Measuring something now might not tell us what will happen next, because we don't really know what we're measuring, or if merely looking at something is causing the event we're trying to measure.

"What I have to tell you tonight," Williams said, "is that the investment world is a lot more like quantum physics than it is like Newtonian physics. There is just too much evidence that our knowledge of what governs financial and economic events isn't nearly what we thought it would be."

That was brilliant insight 30 years ago, and it still is today. It's brilliant because it's obviously right, but so few investors have caught on to it. 

Take an annual survey by Franklin Templeton Investments. Near the start of each year, it asks 1,000 investors whether the S&P 500 (^GSPC 1.20%) went up or down in the previous year.

Now, we live in the age of CNBC and Yahoo! Finance and iPhone apps, where no one lacks the data to know a simple statistic like whether the market went up or down.

Yet year after year, the survey shows that swarms of investors are utterly clueless:

  • In 2010, 66% of investors said the S&P 500 fell in 2009. Yet it was actually up 26.5%.
  • In 2011, about half of investors said the market fell in 2010. Yet it was actually up 15%.
  • In 2012, 53% of investors said the market fell in 2011. Yet it was up 2%.
  • Just recently, 31% of investors said the market fell last year. Yet it was up 16%.

I've seen a few explanations trying to reconcile this gap. One is that the discrepancy between perception and reality is a good thing, because it shows investors aren't swept up by short-term market noise.

But I think there's a simpler explanation. As Williams might say, investors think that markets are much more like Newtonian physics than they really are.

If you think of markets like Newtonian physics, your thought process probably went like this:

I remember 2009. The economy was losing 700,000 jobs a month. GM went bankrupt. Wall Street neared collapse. Economists made comparisons to the Great Depression. We had an $800 billion stimulus package that didn't do much. The government ran a $1.4 trillion deficit.

Of course the stock market fell. How could it not have?

Same thing for 2011. Do you remember 2011? Greece nearly collapsed. Congress nearly defaulted on the government's debt. The U.S government lost its triple-A credit rating. The phrase "double dip recession" was mentioned in the media more than 10,000 times, according to Google. Of course the stock market declined. How could it not have?

This way of thinking makes so much sense. You just measure what's happening now, and it will tell you what will happen next in ways that are rational and precise. Just like Newtonian physics.

But thinking about markets this way is totally wrong.

People looked around over the past few years and saw a collapsing economy, but from an investment standpoint what they were really looking at were collapsing expectations that set stocks up for a big rally. This isn't intuitive, so you only saw it that way if thought of investing in the chaotic lenses of something like quantum physics. The fact that most investors don't think that way explains why so many missed the rally over the past four years.

Williams quotes physicist Werner Heisenberg, who once said, "Can nature possibly be as absurd as it seemed to us in these atomic experiments?"

Investing and the economy lead us to the same question. We think it should be simple and clean and elegant, but it's really just an absurd slug of confusion and disorder. "One of the tricky things about the subject," wrote New York Times columnist David Leonhardt, "is that almost nothing is certain in the way that, say, two plus two equals four. Economics -- which is at root a study of human behavior -- tends to be messier."

The takeaway is not that there's a way of thinking that will provide you with all the answers. It's actually just the opposite -- that the best we can ever do is have a healthy appreciation for how chaotic and quantum-physics-like markets are. That means taking a simple approach to investing, not taking anyone's forecast or analysis too seriously, not following the herd, and giving yourself a lot of room for error. And perhaps checking to see what the market actually did, rather than assuming what it should have done. 

Check back every Tuesday and Friday for Morgan Housel's columns on finance and economics.