Photo credit

Of all the traits needed for success -- hard work, intelligence, creativity -- the most overlooked is luck.

Bill Gates went to one of the only high schools in the United States that had a computer. Warren Buffett was born in a capitalist country. Donald Trump had a wealthy father. All of these guys earned their success, but in slightly different circumstances outside of their control, things could have turned out far differently. To put it another way: People with just as much ambition and intelligence are broke nobodies only because they were born in a different time and place.

This same thing happens with all investors. And there's an easy way to prove it.

The long-term history of stocks tells us that:

  • Stocks earn a reasonable return after inflation.
  • The longer you stay invested, the better the odds of achieving that return.

Here's what this looks like. This chart shows the best and worst annual returns stocks generated over the last 141 years based on different holding periods: 

Source: Robert Shiller, author's calculations. 

This chart, which I've used before, is valuable in helping investors learn what to expect from stocks over time. But it has two flaws.

One, it assumes you start with a lump sum investment and hold it for 10, 20, 30 years. Nobody does that. They invest in little bits over time, ideally dollar-cost averaging every month or every quarter.

Two, the difference between earning 3.2% per year and 10.2% per year over 20 years might look small, but it adds up to an enormous disparity over time.

I used data from Yale economist Robert Shiller to show what would happen if someone invested $500 a month, every month, in the S&P 500 throughout every possible 20-year period from 1871 to 2013. Here are the top three and bottom three results (all of these numbers are adjusted for inflation and dividends): 

Source: Robert Shiller, author's calculations. 

These people invested the exact same way, with the exact same amount of money, for two entire decades. Yet one group ended up with seven times as much money. Not because they were smarter, but because they were born at a different time and saved during a different era. They got lucky. 

Stretching this out to 30 years shows something similar. The lucky investors finish with five times as much money as the unlucky ones:

Source: Robert Shiller, author's calculations. 

What amazes me is that these hypothetical investors would be considered some of the smartest around, investing steadily every month no matter what the market was doing, for decades on end. Doing this is emotionally taxing, and few investors can keep it up over time. In the real world, investors are more likely to buy after stocks have boomed, and to sell after a crash -- which devastates returns. Yet even with hypothetically perfect behavior, the difference in results between investors born in different generations can be the difference between no retirement and a lavish retirement. And it's mostly a factor of luck.

I found two takeaways from this.

One is the importance of diversification. Having a mix of stocks, bonds, cash, and real estate can reduce the risk of having all your assets in one investment that suffers a bad decade. A lot of the results in these charts can be explained by a simple factor: stock valuations at the end of the period were either in a bubble or a bust. Diversification can offer protection from extremes.

Two, not needing your money at a specific period of time, even if it's decades in the future, can be vital. When people say, "I need this money at 65," they're assuming the market is going to cooperate with that goal on their 65th birthday. But it might not. Having a flexible investment outlook helps put the odds of success in your favor. That could mean the ability to work a few years longer than you anticipated, or having enough liquid funds to tap for years before needing to withdraw from your stock portfolio.

In all the red scenarios above, holding stocks for just a few years longer would have dramatically improved returns. Without the ability to hold out a few years for a recovery, your experience as an investor relies on luck. And good luck with that.

Check back every Tuesday and Friday for Morgan Housel's columns on finance and economics.