We know a lot about where Warren Buffett thinks you should put your money. But he also has a very strong opinion about where you should not keep your money.

The massive risk
When discussing the broader stock market in his 2012 letter to Berkshire Hathaway (NYSE: BRK-A)(NYSE: BRK-B) shareholders, Buffett noted (emphasis added):

Since the basic game is so favorable, Charlie and I believe it's a terrible mistake to try to dance in and out of it based upon the turn of tarot cards, the predictions of "experts," or the ebb and flow of business activity. The risks of being out of the game are huge compared to the risks of being in it.

It's a fascinating quote to consider. All too often the perceived risks associated with the stock market become front-page news, and images of our life saves being wiped away haunt our thoughts.

Or perhaps, because the market has been on such an incredible run recently, there's concern that the opportunity has passed to make money. As Isaac Newton once said, "What goes up must come down," so investors might think now it's best to keep their money out of the market altogether. 

But this too is incorrect, because even despite the peaks and valleys, with dividends included, from 1965 to 2013, the S&P 500 delivered an annual gain of 9.8%. What went up, has not come down.

And missing out on those long-term gains is what Buffett believe is the greatest risk to investors.

Saving for retirement means not spending every dollar we earn, but also investing money into the stock market, and more specifically low-cost index funds that mirror the returns of a certain sector of the market. As Buffett tries to tell us, trying to jump in and out of the market is a dangerous game.

Consider this, investment management company Invesco noted that $1,000 invested into the S&P 500 at the end of 1927 would be worth $8.1 million by 2012. But if you missed just the 10 best days over that period -- which spanned more than 20,000 trading days -- your return would stand at $2.6 million. Put differently, by missing just 0.05% of the days, you'd end up with 67% less money.

Of course, you would end up with more if you missed the 10 worst days over that time frame. But, as shown below, the problem there is that the best and worst days are almost always near each other.

Source: Invesco, The Tale of 10 Days.

More recently, J.P. Morgan Asset Management revealed another dramatic example.

$10,000 invested at the end of 1993 would be worth nearly $60,000 by 2013. Yet missing the 10 best days would cut the value of the investment by half:

Source: J.P. Morgan Asset Management, Guide to Retirement 2014 Edition.

Don't just take Buffett's word for it, look at the evidence -- this is not a game you want to be on the sidelines for.

The key takeaway
Does this mean we should blindly pour our money into shares of specific companies? Absolutely not. Buffett has long cautioned the dangers of attempting to pick "winners and losers," and instead highlighted the true value of low-cost index funds as places to park our money.

So listen to Buffett, have a retirement plan, stick to that plan, and stay in the game -- you'll be glad you did.