When we see a great bargain, we just can't help ourselves. We want to buy. I know that I'm always ready to call a stock's bottom -- that moment when everyone else has given up on it -- and invest a bunch of cash, earning massive returns as it comes back from the dead.

But time and again, my attempts to time the market backfire. What's more, even the experts can't pull it off, as a recent brief from the Center for Retirement Research at Boston College shows.

Frustrated man watching investments go down.

Image source: Getty Images.

The researchers looked at target-date funds, which are designed to gradually reduce risk as investors age by changing the portfolio's asset allocation. For example, look at the current allocation of these two Vanguard target-date funds, which hold funds for people planning to retire in 2025 and 2060, respectively. 

 Vanguard 2025 FundVanguard 2060 Fund
Stocks 65% 90%
Bonds 35% 10%

Source: Vanguard.

As 2060 gets closer, investors in the Vanguard 2060 Fund will see their stock/bond mix shift toward a more conservative allocation, like the 2025 fund's today. This gradual reduction in riskier assets is known as the "glide path."

But when target-date fund managers see an opportunity to time the market, some will temporarily deviate from the glide path. After all, their expertise is why we pay them the big bucks – they're the people who should be able to identify a short-term opportunity to profit.

The beauty of the Center for Retirement Research's study is that it looks specifically at fund managers who try to time the market, isolating that market-timing impulse so we can understand what happens when fund managers make that decision.

Timing the market fails

The researchers found that target-date funds that deviated from the glide path underperformed their peers by 14.1 basis points (or 0.141 percentage points) each year when returns are weighted by age of fund. By trying to time the market and seize opportunities, they lost money.

Now, that underperformance wasn't too severe: If you invested for 30 years in two funds, one of which underperformed the other by 0.141 percentage points annually, you'd see a total difference of about 3.8% in their returns. But the difference was statistically significant, and it shows that even the professionals lose when they try to time the market.

Individual investors, incidentally, do a lot worse. According to the 2016 Dalbar Quantitative Analysis of Investor Behavior Study, equity mutual fund investors lost money even while the S&P 500 posted a small gain in 2015. And the gap in 20-year annualized returns was even bigger:

 2015 ReturnAnnualized Return, 1996-2015
S&P 500 1.38% 8.19%
Average equity mutual fund investor (2.28%) 4.67%

Source: 2016 Dalbar Quantitative Analysis of Investor Behavior Study.

A hypothetical investor with $100,000 would watch their investment grow to $249,000 (before taxes) over 20 years at a 4.67% annual rate of return. That same investor, earning, 8.19% per year, would end up with $483,000 -- nearly twice as much.

Why it doesn't work

Curious why investors are so woefully underperforming? I'll quote directly from the study (emphasis mine):

The data shows that the average mutual fund investor has not stayed invested for a long enough period of time to reap the rewards that the market can offer more disciplined investors. The data also shows that when investors react, they generally make the wrong decision.

They make those decisions because of a number of returns-killing biases, and there is a big psychological issue that underpins many of these biases: apophenia, or the tendency to see patterns and connections where there are none. While conspiracy theorists are the most well-known apophenia sufferers, plenty of investors fall into the same trap.

Technical day-traders are the quintessential apopheniac investors, as they stare at price charts and try to divine the future based on perceived patterns in the market that they think they've seen before. The results of this are clear: A 2010 Berkeley study on day traders in Taiwan found that the average trader lost 0.25% of their portfolio daily, and 93% quit day trading within five years.

How many times have pundits attempted to use historical indicators to predict when the Fed would increase interest rates? (I remember people predicting "imminent" rate increases in 2013. The Fed didn't start raising rates until late 2016.) Ever heard of the so-called "January effect," which uses January's market returns to predict which way the market will go for the rest of the year? It's basically a coin-flip, with January "predicting" the wrong market direction four of the past 11 years.

Even professional investors are clearly not immune to this delusion, as many target-date fund managers try to time the market -- and underperform as a result. The fact is that we humans can't help but search for patterns and meaning everywhere we look, and that leads us to making dumb decisions in investing and elsewhere.

The simple solution

If you're going it alone with your investments, the best way to resist your urge to time the market is dollar-cost averaging, which means regularly adding a fixed amount of money to a basket of investments no matter what their prices are doing. While dollar-cost averaging isn't a perfect solution, it's one that can work broadly for most people. Note that I'm not suggesting that you can't beat the market -- I firmly believe that you can -- but merely that you can't time the market. No one can predict what will come next. So long as you've done thorough analysis on the stocks you're buying and are confident that they will deliver outsize returns, dollar-cost averaging is the simple way to control for your tendency to try -- and, most likely, fail -- to time the market. Some index funds can be a real asset here, as they are set up to passively follow an index no matter what's going on -- meaning fund managers generally aren't involved in market-timing. A basket of five or so broad index funds can provide the diversity your portfolio needs without you needing to track a lot of individual stocks.

If you are interested in a target-date fund, there are a few different ways to figure out whether your fund manager tries to time the market. Your fund prospectus should have some information on your manager's overall allocation philosophy, which will give you some insight. The prospectus should also share the fund's annual portfolio turnover, with a higher number meaning more investments were bought or sold in the last year. A lower number likely means that the fund is more passive -- or, in the case of a target-date fund, content to stick to the glide path. While there isn't a hard-and-fast rule for turnover, I personally prefer funds with annual turnover of less than 20% (The Vanguard S&P 500 Index Fund, for example, has an annual turnover of 4%). Also consider the expense ratio, as a higher expense ratio is usually (though not always) an indicator of a more active fund that's trying to time the market. I seek to invest in funds with expense ratios under 0.5%, both for greater passivity and because I like saving money on fees where possible.

It's only human to try and game the system by timing the market. But consider this: Nobel Laureate William Sharpe found in 1975 that a market timer would have to be accurate 74% of the time to beat a passive portfolio. Even a slight outperformance probably wouldn't be worth the energy -- but given that even the experts generally fail at it, market timing just isn't an investing solution worth pursuing.