"Be fearful when others are greedy and greedy only when others are fearful."
-- Warren Buffett, 2004 letter to shareholders.

The Oracle of Omaha has a great many quotes that have worked their way into our investing lexicon, but none more so than the one above.

Sadly, it's also the most misunderstood quote of Buffett's time. Below, I'll offer up some context behind Buffett's quote and give some hard evidence to add perspective.

Poor investor results
When Buffett wrote his 2004 letter to shareholders, he wanted to point out that though the market had produced incredible returns over the previous 35 years, the individual investor had largely missed out on such gains. He identified three culprits: fees associated with investment management or frequent trading, making decisions based on fads instead of fundamentals, and a "start-and-stop" approach to investing that leads to poor entry points.

Buffett wraps up the paragraph by saying: "Investors should remember that excitement and expenses are their enemies. And if they insist on trying to time their participation in equities, they should try to be fearful when others are greedy and greedy only when others are fearful." [Emphasis added.]

Though there's certainly truth to Buffett's assertion -- that prices are low and attractive when others are bearish and vice versa -- he never meant for it to become the mantra of the individual investor.

His statement is more akin to a health guru stating, "You shouldn't drink soda at all, but if you insist, stick to diet soda." There's no doubt those who abstain from the beverages are -- all things being equal --- better off, but legions of diet drinkers are given solace from the statement.

Want proof?
Luckily for us, there's a way to back-test how effective Buffett's strategy would have been in the past. Starting in 1987, the American Association of Individual Investors has taken a sentiment survey on how investors think the market will fare over the next six months.

The survey is taken every week, but in order to smooth out aberrations, let's focus on the eight-week rolling average. The long-term average is for 39% of investors to be bullish on the market. We'll say that investors are being fearful when that average is one standard deviation below the norm, and say that they're greedy when it's one standard deviation above the norm.

If, starting in 1988 and ending in 2012, the market timer were to only buy stocks when investors became "fearful," and sell them when they became "greedy," here's what the past 24 years of the S&P 500 (INDEX: ^GSPC) would look like.

Source: Yahoo! Finance, AAII.com, author's calculations. Green represents "buys," red represents "sells."

In the end, taking this approach would have yielded 415%, versus the market's return of 409%. Indeed, following the quasi-Buffett method of buying into the market at certain times slightly outperformed the market.

Are we missing something?
Remember, Buffett said the average investor underperformed because of both expenses and a start-and-stop approach.

Even if we give the market timer the benefit of the doubt on expenses and taxes -- they have zero commissions and all of their holdings are in a tax-advantaged Roth account (even though they didn't exist back in 1987) -- we are forgetting about one key thing: dividends. Every time the market timer sells and sits on his money, he is missing out on the dividends offered up by companies.

When we factor dividends into the equation, we see that the long-term buy-to-hold investor outperforms the market timer by more than 100 percentage points.

Source: YCharts, AAII.com, author's calculations.

But we're still missing the point
Without a doubt, for the average investor who has better things to do than spend all day crunching numbers on the computer, buying and holding an index fund is an excellent choice. But if we were to take Buffett's advice one step further, we could really juice our returns.

Buffett has said several times that the key to investing success is buying companies with great economics in their favor at reasonable prices, and holding them for the ultra-long run. How would an approach like that work? Take a look at how the book value of Buffett's Berkshire Hathaway (NYSE: BRK-B) has appreciated over the same time frame we've been investigating.

Source: YCharts, AAII.com, author's calculations, Berkshire Hathaway annual letter.

The takeaways here are three-fold. First, though valuation matters, trying to time the market is simply a waste of time and energy. Second, dividends matter -- the second chart should make that crystal clear. Finally, buying with an eye on fundamentals and a decades-long future can give you a huge advantage over Wall Street traders.

If you're interested in starting to build your retirement portfolio with quality companies, check out our new special free report: "3 Stocks That Will Help You Retire Rich." I'll give you a hint and tell you that Buffett's Berkshire is actually one of these three, but to find out what other two companies we hold in such high esteem, you need to get a copy of the report today. It's absolutely free!