You're Smarter Than the S&P 500

Guess who held onto Bear Stearns while it fell from $170 a share to $10? You did -- if you own an S&P 500 index fund.

As much as we tend to like index funds at the Fool, the companies that manage indexes make some silly decisions sometimes. S&P had to wait for Google to go from $85 to $390 before adding it to the benchmark. Similarly, during the tech bubble that started in the late 1990s, S&P added highfliers like Qualcomm and JDS Uniphase -- right before they plummeted.

Standard & Poor's finally got around to closing the barn door and removing Bear from the index last month, once JPMorgan Chase (NYSE:JPM) got shareholder approval for its buyout of the beleaguered investment firm. But you'll find plenty of other losers within the widely followed benchmark. Here are just a few among the largest companies in the index:

Source: Morningstar, Yahoo! Finance. YTD = year to date.

As you look at the losses in the S&P 500 -- off more than 9% year to date and more than 11% in the past year -- you have to ask yourself: Couldn't you do better than that on your own?

Ignoring the obvious
At first glance, it seems like it would have been easy to improve on the S&P 500's investing strategy. Rather than simply buying hundreds of stocks weighted by market capitalization, exercising some judgment about exactly what companies you choose sounds like an easy way to earn better returns.

Look at the current top sectors in the S&P 500 and you'll see a perfect example. Even after the huge losses they've suffered recently, financials are still the largest sector in the S&P, at almost 17%. Meanwhile, energy stocks, despite huge gains from components like Chesapeake Energy (NYSE:CHK) and XTO Energy (NYSE:XTO), make up just 13.5% of the index.

A year ago, all you would have had to do to outperform the S&P 500 was to invest more into energy and less into financial stocks than the index did. Easy money, right?

Perfect hindsight
It's easy to look back a few years and think how obvious it should have been that financials would do poorly, while energy stocks were just getting started on a powerful run. As events have played out, early optimistic assessments of how long the credit crisis would last and how bad it would be seem downright silly. Furthermore, the rise of energy-hungry emerging markets like China and India has been going on for years -- their increasing demand for oil and gas, and the corresponding boon for energy stocks, should have been a no-brainer.

But figuring out how long certain sectors will yield superior results isn't quite as easy as looking up past results. Adjusting sector weights is really just another form of market timing. To reap the biggest gains, you have to invest more in a sector near the beginning of a trend and then reduce your allocation to that sector before the trend is over. In practice, you have to ignore both the many skeptics who underestimate a trend's impact, as well as the supporters who end up hanging on too long.

So even if you think you're smarter than the S&P 500, you still might not be able to beat its results on your own. You also need that rare ability to keep a disciplined approach to investing, even during the toughest of times. Without that determination to be greedy when others are fearful, you're better off sticking with the index.

For more on investing and stock indexes, read about:

• How you can beat the market in your sleep.
• What to do if you could make only one investment.
• A guidebook to investing in down markets.