Can Individual Investors Beat the Market?

That's a good question -- and the title of a now-famous academic paper. For most investors, the answer is an emphatic "no." At least, not by any meaningful degree ... but more on that later.

Fortunately, you have options. You can buy an index fund, something we recommend for a good chunk of your portfolio. That way, you essentially bet with the house. But indexing has its downsides, not the least of which is lost opportunity.

Let's take an example. Imagine that, on top of your core index holdings, you earmark an additional $50,000 as market-beating ammo. If that $50,000 earns 10% yearly -- something you might hope for from the Standard & Poor's 500 -- you'll have $336,375 after 20 years.

Sounds good, but wait.

How about an extra half-million?
If you can muster 15% instead of 10% with the more aggressive portion of your portfolio, you'll walk out of the gates with $818,327! That extra 5% per year gets you an additional $481,952. I don't know about you ... OK, yes I do: We'd all love to grab half a million dollars extra.

Of course, that extra 5% doesn't do you much good if you don't get it. Sadly, most individual investors don't, for many reasons. Here are just two.

  • Never letting go. In The Courage of Misguided Convictions: The Trading Behavior of Individual Investors, Barber and Odean find that we are 50% more likely to sell a winner than a loser. Our tendency to avoid pain -- in this case, refusing to take a loss that already exists -- is just one psychological weakness that leads us to poor investing decisions.
  • That irrational exuberance. Another is a sort of self-perpetuating prophecy: We pursue exciting opportunities to the point where they may no longer be such good investments. Comcast (NASDAQ:CMCSA) may indeed have potential as an investment, but is its lofty valuation justified?

OK, it's smoky in here ...
But given the choice, where will you sit? At the table of visor-clad sharks or the one with the Hawaiian-shirted tourists? Sounds simple, but odds are you've been sitting down with the sharks in the stock market, playing a tougher game than you have to.

Lakonishok, Shleifer, and Vishny -- three professors I'll call "LSV" for short -- suspected as much. In 1994, the trio set out to investigate why certain types of stocks consistently tend to outperform. They started by dividing stocks, using a variety of factors, into two groups:

  1. Unloved, low-expectation nobodies -- value stocks
  2. High-priced, high-expectation glamour stocks

Picture shoelace makers and semiregional banks versus, say, Network Appliance (NASDAQ:NTAP), Cognizant Technology (NASDAQ:CTSH), or Gold Fields Limited (NYSE:GFI). Tell me, where do you think the sharks are playing?

So how about that extra 5%?
Breaking stocks into a 10-group spectrum ranked by earnings yield (earnings divided by price, or E/P -- academics prefer to rearrange familiar measures to maintain an air of sophistication), the trio shows that the high-E/P, value end of the spectrum bested the glamour end by about four percentage points per year.

That's not pocket change. And it wasn't news to many of history's greatest investors. The likes of Dodd, Graham, Buffett, and many others (will we hear Tom Gardner in there someday?) have espoused buying stocks with low P/Es. It's great to see confirmation of the low-P/E ideology from both academia and investors past, but there's work to be done.

You don't have to be Warren Buffett to notice that E/P, however powerful, is a simple, imprecise measure. LSV knew this -- in fact, the study aimed in part to expound upon a study (by Fama and French) that harped on the book-to-market ratio (B/M), a variable that LSV said wasn't "clean," beacuse of its oversimplicity and inability to capture other forces -- namely, growth.

Investors gone wild
Indeed, growth is sorely missing from measures like B/M (again, an academic inverse of the familiar price-to-book value ratio) measure. So LSV gave growth special attention.

The trio again broke the universe into deciles, this time using a formula based on historical sales growth. Amazingly, value stocks -- the low-expectation, low-growth nobodies -- again walloped glamour, this time by 7.3 percentage points per year.

It's important to understand why. The gang explicitly rejected the knee-jerk academic explanation that if value stocks do better, they must somehow be riskier. Instead, they identified -- and politely termed "suboptimal" -- inferior behavior on the part of investors as the culprit.

Investors overpursue today's hot performers to the point that returns are no longer lucrative. And, to be more specific, LSV's glamour stocks are high-sales-growth stocks that don't have much in the way of assets or cash flows. In other words, stocks such as WellPoint (NYSE:WLP), Kinder Morgan (NYSE:KMI), or maybe even BHP Billiton (NYSE:BBL), which have experienced strong sales growth in the past, wouldn't fit the "glamour" moniker. Moreover -- and I'm probably butchering once-eloquent thoughts -- investors don't adequately understand mean reversion, a concept best explained by analogy.

Odds are that a car going 100 mph on the freeway will be traveling closer to the speed of traffic five minutes later. Similarly, a driver clocked at 35 may be going slowly only temporarily (unless he's driving in front of me) and will, the odds dictate, also be traveling closer to the speed of traffic five minutes hence.

It's not impossible to make money in glamour stocks. In fact, some people make a killing in them. But it's a mistake to assume that current trends will continue indefinitely -- and clearly, the glamour table is a tougher game.

Now I'll drop the bomb
The real secret of the LSV study was the magic of combining factors. A portfolio favoring high (cheap) E/Ps and low growth outperforms its glamour opposite by 11% per year. Now that's astounding. Wouldn't you be eager to sit down at a card table knowing you had an 11% advantage?

And remember the study about individual investors failing to beat the market? There's more to the story, both good and bad. A precious handful -- the top decile -- of investors do beat the market, and nicely, earning "excess returns" of 0.12% to 0.15% per day. But that's largely offset by the bottom decile's 0.11% to 0.12% loss. In other words, for every couch-potato investor doing well, one does poorly -- ironically, so poorly that we'd do well to sell short his favorite picks and profit from their declines.

Bottom line for you
If you're just jumping into the game, odds are you won't win. Your odds of consistently beating the market are about the same as those of consistently letting the market beat you. Of course, we'd all like to think we're in that special 10%, but with your retirement at stake, isn't a little honesty in order?

One option is to find someone in that 10% -- a proven top-deciler -- and keep him or her on a short leash. If you either (a) don't have the time, effort, skill, or inclination to beat the market, or (b) don't know anyone who does, I highly recommend you take a look at Tom Gardner.

Cheating off Tom makes all kinds of sense for the "bottom 90%" of us. Since he launched his Motley Fool Hidden Gems newsletter service in July 2003, Tom's picks have returned 19% -- a mind-blowing seven percentage points greater than the S&P 500. If you like those odds, Tom is offering a no-obligation, 30-day free trial to Hidden Gems, full privileges included -- simply click right here to learn more.

This article was originally published on Jan. 14, 2005. It has been updated.

James Early owns none of the stocks mentioned in this article. The Motley Fool has a disclosure policy .