Can Individual Investors Beat the Market?

That's a good question -- and the title of a now-famous academic paper. For most investors, the answer is an emphatic "No." At least not by any meaningful degree, but more on that later.

Fortunately, you have options. You can buy an index fund, something we recommend for a good chunk of your portfolio. That way, you essentially bet with the house. But indexing has its downsides, not the least of which is lost opportunity.

Let's take an example. Imagine that, on top of your core index holdings, you earmark an additional $50,000 as market-beating ammo. If that $50,000 earns 10% yearly -- something you might hope for from the Standard & Poor's 500 -- you'll have $336,375 after 20 years.

Sounds good, but wait.

How about an extra half-million?
If you can muster 15% instead of 10% with the more aggressive portion of your portfolio, you'll walk out of the gates with $818,327! That extra 5% per year gets you an additional $481,952. I don't know about you. ... OK, yes I do: We'd all love to grab half a million dollars extra.

Of course, that extra 5% doesn't do you much good if you don't get it. Sadly, most individual investors don't, for many reasons. Here are just two.

Never letting go. In The Courage of Misguided Convictions: The Trading Behavior of Individual Investors, Barber and Odean find that we are 50% more likely to sell a winner than a loser. Our tendency to avoid pain -- in this case, refusing to take a loss that already exists -- is just one psychological weakness that leads us to poor investing decisions.

That irrational exuberance. Another is a sort of self-perpetuating prophecy: We pursue exciting opportunities to the point where they are no longer such good investments. AMD (NYSE:AMD) may have merit as a company, but was it worth a triple-digit P/Eback in 2005? Probably not. And that's one of the reasons the stock has dropped even as the company has grown earnings.

OK, it's smoky in here ...
But given the choice, where will you sit? At the table of visor-clad sharks or the one with the Hawaiian-shirted tourists? Sounds simple, but odds are you've been sitting down with the sharks in the stock market, playing a tougher game than you have to.

Lakonishok, Shleifer, and Vishny -- three professors I'll call "LSV" for short -- suspected as much. In 1994, the trio set out to investigate why certain types of stocks consistently tend to outperform. They started by dividing stocks, using a variety of factors, into two groups:

  1. Unloved, low-expectation nobodies -- value stocks.
  2. High-priced, high-expectation glamour stocks.

Picture shoelace makers and semi-regional banks versus, say, Infosys Technologies (NASDAQ:INFY), CNET Networks (NASDAQ:CNET), or Wind River Systems (NASDAQ:WIND). Tell me, where do you think the sharks are playing?

So, how about that extra 5%?
Breaking stocks into a 10-group spectrum ranked by earnings yield (earnings divided by price, or E/P -- academics prefer to rearrange familiar measures to maintain an air of sophistication), the trio shows that the high-E/P value end of the spectrum bested the glamour end by about four percentage points per year.

That's not pocket change. And it wasn't news to many of history's greatest investors. The likes of Dodd, Graham, Buffett, and many others have espoused buying stocks with low P/Es. It's great to see confirmation of the low P/E ideology from both academia and investors past -- but there's work to be done.

You don't have to be Warren Buffett to notice that E/P, however powerful, is a simple, imprecise measure. LSV knew this -- in fact, the study aimed in part to expound upon a study (by Fama and French) that harped on the book-to-market ratio (B/M), a variable described by LSV as "not 'clean'" for its oversimplicity and inability to capture other forces -- namely, growth.

Investors gone wild
Indeed, growth is sorely missing from measures like B/M (again, an academic inverse of the familiar price/book value ratio) measure. So LSV gave growth special attention.

The trio again broke the universe into deciles, this time using a formula based on historical sales growth. Amazingly, value stocks -- the low-expectation, low-growth nobodies -- again walloped glamour, this time by 7.3 percentage points per year.

It's important to understand why. The gang explicitly rejected the knee-jerk academic explanation that if value stocks do better, they must somehow be riskier. Instead, they identified -- and politely termed "suboptimal" -- inferior behavior on the part of investors as the culprit.

Investors overpursue today's hot performers to the point that returns are no longer lucrative. And, to be more specific, LSV's glamour stocks are high-sales-growth stocks that don't have much in the way of assets or cash flows. In other words, stocks like Apollo Group (NASDAQ:APOL), CA (NYSE:CA), or (arguably) even eBay (NASDAQ:EBAY), which have experienced strong sales growth in the past, wouldn't fit the "glamour" moniker. Moreover -- and I'm probably butchering once-eloquent thoughts -- investors don't adequately understand mean reversion, a concept best explained by analogy.

Odds are that a car going 100 mph on the freeway will be traveling closer to the speed of traffic five minutes later. Similarly, a driver clocked at 35 may be going slowly only temporarily (unless he's driving in front of me) and will, the odds dictate, also be traveling closer to the speed of traffic five minutes hence.

It's not impossible to make money in glamour stocks. In fact, some people make a killing in them. But it's a mistake to assume that current trends will continue indefinitely -- and clearly, the glamour table is a tougher game.

Now I'll drop the bomb
The real secret of the LSV study was the magic of combining factors. A portfolio favoring high (cheap) E/Ps and low growth outperforms its glamour opposite by 11% per year. Now that's astounding. Wouldn't you be eager to sit down at a card table knowing you had an 11% advantage?

And remember the study about individual investors failing to beat the market? There's more to the story, both good and bad. A precious handful -- the top decile -- of investors do beat the market, and nicely, earning "excess returns" of 0.12% to 0.15% per day. But that's largely offset by the bottom decile's 0.11% to 0.12% loss. In other words, for every couch potato investor doing well, one does poorly -- ironically, so poorly that we'd do well to sell short his favorite picks and profit from their declines.

Bottom line for you
If you're just jumping into the game, odds are you won't win. Your odds of consistently beating the market are about the same as those of consistently letting the market beat you. Of course, we'd all like to think we're in that special 10%, but with your retirement at stake, isn't a little honesty in order?

One option is to find someone in that 10% -- a proven top-deciler -- and keep him or her on a short leash. If you either (a) don't have the time, effort, skill, or inclination to beat the market, or (b) don't know anyone who does, I highly recommend you take a look at Tom Gardner.

Cheating off Tom makes all kinds of sense for the "bottom 90%" of us. Since he launched his Motley Fool Hidden Gems newsletter service in July 2003, Tom's picks have returned 47% -- a mind-blowing 26 percentage points above the S&P 500. If you like those odds, Tom is offering a no-obligation, 30-day free trial to Hidden Gems, full privileges included -- simply click right here to learn more.

This commentary was originally published on Jan. 14, 2005. It has been updated.

James Early owns none of the stocks mentioned in this article. CNET is a Rule Breakers pick. eBay is a Stock Advisor recommendation. The Motley Fool has adisclosure policy.