According to a Prince and Associates survey conducted during the height of the financial crisis, four out of five high-net-worth investors were planning to take money away from their financial advisors. Surprised? I'm not.

The odds are stacked against us
Especially if we have money invested with a mutual fund conglomerate we saw advertised on TV. And, as you're about to see, the cumulative damage to your long-term wealth can be devastating.

Today, I'll reveal three reasons why I believe we have been set up to fail. Plus, why I'm convinced the stakes have never been higher, and why the mantra "You can't make money buying and holding stocks anymore" is nonsense.

If you agree that what I say makes sense, I'll offer an alternative you can consider. So let's start with the No. 1 reason why most U.S. investors fail ...

1. Thinking inside the box
You probably know the Morningstar style box -- that nine-box grid that shows whether your mutual fund invests in large-cap, mid-cap, or small-cap stocks. And whether the focus is on growth or value -- or some blend of the two. OK, fine. Except for one thing.

The more the style box was used to classify fund managers, the more they seemed to me to be pigeonholing themselves (and us) into an investment style. To see how much this might be costing us, consider the path chosen by renegade investor Peter Lynch at Fidelity Magellan Fund. For 13 years, his investors earned 29.2% per year.

Lynch did it by thinking outside the style box. Sometimes he'd stock up on growth, other times value. If he were running Magellan today, he'd as likely buy slumping big pharma Merck (NYSE:MRK) as tech highflier Research In Motion (NASDAQ:RIMM); stodgy large cap Johnson & Johnson (NYSE:JNJ) or tiny Cell Therapeutics (NASDAQ:CTIC).

In short, Lynch was a financial mercenary who refused to be hemmed in by some "box." That's how he doubled his Magellan shareholders' money in less than three years, then did it again and again for 13 years. That kind of courage is hard to find today, which brings us to the second reason we've been set up to fail.

2. Following the herd
Why anyone would imagine that "following the herd" could make them money in this market is beyond me. But I can tell you (in six words) why professional money managers do it: Picking stocks is a lonely business.

If you're all about keeping your job, it's safer to buy what others buy. Don't believe me? Guess what American Funds' Growth Fund of America, the most widely held U.S. mutual fund, lists among its top holdings. How about Microsoft (NASDAQ:MSFT), Oracle (NASDAQ:ORCL), and Apple (NASDAQ:AAPL)? I know, shocker!

Of course, it's much the same for any "large-cap growth" fund your advisor will get you into. You don't need a certified financial planner to tell you that owning the same stocks as everybody else is no way to get ahead. Or that you can never expect to beat the market by owning the market. Right?

Here's proof ...
What I've just described is called closet indexing. It's rampant on Wall Street, and investors pay billions in "management" fees each year for the favor. Yet of the 8,000 or so who invested money for U.S. investors in 2008, you can expect 78% to 95% to fail to beat the market, according to Yale University's David Swenson.

I knew it was bad, but 78% to 95%? That surprised even me. But I got that figure from John Bogle's new book, Enough, so I believe it. And at the risk of being labeled a Boglehead, I'll cite him again, because no one speaks more eloquently on the third and most important reason investors fail.

3. Getting killed by costs
I don't just mean the fund management fees we've discussed, but investment turnover, too. Not to mention capital gains taxes and trading commissions.

In his book, Bogle shows how, assuming market returns of 8.5% per year, you can expect these intermediation costs to eat up to 80% of your profits over the course of a 40-year investing career. Again, it sounds unlikely, but Bogle runs the numbers in the book; it's worth checking out.

But here's the point
Whether Bogle's 80% figure is high or low, we can agree: Coupled with the constraints placed on fund managers by the style box, and the understandable temptation to follow the herd, investors have a high hurdle to overcome -- especially if they rely on mutual funds.

Add to that a new, bear-market-inspired belief among certain financial advisors that they can -- and should -- help you time the market, and you can see why I say the stakes have never been higher. I mean, come on. You could argue that these market-timing converts might have come in handy in October 2007.

But we sure didn't need them "rotating" us out of stocks at the bottom, even though it meant racking up transaction costs and missing out on the ensuing huge rally. Of course, that's exactly what happened to many U.S. investors this year. Some may stay out of stocks for years -- and that's a crime.

Now your solution ...
Listen: None of this is rocket science. Neither is my solution -- namely, that you start managing some of your own investments, and thus avoid the three reasons investors fail. If you're a purist like John Bogle, buy a low-cost index fund and be done with it.

Bogle doesn't believe we can beat the market with individual stocks. But I do. In fact, I've seen it. Motley Fool co-founders David and Tom Gardner have been recommending stocks of all shapes and sizes in Motley Fool Stock Advisor for seven years now, with remarkable results.

Since they started in March 2002, their recommendations have outperformed the S&P 500 by nearly 10% on an annualized basis -- while the market has been dead flat. I have some ideas, but I'd be a liar if I told you I knew exactly how they do it. But I have seen it with my own eyes.

If you're ready to break free
It certainly helps that David and Tom Gardner are two very different investors with distinct styles, even if they are brothers. Short of finding another Peter Lynch, they may offer your best bet to break from the style box that hems in most professional investors and steer clear of the herd mentality on Wall Street.

And even if Bogle's right and you can't beat the market picking stocks, you can avoid the third, most deadly threat to your long-term wealth: crippling financial intermediation costs. Especially now that you can get David and Tom Gardner's top stock picks for 30 days absolutely free.

And if, like most investors, you do decide stay on, it won't set you back 80% of your rightful profits. Most important, you'll get the advice and support you need to stay invested, even when those who should know better cut and run. To learn more about this special free trial offer to Motley Fool Stock Advisor, click here.

Fool writer Paul Elliott owns shares of Merck and Johnson & Johnson, a Motley Fool Income Investor recommendation. Apple is a Motley Fool Stock Advisor recommendation. Microsoft is an Inside Value recommendation. The Motley Fool owns shares of Oracle and has a disclosure policy.

This article represents the opinion of the writer, who may disagree with the “official” recommendation position of a Motley Fool premium advisory service. We’re motley! Questioning an investing thesis -- even one of our own -- helps us all think critically about investing and make decisions that help us become smarter, happier, and richer.