We spend a lot of time harping on mutual funds. Frankly, they deserve it. Most underperform their benchmarks and charge fees multiple times higher than passive index funds. The result is a giant wealth transfer from investors to fund managers. 

But after speaking with a fund manager recently, I realize this story is more complicated than I've made it out to be. Mutual fund investors may only have themselves to blame for awful returns.

Most dismal mutual-fund returns are the result of managers engaging in the classic "buy high, sell low" dance. But those buy and sell decisions don't necessarily reflect the will of the investment manager. Fund investors are constantly adding to and withdrawing from the fund's they invest in -- almost always at the worst time possible.

"You would be surprised how easy it is for a fund's investors to take control of the fund," the manager told me.

Imagine you're a smart fund manager who thinks stocks are overvalued. You don't have any good ideas to invest in. But you come into the office one morning and your secretary says, "Congratulations, your investors just sent you another $1 billion." What do you do? You can:

  • Keep it in cash or bonds.
  • Close down your fund and refuse new investments.
  • Grit your teeth and buy overvalued stocks.

The first choice isn't even an option for some funds, as their charters mandate that they stay almost fully invested. Even if they can, bulking up cash dilutes the investments of existing investors. Fund managers rarely take this option -- equity mutual fund cash levels have fluctuated in a tight band of between 4%-6% over the last decade.

The second option is the noble choice, but rarely occurs because funds earn fees on assets under management. When a fund manager goes to his or her boss and says, "I'd like to turn down $10 million in annual fees," the results are entirely predictable. Greenwich real estate doesn't buy itself, you know.

Option three is usually what happens.

As Maggie Mahar writes in her book Bull!:

Everyone realized that as a fund manager, you were basically just a purchasing agent. As a purchasing agent, it was your job to put the money that showed up in whatever stocks your fund was supposed to invest in -- large-cap growth or technology or whatever. If you're a purchasing agent, price is not the issue ... you're like the produce manager in the supermarket -- you have got to have lettuce on sale the next day. No matter what the price. Maybe you can decide to buy the curly lettuce instead of the romaine. But the equity fund manager has to have stocks. You have limited control over what you're doing. 

Now imagine it's 2009, and everything is going to hell in a handbasket. Stocks are the cheapest you've seen in your career, and the last thing you want to do is sell them. But you come into the office one morning and your secretary says, "Your investors want to withdraw $5 billion."

Panic

Source: WikiMedia Commons.

You only have one option to meet that demand: sell cheap stocks. Forget about all the buying opportunities -- your traders are working overtime to liquidate the portfolio whether you like it or not. 

Sadly, that affects all of a fund's investors. Even if one fund investor has a long-term outlook and no intention of selling, the fund's buy and sell actions can be dictated by maniac deposits and panic withdrawals. Other investors' decisions can hurt you. That's why they call it a mutual fund.

For talented fund managers, this cycle is accentuated by "the curse of success." Once the media labels you a "star," investors are going to break down your doors and throw more money at you than you know what to do with. Then, once you have a bad year, they're going to rip it away as fast as it came in.

Take Bill Miller of Legg Mason. Miller was one the best investors in the 1990s and early 2000s before suffering huge losses during the financial crisis that sullied his long-term track record.

What happened? In part, he made some bad calls. But Miller's early success and media fame led investors to give him a net $4.4 billion in new cash to invest just as stocks were getting expensive last decade. As his skill came into question, they then yanked nearly $10 billion out just as stocks were the cheapest they had been in years. Miller's wisdom didn't really matter last decade. His investors were calling the shots.

Think of it this way, and Warren Buffett's success is likely due in part to Berkshire Hathaway's business model. As a public company, rather than a mutual fund, investors can sell Berkshire shares, but they can't take capital away from Buffet's hands. He's in control.

So, what do we learn from this?

One, be wary of celebrity fund managers. They aren't as adored today like they were in the 1990s, but every few years, about half a dozen fund managers grace the covers of the big finance magazines, get praised as geniuses on CNBC, and have buckets of money thrown at them. The results are almost always the same: eventual disappointment. The correlation between fame and regret in the mutual fund world is highly negative.

Two, this is a good reminder of how important it is to learn to invest on your own. Whether that's passively through index funds or actively by buying a portfolio of high-quality stocks, your money is ultimately your responsibility. Unless you want to leave the outcome in a stranger's hand, you need to learn the ropes and take control. 

Fool contributor Morgan Housel owns shares of Berkshire. The Motley Fool recommends Berkshire Hathaway. The Motley Fool owns shares of Berkshire Hathaway. Try any of our Foolish newsletter services free for 30 days. We Fools may not all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors. The Motley Fool has a disclosure policy.