Americans have collectively invested trillions of dollars in mutual funds in search of big gains in the rising stock market. Giving your money to a professional manager may seem like a no-brainer: Who would be better equipped to deliver market-beating performance than a pro?

Unfortunately, beating the market isn't as simple as letting a professional manage your money. First of all, since most of the market is made up of professionals buying and selling stocks, it's not possible for the average fund manager to reliably beat the market. A recent study found that actively managed funds outperformed index funds by just 0.12% annually, over the past 10 years. (This meant that only 24% of managers outperformed the index funds after accounting for fees -- more on that later!)

Second, as I'll expand on in a moment, large actively managed funds have an additional handicap in that they have to invest a lot of cash, which means going well beyond the manager's "best ideas." 

Third, you'll pay a hefty sum in management fees for the privilege of investing in a mutual fund.

Combine these three problems, and it's suddenly not so surprising that the majority of mutual funds underperform the market.

Given these three issues, it's a wonder that anyone still invests in mutual funds. The easy availability of broad index ETFs such as the SPDR S&P 500 (NYSEMKT: SPY) ETF (which tracks the S&P 500) allows investors to capture the market's gains without paying high management fees. Alternatively, disciplined individual investors have a good chance of beating the market by buying and holding solid companies that are trading at reasonable prices.

The disadvantage of size
With so many mutual fund and hedge fund managers trying to maximize returns on trillions of dollars of capital, it's very difficult for any manager to consistently beat the market. One problem is that managers confront a relatively limited menu of investment options. While many experts argue that small-cap stocks are more attractive than large caps, a big fund manager may not be able to put enough money to work in a $500 million company to justify the research effort. This leaves all the fund managers chasing a relatively small set of large-cap stocks.

Occasionally, mutual funds will be able to find companies that are very large and still growing quickly, such as Apple (NASDAQ: AAPL) circa 2010. Yet Apple could be considered the exception that proves the rule. The stock has dropped dramatically over the past eight months, weighing down many funds. However, managers who have decided to reduce their exposure to Apple don't have many great alternatives: High-growth large caps are a rare lot.

Consider this ...
Fidelity's Magellan Fund is a perfect example of the diseconomies of scale in the investment world. The Magellan Fund performed remarkably well under the management of super-investor Peter Lynch from 1977 to 1990 (with an average annual return of 26.4%). However, the fund had just $100 million in assets when it opened to the public in 1981.

By contrast, the Magellan Fund exceeded $100 billion in assets by 1999 -- although it has shrunk significantly since then. The fund has underperformed the S&P 500 by a wide margin over the past 10 years, despite changing managers several times. Much of the poor performance can be attributed to having managers who were far less talented than Lynch, but it would be wrong to ignore the obstacles the mutual fund's massive size has created.

Will you get what you pay for?
Not only do fund managers have trouble beating the market consistently, but they usually charge high management fees. Some of the larger funds have annual management fees under 1% -- for example, Fidelity's Contrafund has an expense ratio of 0.74% -- but many mutual funds have an expense ratio around 1.5%. Some funds even charge a fee (or "load," in the industry lingo) for investing with them: These are essentially sales commissions and further dampen investors' returns.

Given the limitations that hurt big funds' performance, even an expense ratio below 1% makes it very challenging to beat the market. Not surprisingly, there are a few outliers that have managed to outperform. Contrafund has beaten the S&P 500 by nearly 3 percentage points over the past 10 years. Still, it's difficult to maintain that level of performance; looking at just the last five years, Contrafund is even with the market's rate of return.

Two alternatives
While a few mutual funds may be able to beat the market, it's essentially impossible to predict in advance who will do well. (After all, strong past performance does not guarantee future gains.) The power of index funds is that they don't run up big expenses for managers and marketing. The aforementioned SPDR S&P 500 ETF has an annual expense ratio of just 0.09%, a fraction of even the cheapest actively managed funds. For that low price, you'll almost certainly earn a return just a tiny bit under the market's return. Given the mutual fund industry's track record, you're unlikely to do any better in an actively managed fund.

Alternatively, if you're willing to invest some of your valuable time as well as your money, you may be able to beat the market as an individual investor. Mutual funds may have high-paid managers with MBAs, but as a small investor, you have the ability to focus on a few stocks that look particularly promising. We at The Motley Fool can help you identify great buy-and-hold opportunities: stocks that are likely to outperform even the best fund managers' portfolios.

The Motley Fool's free report "3 Stocks That Will Help You Retire Rich" names stocks that could help you build long-term wealth and retire well, along with some winning wealth-building strategies that every investor should be aware of. Click here now to keep reading.

Fool contributor Adam Levine-Weinberg owns shares of Apple. The Motley Fool recommends and owns shares of Apple. Try any of our Foolish newsletter services free for 30 days. We Fools don't all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors. The Motley Fool has a disclosure policy.