Wall Street offers investors the prospect of great riches, and many Americans have indeed used the products that financial institutions offer to help them make money from their investment portfolios. From mutual funds and exchange-traded funds to insurance-linked products like annuities, you can find a host of different financial products that can let you tailor your investment exposure in just about any way you want.

Those products come at a price, of course, but most financial institutions make those fees seem extremely reasonable. Typical actively managed mutual funds charge just 1% of your assets on an annual basis, and because most funds take those fees out of the interest, dividends, and other investment income that their holdings produce, most investors are apt never even to see the money come out of their investment accounts.

A 1% fee might seem reasonable, especially at the beginning of your career when you don't necessarily have all that much money to invest. If you start out with a financial advisor and only have $1,000 or $2,000 to invest, then the advice you get might seem like it's worth quite a bit more than just $10 or $20.

However, the problem is that that 1% fee has to come from somewhere, and because Wall Street usually just takes it off the top, the net impact is to reduce your long-term returns by a percentage point. Again, that might not seem like a big deal until you look at the following chart.

Chart showing growth of $10,000 at various rates of return for 35 years.

Data source: Author's calculations. Chart by author.

First of all, you have to remember that when you consider typical returns of somewhere between 5% and 10% annually, skimming a full percentage point off the top isn't just 1% of profits. Rather, it's between 10% and 20% of what you make each year. Even worse, because you lose the power of compounding on the money that goes to your financial advisor, the impact on your final nest egg is even larger. Pay a 1% fee on an investment returning 10% for 35 years, and you'll take a cut of between 25% and 30% of your final retirement savings.

You're better off finding ways to avoid big fees, whether it's by investing directly in individual stocks or by preferring index-tracking mutual funds or ETFs that can carry expenses of 0.1% or less. That way, you'll be able to keep more of what you've worked so hard to save.