There's a reason many investors are quick to load up on dividend-paying stocks. The idea of a steady income stream can be quite appealing. And also, during a down market, ongoing dividend payments could help offset losses in your portfolio. Once you secure a steady stream of dividend payments, you'll have the option to reinvest that money for added growth.

While it's easy to see the appeal of stocks that pay dividends, they have their pitfalls, as well. As such, here are a few reasons why you may not want to go too heavy on dividend stocks in your portfolio.

A person at a laptop.

Image source: Getty Images.

1. They could increase your tax burden

If you hold dividend stocks in a tax-advantaged account like an IRA or 401(k) plan, you won't have to worry about paying taxes on your dividend income year after year. But if you receive dividends in a taxable brokerage account, those payments could add to your IRS burden in a big way. This holds true even if you decide to reinvest your dividends, as opposed to cashing them out and spending that money.

The good news is that qualified dividends are taxed at a more favorable rate than ordinary income, so that tax hit may not be as severe. But you'll still need to keep that tax consequence in mind, especially if you tend to owe the IRS a lot of money most tax seasons.

2. You might sacrifice share-price appreciation

Companies that pay generous dividends often do so at the expense of reinvesting in the business itself. And that could limit the extent to which those companies are able to grow and generate revenue.

While dividend-paying stocks might serve as a nice source of ongoing income for you, your shares may not appreciate at the same rate as the stocks you own that don't pay dividends. If your primary goal is to generate a lot of long-term growth in your portfolio, that could be a problem.

3. You might invest in companies that aren't really strong businesses

It's easy to see the appeal of a stock with a generous dividend -- you get a steady stream of income to look forward to. But do remember that a high dividend yield isn't always indicative of a strong underlying business.

In fact, if you're going to buy dividend stocks, rather than look at a company's current dividend, you may want to look at its dividend payment history. A company that's consistently paid or raised dividends may be a much better, more stable bet than a company with a dividend yield that recently increased to a large degree.

Look at the big picture

It's easy to get caught up in the idea of chasing dividends. But a better idea is to invest in quality businesses with strong growth potential. If those companies happen to pay dividends along the way, great -- but that really shouldn't be the main driver in your decision to invest in a given business or not.