Dividend-paying stocks have their advantages. They're dependable and resilient. They don't rise or fall too fast. Plus, they pay you to own them.

Still, dividend stocks aren't universally a great investment. Depending on your situation and your financial goals, investing in dividend payers could be the wrong move. Here are four reasons you'd want to avoid holding dividend stocks in your portfolio.

1. Lower share price appreciation

The total return on a dividend stock has two components -- dividend yield and share price appreciation. Combined, those two values can deliver decent returns. But compared to higher-growth companies, "decent" often means unimpressive. And many dividend payers look even more marginal if you compare them on share price appreciation alone.

Adult couple reviews paperwork in front of laptop at home.

Image source: Getty Images.

As a granular example, longtime dividend payers like Johnson & Johnson and Procter & Gamble have underperformed Amazon and Alphabet, two popular stocks that don't pay dividends. You can see these stocks' annualized returns over the past five years in the table below. 

JNJ Chart

JNJ data by YCharts

The above comparison obviously isn't scientific or conclusive. To be fair, there are dividend stocks -- like Microsoft -- that have appreciated well even while making shareholder payments.  

A broader view, though, does support the theory that longtime dividend payers grow slowly. The table below shows the entire S&P 500 and the S&P 500 Dividend Aristocrats index. The Dividend Aristocrats index includes only S&P 500 companies that have raised their dividend for 25 or more consecutive years. Over the last five years, the broader S&P 500 index has outperformed the aristocrats in price, 16.4% vs. 12.6%.

^SPX Chart

^SPX data by YCharts

2. Tax consequences

Dividend income is taxable, either at the capital gains rate or the higher ordinary income tax rate. By comparison, you don't pay taxes on share price appreciation unless you sell and realize the gain.

The tax rules apply only if you are holding your dividend-paying stocks in a taxable brokerage account. Taxes on dividends are deferred in your 401(k) and traditional IRA, and they're tax-free in a Roth IRA as long as you only take qualified withdrawals.

3. Uncertain income

Dividends are not guaranteed. While some companies have established decades-long dividend track records, those shareholder payments can be reduced or disappear entirely. 

In 2018, for example, General Electric cut its dividend from $0.12 per share to $0.01 per share. The company had been paying its dividend for 119 years and only slashed it by more than half one other time since the Great Depression.

Other epic dividend cutbacks include General Motors' dividend suspension in 2008 and Citigroup's suspension in 2009.

4. Share prices are reactive to dividend policy

Investors don't like dividend cutbacks. If a company makes a dividend announcement that investors find disappointing, the share price will reflect the negative sentiment. When GE slashed its dividend in 2018, the share price fell 34% from about $78 to $51.54 in six weeks. As a result, shareholders had to absorb two negative consequences -- a loss of income and a loss of value.

Reliable, but no guarantees

Dividend stocks may not be a great fit if you have a long investment timeline and a good appetite for risk. The tax implications of dividend stocks may also give you pause. While most dividends are taxed at the lower capital gains rate, any taxability compares unfavorably to unrealized gains on your buy-and-hold stocks -- which aren't taxed at all.

Finally, you may not like the core value proposition of dividend stocks. Dividend-payers promise reliability and resilience in exchange for modest growth, but no stock can guarantee the dividend or the modest growth will continue indefinitely. And when a dividend stock falters, the consequences to shareholders can be severe.

Since you face a risk of loss with any equity investment, you might prefer a little more risk in exchange for higher growth potential. If you have the nerves and the time to ride out some volatility, growth-oriented stocks have a brighter future than many dividend payers.