The appeal of paying off your mortgage ahead of schedule is easy to understand. Why carry a big debt if you don't have to? If your mortgage permits prepayments without penalties, you may well be thinking of making some extra payments. Think again, though, because it's not always your best move.

Other investments, other returns
For starters, consider that if you have a mortgage with a 5% interest rate, any prepayments you make on that loan will essentially earn you a 5% return. That's great. But hold on -- you can make other kinds of investments and earn other kinds of returns. Is paying off your mortgage early your best bet?

The stock market, for example, has averaged roughly a 10% annual gain over many decades, and you can invest in most of the market via a simple, inexpensive index fund. Even if you only average a 7% or 8% return over your investment period, that beats the 5% from your mortgage prepayment. (And, of course, you might also average more or less than 10% in the stock market. There's just no way to know for sure.)

You might alternatively, invest in a few select stocks, such as ones that pay dividends. A healthy and growing company that pays out a dividend will generally do so in both bull and bear markets. It will typically increase its dividend over time, giving shareholders an ever larger effective yield. And it also offers stock-price appreciation. If you have long-term faith in the tobacco industry serving its addicted customers, you might consider Marlboro maker Altria (MO 1.85%), which recently yielded 5.1% -- more than your 5% mortgage! (It has averaged annual gains of about 17% over the past 20 years, too.) Or consider New York Community Bancorp (NYCB 0.68%), which is well regarded for smart management and has succeeded without ever receiving TARP money. The bank recently yielded 6%.

Better still, you might grab a solid dividend and instant diversification across many companies via a dividend-focused exchange-traded fund. The iShares Select Dividend ETF (DVY 1.51%) and the SPDR S&P International Dividend ETF (DWX 0.62%) recently yielded 3.1% and 6.9%, respectively. They charge just 0.40% and 0.45% in annual fees, too, and save you the trouble of selecting individual dividend-paying stocks by instantly plunking you into a big bunch.

One step forward, two steps back?
Another key consideration is any high-interest debt you might be carrying. If you've got a mortgage charging you 5% interest and a credit card charging you 20%, you should definitely put any excess cash you can toward paying down that high-interest debt. Extra money paying down your mortgage principal will indeed be like earning 5% -- but high-interest rate loans are the opposite of wealth-building. Instead of growing your wealth, they shrink it. (To be fair, they do grow the wealth of credit card companies.) Your credit card debt will grow by 20% annually if you don't tackle it aggressively, and at 20% a $5,000 obligation can become tens of thousands of dollars of debt in just a decade or less.

Speaking of interest, that's another factor to weigh when you're thinking of prepaying your mortgage. Mortgage interest payments are currently deductible on your taxes, reducing your taxable income. Thus, they benefit those in higher tax brackets the most. If you're in the 35% tax bracket and you deduct $5,000 in mortgage interest, that saves you $1,750. If you're in the 25% bracket, your savings fall by $500 to a still-not-insignificant $1,250. Keep in mind, though, that to reap this benefit, you must itemize your deductions. Many taxpayers don't do that, instead taking the standard deduction. When deciding whether to prepay your mortgage, run some numbers to see if it makes sense, tax-wise.

The authors of a recent study, "Unmasking the Mortgage Interest Deduction: Who Benefits and by How Much -- 2013 Update," state that only about 25% of taxpayers claimed the mortgage interest deduction in 2012. Among homeowners with incomes between $40,000 and $50,000, the average tax savings was just $54 per month, or a total of $648. The total revenue lost to the U.S. government due to the deduction was close to $70 billion, which is why some are calling for a reduction or elimination of the deduction.

The sure thing
Despite these and other reasons that might have you deciding to not prepay your mortgage, there does remain a compelling argument for doing so: It offers a guaranteed return. With bank accounts generally offering well below 1% in interest these days, and five-year CDs not offering much more than 2%, a guaranteed 5% return is nothing to scoff at.

There's no one-size-fits-all answer to the question of whether to pay down your mortgage early. Weigh the pros and cons in light of your particular circumstances, crunch some numbers, and see what makes the most sense for you.