Paying your fair share of taxes is important. They provide the money necessary to build the roads you drive on, pay for the schools you will (or did) send your kids to, and many other conveniences. At the same time, there's no need to pay more than you have to.

In an interview published years ago in Kiplinger's Personal Finance, former IRS commissioner Fred Goldberg revealed one tax "break" in particular that millions of Americans forget to take advantage of.

I put "break" in quotation marks because the tax savings I'm referring to don't come from a credit or deduction. Rather, it's a simple way for investors to avoid overpaying their taxes by a potentially huge amount. And unfortunately, many investors miss it entirely.

A quick primer on investment taxes
Usually, the only time you'll owe the IRS money on your investments (excepting those held in a Roth IRA) is when you cash them out. You could own shares of Company A for 10 years -- during which their value increases tenfold -- and never pay a dime in taxes for the gains that you see on paper. You only get a tax bill on the profits when you sell your shares.

But for millions of investors, there's more to the tax equation than straightforward capital gains. Namely, you need to keep an eye on your dividends received. Even if you have your dividends automatically reinvested in more shares (which is the smart choice, given how much it can accelerate your compound interest), you will owe taxes on those dividends in the year they were paid out. Your tax rate will vary based on your overall income.

There's nothing wrong with paying those taxes. In fact, it's illegal to avoid them. However, if you've been reinvesting those dividends without adjusting your cost basis with every payout, then you could be paying Uncle Sam more than his fair share.

Where the big mistake is made
Let's look at an example to demonstrate how investors are unwittingly overpaying their taxes. (Note: The scenario below isn't necessarily realistic, as it involves a one-off transaction, but we'll use it for simplicity's sake.)

Let's say you bought $10,000 worth of shares of the Vanguard Total Stock Market ETF (VTI -0.59%) back in October 2002, and you've been reinvesting the dividends ever since. On Dec. 1, 2015, that investment was worth $40,340.

VTI Total Return Price Chart

VTI Total Return Price data by YCharts.

Let's say you decided to cash out the entire stake on Dec. 1. Here's how that investment would be taxed.

Cost Basis

Cash Out Amount

Capital Gains

Tax Rate

Total Taxes Owed

$10,000

$40,340

$30,340

15%

$4,551

Here's the thing: As ordinary as those numbers may seem, you would in fact be overpaying by a wide margin. How? Well, every time you received a dividend and reinvested it, you paid taxes on your dividends. With each reinvestment, you should also be increasing your cost basis. Failing to do so will cause your dividends to be taxed twice.

How big of a deal is this? That depends. But for our example investment in Vanguard's Total Market ETF, take a look at the fund's performance from 2002 to 2015:

Source: Google Finance.

Each one of those blue Ds represents a dividend payment that, if not accounted for in the adjusted cost basis, will be taxed twice. After doing a little math, I found that the cost basis for this investment would have increased all the way to $12,735 because of dividend reinvestments. That means the real tax situation looks like this.

Cost Basis

Cash Out Amount

Capital Gains

Tax Rate

Total Taxes Owed

$12,735

$40,340

$27,605

15%

$4,141

Thus, in this scenario, the taxpayer who failed to account for dividend reinvestment would be overpaying by $410 -- about 10%. But as I said before, this isn't the most realistic example. In real life, folks might cash out of investments that paid bigger dividends or that they purchased multiple times. Those factors could make the double taxation effect really add up.

First the good news, then the bad news, then the good news
Fortunately for those who started investing after 2012, the IRS now requires all firms to keep track of the cost basis for their customers -- including dividend reinvestment. So this won't be much of a problem a few decades from now.

Unfortunately, that doesn't help people who were invested in securities before 2012 -- a group that likely includes the vast majority of investors reading this article and nearing retirement. However, if you break out the elbow grease and contact your brokerage firm, you should be able to get the information you need to make sure you aren't overpaying on your capital gains taxes.

Make the effort. You might be surprised at what a big difference it makes.