Image source: SSA Office of the Inspector General.

After a long career, most Americans see their Social Security benefits as something they've earned in full. That's why it comes as such a shock to many Social Security recipients that the IRS can stake its claim on their monthly checks. Not everyone on Social Security has to pay tax, but those who do can end up having to include as much as 85% of their benefits as taxable income on their tax returns. Below, we'll look more closely at how taxation of Social Security works and how many Americans end up having to pay the tax man.

Why would you have to pay tax on Social Security?
The rules governing Social Security taxation are pretty straightforward. If you earn above a certain amount of income in retirement, then you'll have to add in some of your benefits in calculating your income tax.

To calculate whether some of your Social Security will be treated as taxable income, first take all of your other sources of income, including wages, investment income, and taxable private pensions. Then add in half your Social Security income. If the result is greater than the first threshold amount below, then you could end up paying tax on as much as 50% of your Social Security benefits. If it's greater than the second threshold, then that taxable percentage could jump to as much as 85%.

If Your Filing Status Is...

Then You Could Pay Tax on Up to 50% of Your Social Security if You're Above This Threshold...

And You Could Pay Tax on Up to 85% of Your Social Security if You're Above This Threshold...

Single, Head of Household, Qualifying Widow(er)

$25,000

$34,000

Married Filing Jointly

$32,000

$44,000

Data source: IRS.

When those thresholds were originally established in 1984, they were designed to include only a small number of higher-income taxpayers. But the thresholds weren't indexed for inflation, and so over more than 30 years, the number of Americans affected by the provision has risen dramatically. In 2013, the most recent year for which IRS data are available, more than 27 million taxpayers included Social Security benefits on their tax returns. Of those, more than two-thirds -- 18.5 million -- indicated that some of their benefits were taxable.

The truly staggering numbers involve how much in Social Security benefits was subject to tax. Out of a total of $553.5 billion in benefits listed, taxpayers included more than $243.3 billion in their taxable income. Taking $243.3 billion and dividing by $553.5 billion works out to a benefit-weighted average of 44% for American taxpayers. That amounts to almost $13,150 on average for those who indicated any taxable benefits, or almost $9,000 even when you include those who indicated a zero taxable amount.

Image source: Wikimedia Commons.

How you can fight back against Uncle Sam
If you don't want to pay tax on your Social Security benefits, there are several things you can do. First, take advantage of ways you can get tax-free income in retirement. Municipal bond income doesn't work because the IRS forces you to include it in the calculation for purposes of taxing Social Security, but distributions from Roth IRAs don't count toward the thresholds.

Second, consider whether deferring your benefits could help you avoid tax. For instance, taking early Social Security benefits while you're still working can be tempting, but it also increases the likelihood of getting your benefits taxed. Even if you've already retired, taking benefits while your spouse is working can push your overall income over the threshold for joint filers. In some cases, if you wait to claim benefits until later, it can result in that additional income having disappeared and leaving you with more after-tax benefits intact. Similarly, choosing when to take withdrawals from traditional IRAs and 401(k)s can affect your taxable income and therefore how much of your Social Security gets taxed.

Finally, look at ways to accelerate income in years in which you don't meet the threshold. Some retirees choose to wait until the maximum age of 70 1/2 before taking IRA and 401(k) withdrawals, but doing so can require you to take minimum distributions that are large enough to trigger Social Security tax. If you're more careful about spreading out those withdrawals over time, it can help you avoid to be taxed on your benefits.

Taxes on Social Security benefits come as a shock to many retirees. By knowing it's out there, though, you can take steps to avoid it or at least minimize the impact.