For many seniors, working in retirement is unattractive because they fear they'll lose some of their Social Security benefits. The reality is that this can happen under certain circumstances, but not for the reason seniors may think.

When will working in retirement cost you Social Security benefits?

For most seniors, concern about working in retirement comes from the rule that you temporarily forfeit part of your Social Security benefits if you earn above a certain limit and you're under full retirement age (FRA). For 2020, if you'll be under FRA for the entire year, you'll have $1 in Social Security benefits withheld for every $2 earned above $18,240. If you'll hit your FRA some time later in the year, you'll have $1 withheld for every $3 in earnings above $48,600.  

Senior woman working and stocking shelves.

Image Source: Getty Images.

But when you forfeit these benefits, there's a payback. Instead, at full retirement age, your benefits will be recalculated to account for the forfeited funds and your monthly checks will get larger. If you live long enough for the higher checks to make up for the missed benefits, you'll get it all back over time.

Even so, there is one circumstance where earning too much in retirement can actually cause you to lose some of your benefits for good: when you earn enough that benefits that wouldn't have otherwise been taxable become subject to tax. 

How taxes on Social Security work

You don't owe any taxes on your Social Security benefits if your provisional income adds up to less than $25,000 as a single tax filer or $32,000 as a married couple filing jointly. But once your income hits those thresholds, as much as 50% of your monthly benefit is taxed on the federal level. For single filers with provisional income above $34,000 or joint filers with provisional income above $44,000, up to 85% of monthly benefits becomes taxable. 

Provisional income is the total of half your Social Security benefits, some nontaxable income (such as municipal bond interest), and all taxable income. Earnings from a job are taxable, so they count in this calculation. That means once you've earned enough to push your provisional income above $25,000 or $32,000 (based on filing status), you're going to lose some of your benefits to the IRS. 

These taxes never go away if you continue to earn above the threshold, unlike the rules regarding forfeiting benefits that apply only before FRA. And depending on where you live, your state may also tax some of your Social Security benefits if you earn too much while receiving them. 

None of this is necessarily a reason not to work. But you do need to be prepared for the tax implications if you hold a job, since you could owe much more to the IRS and keep much less of your retirement money if your paycheck renders your benefits taxable.