Chances are good Social Security will be one of your most important sources of income as a retiree. Unfortunately, some decisions could cost you some of these benefits, and a few of those choices may, at first glance, seem to have nothing to do with your checks from the Social Security Administration.

In fact, here are four actions that could mean smaller retirement checks, or even no checks at all. 

Social Security Card with money sitting on it.

Image source: Getty Images.

1. Withdrawing too much from retirement accounts

Did you know that the amount you withdraw from traditional retirement accounts, such as a 401(k) or an IRA, could end up reducing the amount of your Social Security benefits

That can happen if your taxable withdrawals raise your income too much. Your Social Security benefits are not taxable if your provisional income is below $25,000 as a single filer or $32,000 as a married joint filer. Provisional income is the total of all your taxable income, half your Social Security benefits, and some nontaxable income.

Once you exceed these thresholds, you could be taxed on up to 50% of your benefits. And if your income goes above $34,000 as a single filer or $44,000 as a married joint filer, you could see up to 85% of your benefits subject to federal tax. So taking too much out of your retirement accounts could lead to losing some of your benefits to the IRS.

To avoid this, be careful about the amount you withdraw (just make sure to make your required minimum distributions, which apply in most years but not 2020 due to the coronavirus). You can also use Roth accounts to save for retirement throughout your career so your distributions don't count toward your provisional income and you can take out as much as you want. 

2. Earning too much money as a retiree

For some retirees, making too much money can cause another problem besides taxes. If you are under full retirement age (FRA) and you earn too much money from work, you can end up forfeiting some of your benefits.

You're subject to a retirement earnings test if you work while collecting benefits and you haven't yet hit FRA (which is between 66 and 67). If you are under FRA for the entire year when you'll be working, you'll have $1 in benefits withheld for every $2 earned above $18,240 (in 2020). If you'll hit FRA at some point during the year, you'll have $1 in benefits withheld for every $3 earned above $48,600. 

You don't necessarily lose this money forever. Once you reach FRA, your benefit checks are recalculated to account for forfeited funds, and you'll see higher checks as a result. Over time, the higher checks will make up for the missed benefits. But that won't help much if you receive it slowly over time when you were counting on having funds from both work and Social Security earlier in your retirement. 

3. Living in the wrong state

In most places, you don't have to worry about being taxed at the state and federal level on your Social Security benefits. But in 13 states, which are home to more than 36 million Americans, you could be taxed by your state government.

The rules for how these 13 states tax benefits differ, but if you're subject to it, you could lose even more of your benefits. 

4. Divorcing or remarrying at the wrong time

Spousal and survivors benefits are available to some retirees even after a divorce, but you must have been married for at least 10 years in order to be able to claim on a spouse's work history. If you are close to the 10-year mark and your spouse was the higher earner, getting divorced before hitting that milestone could mean forgoing substantial benefits. 

You also have to be careful about remarrying. If you are receiving survivors benefits as a widow(er), regardless of whether you were divorced or married at the time your spouse died, you could lose those benefits if you get married again before age 60 (or before 50 if you're disabled). And remarrying also ends spousal benefits if you were receiving them after a divorce. 

You may not want to make decisions about marriage and divorce solely based on Social Security. But the choices you make here, along with the choices about where you live and how much you earn as a retiree, could mean a big difference in the size of the checks the Social Security Administration sends you.