Social Security income is a vital source of financial security in retirement for millions of Americans. However, there are some ways Americans could lose some of their benefits. To name a few, the Social Security earnings test can cause some or all of your benefits to be withheld, not understanding the rules for spousal benefits could be costly, and taxes could eat up more of your retirement benefits than you're anticipating.

Here's a rundown of each of these three things and what effect they could have on your Social Security benefits.

Senior man looking into empty wallet.

Image source: Getty Images.

Earning too much money after claiming benefits

One of the more frequent Social Security questions I hear is "Can I claim Social Security benefits if I'm still working?"

The answer is yes, but there's a big caveat you should know about if you haven't reached your full Social Security retirement age yet. The Social Security earnings test restricts the amount of benefits you can receive if you're still working.

To be clear, if you've already reached your full retirement age, which is between 66 and 67 years old, depending on when you were born, the earnings test does not apply to you. You are free to collect your entire Social Security benefit regardless of how highly paid you are.

On the other hand, if you claim Social Security before reaching full retirement age, here's what you need to know:

  • If you'll reach your full retirement age after 2018, you can earn as much as $1,420 per month without it affecting your Social Security benefits. $1 in benefits will be withheld for every $2 you earn in excess of this amount.
  • If you'll reach your full retirement age during 2018, you can earn as much as $3,780 per month without it affecting your Social Security benefits. $1 in benefits will be withheld for every $3 you earn in excess of this amount, and only the months before the month you'll reach full retirement age will be considered.

Having said all of this, it's important to point out that if your benefits are reduced because of the earnings test, they aren't exactly "lost." Once you reach full retirement age, your benefit will be permanently increased as a result of these withholdings.

Delaying a spousal benefit

It's a well-known fact that if you wait beyond your full retirement age to start collecting Social Security, your benefit will be permanently increased. Current law says that a retirement benefit will increase by 8% per year beyond full retirement age, until as late as age 70.

However, it's important to point out that the same is not true for Social Security spousal benefits. That is, if you are entitled to a benefit based on your spouse's work record, there's no reason to wait beyond your full retirement age to claim it.

Here's another important consideration. The key requirement for collecting a spousal benefit is that the primary-earning spouse is collecting his or her own Social Security retirement benefit at the time you apply. Because there's no such thing as delayed retirement credit for spousal benefits, it's generally not a good idea for a primary-earning spouse to delay his or her own retirement benefit past the spouse's full retirement age, if a spousal benefit is expected.

Taxes -- federal and state

Many new Social Security beneficiaries don't realize that they may have to pay federal income tax on their benefits.

The IRS uses a metric known as your "combined income" to determine whether your benefits will be taxable. That's essentially the sum of all of your other sources of income and half of your Social Security benefits. So if you have $30,000 in other income and a $20,000 annual Social Security benefit, your combined income is $40,000.

Currently, here are the three categories of Social Security taxability:

  • If your combined income is more than $44,000 (married filing jointly) or $34,000 (everyone else), up to 85% of your benefits can be taxable.
  • If your combined income is below these thresholds but is greater than $32,000 (married filing jointly) or $25,000 (everyone else), up to 50% of your benefits can be taxable.
  • If your combined income is below $32,000 (married filing jointly) or $25,000 (everyone else), your Social Security benefits are not considered taxable income.

For the most part, if Social Security is your only significant source of retirement income, this means your benefits will not be taxable. And, if you have substantial amounts of additional income, you'll probably have to pay tax on some of your benefits.

Finally, it's also worth pointing out that although most states don't tax Social Security benefits, some do. There are currently 13 states that tax Social Security benefits, and most of them don't have the same guidelines as the IRS. However, if you live in one of these states and have significant non-Social-Security income, taxes could take a significant bite out of your retirement benefits.