Social Security is the foundation of many Americans' retirement plans. Around half of individuals aged 65 or older receive at least 50% of their household income from the government program, according to data from the Social Security Administration. So, missing out on a portion of that monthly benefits check could severely cut into many people's retirement plans.

The biggest factor cutting into benefits for most retirees is taxes. The federal government will tax Social Security income for retirees exceeding a certain level of income, but most states won't make retirees pay anything extra on their benefits. However, 12 states currently tax Social Security for at least some residents, depending on income level.

Retirees living in any of those 12 states could lose some of their Social Security checks, but that doesn't mean you should avoid living there. Here's what you need to know.

How the federal government taxes Social Security

The federal government taxes Social Security benefits based on a metric called "combined income." Combined income is the sum of your adjusted gross income, any untaxed interest income, and one-half of your Social Security benefits. If your combined income exceeds a certain threshold, a portion of your Social Security benefits exceeding that threshold is subject to income tax.

The following table illustrates what portion of your Social Security income is taxed at different combined income thresholds.

Taxable Portion of Benefits Single Filer Combined Income Joint Filer Combined Income
Up to 50% $25,000 to $34,000 $32,000 to $44,000
Up to 85% $34,001 and up $44,001 and up

Source: Social Security Administration.

There are ways to avoid paying federal income taxes on Social Security, but they require careful tax planning and require retirees to purposely keep their incomes low. For many, the Social Security tax complicates decisions around retirement account withdrawals and capital gains on investments. But retirees in 12 states might have to worry about state taxes on top of all that.

Two Social Security cards lon top of cash.

Image source: Getty Images.

12 states that tax Social Security for some residents

38 states currently don't tax Social Security at all, but the other 12 will tax a portion of benefits for at least some residents. The laws vary from state to state, and the details can be very important for retirees. Readers should do additional research or consult a tax professional to plan around their specific situation.

Here are some of the basics on the 12 states that will tax your Social Security benefits.

Colorado: Taxpayers under the age of 65 with more than $20,000 in taxable benefits on their federal income tax return are subject to Colorado state taxes on their Social Security income. They'll owe a tax of 4.4% on the same amount of benefits as seen on their federal income tax return. Retirees 65 and older are exempt from state taxes on their Social Security no matter how much of their benefits are taxed by the federal government.

Connecticut: Taxpayers with an adjusted gross income exceeding $75,000 for single filers or $100,000 for joint filers will owe state taxes on a portion of their Social Security benefits. The tax is limited to a maximum of 25% of Social Security income, and it's based on total income. The tax rate will range between 5.5% and 6.99%.

Kansas: Taxpayers with an adjusted gross income exceeding $75,000 will owe state taxes on any amount of Social Security income that's also subject to federal income tax. They'll pay a 5.7% tax, the same as the regular state income tax rate.

Minnesota: Taxpayers with adjusted gross income exceeding $78,000 for single filers or $100,000 for joint filers could see a portion of their Social Security income subject to state taxes. For each $4,000 of adjusted gross income exceeding the threshold, an additional 10% of Social Security income that's subject to federal income tax also becomes subject to Minnesota state taxes. The tax rate will range from 6.8% to 9.85%.

Missouri: Taxpayers with an adjusted gross income exceeding $85,000 for single filers or $100,000 for joint filers will be taxed on the portion of Social Security income pushing them over those thresholds. The tax rate is 5.4%. Note that Missouri is set to stop taxing Social Security benefits in 2024.

Montana: Taxpayers with an adjusted gross income exceeding $25,000 for single filers and $32,000 for joint filers will owe taxes on up to 85% of Social Security income. Those with AGIs below $34,000 for individuals or $44,000 for couples can limit the tax to 50% of their benefits. The tax rate is 6.75%.

Nebraska: Taxpayers will pay income tax on any amount of Social Security benefits that are also subject to federal income taxes. Most taxpayers will owe between 3.51% and 6.84% on the taxable portion of benefits. Note that Nebraska is phasing out Social Security taxation by 2025.

New Mexico: Taxpayers with adjusted gross income exceeding $100,000 for single filers or $150,000 for joint filers will pay taxes on any portion of Social Security benefits also subject to federal income tax. The tax rate ranges from 4.9% to 5.9%.

Rhode Island: Taxpayers with adjusted gross income exceeding $95,800 for single filers or $119,750 for joint filers will owe taxes on any amount of Social Security benefits also subject to federal income tax. The tax rate ranges from 4.75% to 5.99%.

Utah: Taxpayers with an adjusted gross income exceeding $45,000 for single filers and $75,000 for joint filers will owe taxes on any amount of Social Security benefits also subject to federal income tax. The tax rate is 4.65%.

Vermont: Taxpayers with adjusted gross income exceeding $50,000 for single filers or $65,000 for joint filers will owe taxes on a portion of Social Security income subject to federal income tax. Individuals with adjusted gross incomes above $60,000 and married couples with AGIs exceeding $75,000 will owe taxes on 100% of Social Security income subject to federal income tax. The tax rate ranges from 3.35% to 8.7%.

West Virginia: Taxpayers with adjusted gross income exceeding $50,000 for single filers or $100,000 for joint filers will owe taxes on any Social Security income that's also subject to federal income tax. The tax rate is 5.12%.

Don't plan your retirement entirely around state taxes

While the above 12 states might tax some of your Social Security benefits, that shouldn't be a reason to avoid them entirely. Many of these states offer great environments for retirees, with low costs of living or great communities for retirees providing a high quality of life. Those factors can more than offset the taxes you'll pay in retirement.

Not every retirement decision should be about optimizing your finances. You worked hard and saved so you could enjoy your golden years. Spend them wherever you want.