Social Security benefits are a major source of retirement money for most Americans. But they aren't enough to live on without outside funds even under the best of circumstances. Sadly, far too many Americans don't make the best choices to maximize their benefits. 

The problem is, there are many ways to reduce your Social Security checks that you may not be aware of. Here are four that could have the most profound impact. 

Older man sitting alone at table eating.

Image source: Getty Images.

1. Working too few years

Most people know their wages throughout their working life determine how much they'll receive from Social Security. What may come as a surprise is that the Social Security Administration (SSA) always considers the same number of working years when determining your benefits. Whether you worked for 25 years or 45 years, the SSA calculates average wages over your career based on the 35 years your inflation-adjusted earnings were highest. 

That means if you've worked longer than 35 years, some of your lowest-earning years won't be included, so your average wage (and monthly benefit) will end up higher. On the other hand, if you've worked less than 35 years, there will be $0 wages included in your average. This means that by either working for less than that time or stopping work during prime earning years and not having your high salary factored in, you'll be giving up some benefits you could've received. 

2. Claiming benefits at the wrong time

Retirees can choose to claim Social Security benefits any time between 62 and 70, with those who claim earlier getting more (but smaller) checks and those who claim later getting fewer (but larger) ones.

For around six in 10 retirees, claiming later makes the most sense because it provides more lifetime income. But for those who pass away before reaching their breakeven age -- typically late 70s or early 80s -- delaying a claim would mean less lifetime money from Social Security. Determining the best age to claim benefits is a tricky calculation because there's no way of knowing your life expectancy. You can assess your likely longevity to guide your choice, or play the odds and assume you'll be one of the majority of retirees who end up better off by waiting. 

3. Living in the wrong place

There are 13 states that tax Social Security benefits, although that will soon be 12. If you live in one of them and are subject to the tax, you'll lose some of your retirement money to your local government. If you're struggling to get by, it may make sense to relocate to a state that won't take a cut -- especially if it's a state with a lower cost of living as well. 

4. Having income above IRS limits

If what Social Security considers to be your "combined income" is too high, you will lose a portion of your Social Security benefits to the IRS. Up to 85% of your benefits could potentially be taxed at the federal level. Combined income includes half your Social Security benefits, some nontaxable income, and all taxable income including distributions from traditional 401(k) and IRA accounts. Around 50% of seniors lose some of their benefits to the IRS

Losing out on Social Security benefits because of taxes or poor timing on claiming benefits is a big problem if you need them to help make ends meet. Understand how benefits work to make an informed choice about when to claim them. And building a hefty nest egg means you won't rely too much on Social Security and end up struggling.