Saving for retirement is always a challenge. In fact, 1 in 3 Americans has absolutely nothing saved for retirement, according to a study from GOBankingRates, and more than half (56%) have less than $10,000 stashed away.

The truth is, though, that you can always be saving a little bit more. Even if you're not earning a six-figure salary or still have a mortgage and car loan to pay down, that doesn't mean you can't save more for retirement.

Person putting coin into a white piggy bank

Image source: Getty Images.

No matter what your financial situation looks like, there are a few things that can seriously set back your retirement saving efforts.

1. Saving for your kids' college fund rather than your retirement fund

Every parent wants to help their child, but if you're putting your savings toward college tuition rather than your retirement fund, you're doing yourself a disservice. While college is expensive -- tuition and fees for public and private universities cost around $25,290 and $50,900 per year, respectively -- retirement is even more costly.

The average household aged 65 or older spends about $45,756 per year in retirement, according to the Bureau of Labor Statistics. If you spend, say, 20 years in retirement, that puts the total cost of retirement at roughly $915,120 (not taking inflation into consideration).

True, you have more time to save for retirement than you do for college. However, you need to save significantly more money for retirement, and there's no such thing as a retirement loan, so it's important to save as early and aggressively as possible. You may think you can put off retirement saving until your kids get through school, but the longer you wait, the harder it will be to catch up.

For example, say you have $10,000 to invest. If you invest that at age 35 while earning a 7% annual rate of return on your investments and don't make any additional contributions, you'll have around $76,122 by age 65. Wait just five years to invest that money, though, and you'll end up with only $54,274 by the time you turn 65. In other words, you need to start saving while time is on your side -- even if it means putting your retirement savings before your kids' college tuition. 

2. Not paying down high-interest debt

Even if you are saving for retirement, if you're struggling to pay down high-interest debt (like credit card debt), you may be doing more harm than good.

High-interest debt is the most toxic form of debt, and depending on how much you owe, you could end up paying thousands of dollars in interest alone -- money that would be better spent elsewhere.

Say you have $5,000 in credit card debt with an APR of 16%, and you're paying $175 per month toward that debt. At that rate, it will take you over 12 years to pay off that debt completely, and you'll end up paying roughly $8,000 total -- around $3,000 of which is interest payments alone.

While it's important to start saving for retirement as early as possible, if you're spending more on interest payments for your debt than you're earning on your investments, then you're not doing yourself any favors.

For instance, say you have $5,000 in credit card debt at a 16% APR, and you also have $10,000 in a retirement account earning a 7% annual rate of return. In this scenario, you have a couple options: Pay down debt and save for retirement at the same time, or pay off all your high-interest debt first and then save for retirement. Let's say you have a total of $300 per month to put toward these two expenses, and $175 per month is currently going toward debt.

In scenario one, if you're paying $175 per month toward debt and $125 toward retirement, your debt will be paid off after 12 years, and you'll end up paying roughly $3,000 in interest. In that same time period, you'll have $50,362 saved in your retirement fund. Subtract the $3,000 in interest, and your net savings amount to around $47,362.

In scenario two, you're paying the full $300 per month toward debt and making no additional contributions to your retirement fund. Here, it will take roughly seven years to pay off your debt, and you'll pay around $1,400 in interest. During these seven years, your retirement fund will grow to around $22,522 -- even without additional contributions. Then, if you start contributing $300 per month to your retirement savings for the next five years, you'll end up with $53,068 total. Subtract the $1,400 in interest payments, and your total savings come to $51,668.

Of course, every situation is different, but the more debt you have, the more you'll pay in interest -- and the more you pay in interest, the less money you'll have to contribute to your retirement fund. The faster you can pay down high-interest debt, the bigger your nest egg will be in the end.

3. Waiting until you earn more money to start saving

It's easy to put off saving until you get a raise or a new job with bigger paychecks, but if you wait too long, you'll have a very hard time catching up on your savings -- even if you're contributing more each month. Thanks to the power of compound interest, the more time your money has to grow, the easier it will be for you to build up your retirement fund.

For example, say you're currently 30 years old and aren't contributing anything to your retirement fund, but then at age 40 you get a raise and start contributing $300 per month for the next 25 years. In another scenario, say you're 30 years old and you're contributing $150 per month for the next 35 years. Assuming you  start with zero savings and earn a 7% annual rate of return in both scenarios, here's what your savings would look like over time:

Age Saving $150 per Month Saving $300 per Month
30 $0 $0
40 $25,803 $0
50 $76,561 $51,606
60 $176,409 $153,122
65 $258,163 $236,241

Data source: Author's calculations.

In other words, even if you doubled your monthly contributions, you couldn't make up for those 10 years in which you saved nothing. So if you're hesitant to start saving because you don't think your smaller contributions will amount to anything, remember that it's infinitely better to start saving now than to wait.

Saving for retirement is never easy, but you may be making it harder for yourself without realizing it. By avoiding these roadblocks, you'll have an easier time saving for the retirement you've always dreamed about.