Retirement should be a time to kick back and relax, whether that means traveling the world or staying at home to play with the grandkids. Unfortunately, far too many seniors are spending their golden years worried about their finances-- and debt is a big reason why. As many as 61.3% of households headed by an adult aged 60 or older have some form of debt and, among senior households with debt, the median total owed is around $40,900 . 

Mature couple doing financial calculations

Image Source: Getty Images.

A combination of too little retirement savings and too much debt combine to create the ideal conditions for financial worry that, in a best case scenario, make it harder to enjoy retirement and, in a worst case scenario, result in actual financial disaster. Bankruptcy rates have been rising among seniors , and almost 3 million senior-headed households experienced food insecurity in 2015, so the threat of a dire financial emergency is very real. 

Prioritizing retirement savings is essential to stave off a financial calamity, as living off Social Security alone is difficult or impossible. You may not have a choice about when your retirement day arrives if you get laid off and can't find another job or if you become too sick to work -- so you need to have a cushion in the bank for when that day arrives. But saving alone isn't enough if you want to reduce the chances your golden years will be filled with money stress. While building a big enough nest egg should be your top financial priority, you should also make it a goal to pay off these three big debts before retirement. 

1. Mortgage debt

Among homeowners aged 55 to 64, just over 40% still have a mortgage on their home. Slightly more seniors aged 65 to 74 have paid off their mortgages, with around 30% of homeowners in this group still owing on a home loan. Even among seniors 75 and older, roughly 14% still owed on a mortgage loan . 

Housing is the largest expense for seniors, and accounts for around 32.9% of annual expenditures in older households . Paying off a mortgage means housing costs substantially decrease, your biggest retirement expense is eliminated, and your savings lasts much longer. If you have an average retirement savings of around $150,000 and an average monthly mortgage payment of $758, you'll spend $9,096 annually on your mortgage -- more than the 4% of savings most experts recommend withdrawing each year. 

If you're still young, you have time to make sure your mortgage debt is retired before you reach you are. Buy a home at the low end of what you can afford and avoid refinancing your home, which usually extends your repayment period. If you're older and nearing retirement, consider working longer, if you can, to get your mortgage paid before retirement. Or, downsize to a smaller home you can afford without a mortgage.

Not only will having your mortgage paid off extend the time your savings lasts, but the home equity in your home also can serve as an emergency fund of last resort since you can tap into the equity if you have a serious financial emergency. 

2. Student loan debt

You'll probably be surprised to learn that seniors 60 and over are the fastest-growing demographic when it comes to taking out student loans. A 2017 report from the Consumer Financial Protection Bureau revealed the number of older borrowers had quadrupled since 2005. Senior-held debts now account for 6.4% of all student loan debts, and seniors collectively owe $66.7 billion in educational debt, with the average amount each senior borrowed doubling from around $12,000 in 2005 to $23,500 in 2015. 

Student loan payments during retirement can be difficult to make. Close to 40% of student loan borrowers over 65 were actually in default in 2015, according to the CFPB. And default rates rise the older you grow. This is very bad news, as student loan debts cannot be discharged in bankruptcy and aggressive collection efforts are permitted that are not allowed with most other consumer debt. Delinquent student loans can even result in your Social Security payments being offset. As many as 40,000 seniors in 2015 lost a part of their Social Security income due to defaulted student loans . 

Most of this debt comes from helping children and grandchildren to pay for their educations. While helping your family is important, your children and grandkids have time to pay off loans. If you're a senior on a fixed income, you can't generate more money to repay these debts and you cannot replace the retirement savings you take out for repayment.  Say no to cosigning or taking out loans in your name, and try to refinance any loans you currently owe for children or grandchildren into their own names as soon as possible.  

3. Credit card debt

Seniors aged 70 and older had an average of $3,780 in credit card debt in 2016 .  Unfortunately, credit card debt is also bad news on a fixed retirement income. Many seniors cannot afford to make extra payments on the money owed, so they spend years in repayment. If you make minimum payments of around $75 monthly to pay off a $3,780 balance on a card with an 18% interest rate, you'd take almost 8 years to pay off the debt and would spend more than $3,300 on interest .

Before you retire, get caught in the minimum payment trap and draw down your retirement savings to pay credit card lenders, try to repay the balance owed. Not only will you have more money during retirement if you can work a little longer to pay off credit card debt, but you'll also have lots of available credit after you're no longer working in case you really need it.  

The Motley Fool has a disclosure policy.