Why You Should Never Use Your Retirement for a Down Payment

As tempting as it may be, borrowing from retirement rarely makes financial sense.

One of the most surprising interviews I've ever conducted was with a mortgage broker in Northern California. Although it's been years, I think of him any time housing prices spike. The broker was proud to talk about the number of people he had convinced to cash out their retirement plans to make a down payment on a home. This man was a fast talker, armed with skewed statistics, and unreasonably confident.

He was also dead wrong. It's never, ever a good idea to use retirement savings to get into a home. Here's why.

Early withdrawal fees can eat you alive

There are two ways to take money from your retirement savings before you retire. The first is early withdrawal. Let's say you have a 401(k) account at work and are tempted to raid it. If you're younger than 59 ½ years of age, the IRS considers funds taken from that account an early withdrawal, and early withdrawals are hit hard with fees and taxes. Here's how.

Right away, you'll be hit with a 10% early withdrawal penalty. So, if you withdraw $100,000 to make a 20% down payment on a $500,000 home, you can kiss $10,000 off the top of it goodbye.

Right now, any money you put into an employer-sponsored retirement plan is deducted from your paycheck before taxes. Let's say you earn $125,000 a year and contribute $19,500 annually to your retirement fund. That means you're paying taxes on $105,500 instead of $125,000.

However, as soon as you take money out of a 401(k), IRA, or other retirement account, it's subject to federal and state income taxes. While a few states don't have an income tax, most do. To pour salt into the wound, there's a mandatory 20% federal tax charged on early 401(k) withdrawals. If you're withdrawing $100,000, that's another $20,000 off the top.

Hardship withdrawal leads to additional debt, and debt stinks

One other way to get money from a retirement account is through a hardship withdrawal. Hardship withdrawals are designed for folks facing real problems. For example, someone might borrow money from their retirement account to cover funeral expenses for a loved one or medical treatment. While many employers do not consider buying a home worthy of a hardship withdrawal, some do.

According to IRS limits, you can borrow up to $50,000 or half the amount in your retirement account, whichever is less. Your plan will stipulate how long you have to repay the loan, but it's often up to 25 years when funds are used to purchase a home. Like money borrowed from a bank, you'll have to pay interest.

I have an old friend who somehow got the idea that borrowing from her husband's retirement account was the smartest way to borrow money. "After all," she'd tell me, "We're paying ourselves back with interest."

I cannot tell you the number of times I've bitten my tongue. Yes, they're paying themselves interest, but at what cost? Let's say they're paying their retirement account back at an interest rate of 5%, but if they'd left their managed account alone, it would have provided an average return of 10%. They're out 5% on the money they borrowed.

And what if my friend's husband loses his job? What if he has to leave for medical reasons or is laid off while there's still a balance due? IRS rules give them until tax filing day of the following year to repay the loan in full. Normally, that falls around April 15.

You miss out on the 'good stuff'

The magic sauce that allows your retirement investments to grow is compound interest. For example, let's say you're 37 years old and want to retire at age 67. If you began investing $1,000 each month into a retirement account with a long-term average return of 7%, in 30 years you'd have more than $1.1 million available. And that's not counting any employer contributions.

Any money you withdraw loses steam. Sure, you can contribute more later, but once you've made a withdrawal you either stop or slow its growth.

Any time someone tells you it's okay to take money from retirement to pay for a non-life-threatening situation, ask yourself one question: What's in it for them? Because they're certainly not looking out for you.