Retiring comfortably takes a lot of money -- often $1 million or more. You don't need a six-figure income to save that much, but you do need a plan and the discipline to stick to it. Yet many people make shortsighted choices that dash any hopes of retiring with enough money. Here are three mistakes you can't afford if you hope to retire wealthy.

1. You're guessing at how much you need to save 

Study after study has shown that people tend to underestimate how much they need to save for retirement. Transamerica found that baby boomers and Gen Xers estimate that they'll need $500,000 on average for retirement, while millennials believe they'll be able to get by with just $400,000. But the average retiree's spending habits tell a different story.

Confused couple going over finances

Image source: Getty Images.

The Bureau of Labor Statistics says that the average household headed by an adult 65 or older spends nearly $50,000 per year. If your retirement lasts 20 years and you spend that much each year, you'll need $1 million to cover all your expenses. You don't have to save all of that on your own because Social Security will cover some of it, but $400,000 to $500,000 in personal savings probably isn't going to be enough. If you live into your 90s, which is becoming increasingly likely thanks to advances in medical care, your retirement could last even longer than 20 years, costing you even more.

To avoid depleting your retirement savings while you're still alive, you need to come up with a personalized estimate of how much you need to save. Start by subtracting your ideal retirement age from your estimated life expectancy (plan to live into your 90s if you're reasonably healthy). That will give you the number of years you should plan to be retired.

Next, total up your estimated annual living expenses, keeping in mind that some expenses, like healthcare, may go up, while others, like child care, may go down or even disappear. Multiply your living expenses by the number of years of your retirement, adding 3% annually for inflation. A retirement calculator will do this part for you. Use 5% to 6% for your annual investment rate of return to be conservative. Your calculator will then tell you how much you need to save overall and per month to hit your goal.

Subtract from these totals any money you expect from an employer 401(k) match, a pension, or Social Security. You can estimate your Social Security benefit by creating a my Social Security account if you're not sure how far your benefits will go in retirement. The remainder is how much you need to save on your own.

2. You're not saving regularly.

The best-laid retirement plan is worthless if you don't follow through with the savings portion of it. Automate your savings if you struggle to remember to set aside the funds on your own. Your 401(k) should enable you to earmark a percentage of your income for retirement each pay period, and your IRA may give you an option for recurring contributions as well. 

Consider reworking your budget if you're not saving for retirement because you can't afford to. Look to cut your monthly expenses by dining out less, canceling subscriptions you no longer use, and finding other trims. Or you could look for ways to boost your income, like getting an extra job or working overtime. If none of that works, you might have to redo your retirement plan. Delaying it by a few months or years reduces how much you need to save while also giving you more time to do it. It could also boost your Social Security checks if you're earning more in your later years than you were in the early years of your career.

3. You're withdrawing money from your retirement accounts to cover other expenses.

In most cases, you'll pay a 10% early withdrawal penalty, plus income tax if the money came from a tax-deferred account, when you withdraw money from your retirement account before age 59 1/2. There are exceptions to this rule for a first-time home purchase, educational expenses, or if you take Substantially Equal Periodic Payments (SEPPs), among other things. But while you may avoid the obvious penalty, you're still hurting your savings' long-term growth, even if you pay that money back with interest over time.

Imagine you borrow $10,000 from your 401(k) for a first-home purchase. You must pay it back with 6% interest over 10 years. You'll end up repaying the initial $10,000 you borrowed, plus another $3,322 in interest for a total balance of $13,322. But if you'd left that money in your retirement account and it had earned a 7% annual rate of return, that $10,000 could've been worth $14,203 after 10 years. And the difference between the two amounts could be even greater if you borrow more, take a longer loan, or have a greater spread between your retirement plan's loan rate and the annual rate of return on your investments.

Avoid dipping into your retirement accounts except as a last resort. Save in an emergency fund to cover unexpected expenses like medical bills or job loss. For large purchases like a home or car, budget a certain amount toward this each month and delay your purchase until you've saved enough to hit your goal. 

If you avoid the above mistakes, you should be able to save enough to last the rest of your life. More importantly, you'll have the peace of mind in knowing you're prepared.