You may love your career, but countless workers look forward to retirement and the less-frazzled lifestyle it offers. In fact, for some folks, the urge to retire is so great that they rush to do so early.

Now, when we talk about retiring early, we generally mean leaving the workforce before reaching full retirement age as designated by the Social Security Administration. Your full retirement age is based on your year of birth, and if you were born in 1954 or earlier, that age is 66. If you were born in 1960 or later, that age is 67. And if you were born after 1954 but before 1960, your full retirement age for Social Security purposes is somewhere in between 66 and 67.

So, when does the typical American retire? According to the U.S. Census Bureau, the average retirement age in the U.S. is roughly 63. But in areas of the country with a higher cost of living, it tends to be higher. Additional data tells us that more than 67% of Americans wind up exiting the full-time workforce by age 66, while 50% leave their jobs between the ages of 61 and 65.

Retirement party

IMAGE SOURCE: GETTY IMAGES.

That said, just because the typical American tends to retire before 66 doesn't mean that it's actually the right move. In reality, the average U.S. worker can't afford to leave the workforce early -- and the sooner more folks realize that, the less financially stressed they'll be in retirement.

We're just not saving enough

The reason the typical American shouldn't retire early boils down to savings, or a glaring lack thereof. The Economic Policy Institute reports that the average American household nearing retirement (aged 56 to 61) has $163,577 saved for retirement. But while that number represents the average, the median savings balance for that age group is only $17,000, which means there are far more people with less than $163,577 than there are with more.

Furthermore, according to IRS data from 2015 (the last year for which such data is available), the average American's household income is $71,258. If we take that number and bump it up a bit to account for typical wage growth, we arrive at an average household income of $74,000 a year.

Now, when we think about how much money we'll need in retirement, it's best to calculate that figure as a percentage of previous income. Though a large number of households actually wind up spending more money, not less, during the early years of retirement, for the most part, having access to 70% to 80% of your pre-retirement income each year should leave you in a fairly secure financial position.

The problem, however, is that the average American hasn't saved nearly enough to support that 70% to 80% income replacement target.

Going back to Social Security, the average recipient currently receives roughly $1,360 a month, or $16,320 a year. Given that one out of every four seniors today will live past age 90, and that the average 65 year old today will live till roughly age 84 to 86, it's fair to say that the typical American who stops working at about full retirement age should plan on needing a minimum of 20 years of retirement income.

So now let's get back to that average savings balance of $163,577. Assuming that's what you have to work with by the time you retire, and that your investments in retirement manage to generate a 4% annual return (which is a pretty reasonable estimate, given that many folks shift their asset allocations away from stocks as they age), over a 20-year period, you'll have about $991 a month in available income, or about $11,892 a year.

When we add in the $16,320 the average beneficiary gets from Social Security, that leaves us with a total of $28,212 -- only 38% of the typical American household income of $74,000. And that's just not enough to live on.

Even if you buff up the numbers by assuming a dual-income household, and take two of those monthly checks from Washington when both spouses retire, the total is only $44,532, about 60% of the average household's income -- better, but not precisely comfortable.

Save now, retire later

So, though you may want to retire early, if you've only saved as much as the average American, you probably shouldn't. Rather, you should use the latter part of your career as an opportunity to pad your nest egg and make up for lost time. Workers 50 and over can contribute up to $6,500 a year to an IRA and $24,000 a year to a 401(k). If you have access to an employer-sponsored plan, and you're able to max out your 401(k) contribution for five extra years, you'll have an additional $130,000 to work with in retirement, assuming those added investments bring in a 4% average annual return during that five-year period.

Even if you're not able to add to your nest egg substantially during those extra years on the job, working longer serves the equally important purpose of letting you leave your savings untouched during that time. In other words, working five extra years means your nest egg will only need to provide income for, say, 15 or 20 years in retirement, as opposed to 20 or 25. And that makes a huge difference.

Another big difference: It gives your nest egg five more years to grow: So if, for example, you have a $163,577 portfolio at age 61, and rather than retire, you work an additional five years but add nothing more, at the end of that time, (assuming that same, conservative, 4% return) you'll have $199,000. Or, contribute $24,000 each year to your 401(k) as suggested above, and you'll be looking at a portfolio balance in the neighborhood of $329,000. Much healthier.

That $329,000 will comfortable provide you with $1,779 a month in available income, or $21,348 a year, rather than the $11,892 a year we arrived at earlier.

Throw in average-sized Social Security checks, and you're either looking at $37,668 a year (for one earner) or $53,988 (for a dual earner household). And that second number works out to be 73% of the average household income, landing you in that target range of 70% to 80% income replacement that we all ought to be aiming for.

One final thing to keep in mind about early retirement is that if you're planning to file for Social Security as soon as you leave your job, you stand to reduce the size of your monthly benefit as well. Claiming Social Security at age 63 when your full retirement age is 66, for example, will permanently lower the size of your monthly benefit checks by 20%. (And that's even before you factor in the increases you might accrue by replacing some of the lower-income, early-career years with higher-income, end-of-career numbers in the government's calculation of your average wages, upon which those benefits are based.)

Of course, not everyone who retires early does so by choice. According to Voya Financial, 60% of Americans end up leaving the workforce sooner than planned, whether due to layoffs, health issues, or the need to care for a family member. But if you do have the option to keep working, and you're behind on savings, you're better off chugging along those few extra years.

Retiring early is a wonderful option for folks who have saved substantially and have robust enough nest eggs to sustain themselves for the long haul. But if that doesn't quite describe your financial situation, it could end up being the worst money move you'll make in your lifetime.