Many people dream of retiring early and leaving the workforce well ahead of their peers. But new data from TD Ameritrade indicates that younger workers may be a bit overzealous with regard to retirement. In a just-released survey, millennials reported that they expect to retire at an average age of 56. But seeing as how only 38% are saving for retirement at present, that's a goal few are on track to achieve.

If you're eager to retire early, whether it's in your early 60s, mid-50s, or even your 40s, you should know that there are steps you can take to get there. For example, if you're willing to start saving from a very early age and invest your money aggressively, you can accumulate a sizable nest egg over the first few decades of your career and then enjoy that income stream at a time when your counterparts remain chained to their desks.

Young male adult at a laptop with mug in hand

IMAGE SOURCE: GETTY IMAGES.

Case in point: Setting aside $1,000 a month from age 22 until 52 will leave you with $1.13 million if your investments generate an average annual 7% return during that time. And if you go heavy on stocks, that sort of average is more than reasonable. But if you continue letting your nest egg fall by the wayside, your chances of retiring ahead of the game are only going to get slimmer with time.

The challenges of retiring early

One major problem with retiring early is not having access to affordable healthcare. Medicare eligibility doesn't kick in until age 65, which means that if you retire prior to then, you'll need to cover the cost of insurance on your own. And that can be prohibitively expensive. Another issue to consider is that you can't file for Social Security until age 62 at the earliest, so if you retire in your 50s, you won't have that income stream to rely on.

Now if you've saved well enough to afford a health plan and get by without Social Security, you're all set. But make sure that's really the case, because while a $1.13 million nest egg (as per the example above) might hold up in conjunction with Medicare and Social Security, it's actually not all that much money on its own.

Remember, 4% has long been the standard withdrawal rate for ensuring that a nest egg lasts. But when we apply that rate to $1.13 million, we get about $45,000 of income per year. Now that's not terrible, but when you're paying, say, $7,000 a year for health insurance and are losing a portion of that income to taxes (which will be the case if your retirement savings aren't housed in a Roth account), that's not a ton of money to work with.

Furthermore, that 4% withdrawal rate is designed to make your savings last for 30 years, but if you retire in your mid-50s, there's a good chance you'll need more than three decades' worth of savings. It's estimated that one in four 65-year-olds today will live past the age of 90, while one in 10 will live past 95. Therefore, you might be looking at a much less aggressive withdrawal rate and a much smaller amount of income in turn.

Can you pull off early retirement?

Still, the above issues can be overcome if you make saving for retirement a priority from the moment you start working. The problem, however, is that most younger employees aren't doing that. In the aforementioned survey, millennials said they plan to start saving for retirement at an average age of 36. But that means losing out on over a decade of growth for those who graduate college in their early 20s.

Imagine you start funding your nest egg at 36 with the goal of retiring 20 years later. Even if you set aside $1,000 a month, you'll end up with just $492,000 if we apply the same 7% return we used above. Again, the final number might seem impressive. But at a 4% annual withdrawal rate, that's less than $20,000 a year of income. And until Social Security kicks in, that's probably not enough to live on.

Now let's imagine you can't manage to part with $1,000 a month, because let's face it -- that's a lot of money. If you save just $500 a month over a 20-year period instead, you'll end up with $246,000 if your investments grow at 7% a year. And that'll give you less than $10,000 in annual retirement income, which, again, won't nearly cut it.

Though it's nice to see that millennials are confident in their ability to retire, it's clear that many need a reality check. Retiring early is possible, but the longer you wait to build your nest egg, the lower your chances of getting to do it. So if you really want to call it quits in your mid-50s, you'll need to start saving as much as you can, as soon as you can, rather than ignore your nest egg until your mid-30s.