Image source: Getty Images.

It's natural to dream of retiring early -- such as at age 66 (the full retirement age for those born between 1943 and 1954, per the Social Security Administration) or earlier -- and many of us hope to do so. There are plenty of good reasons to, such as being able to spend more time with family, more time traveling, and more time simply doing whatever we'd like to do. However, there are also some good reasons to not retire early.

Image source: Getty Images.

1. You might not be able to afford to.

Retiring while you're still relatively young can be great, but not if you're stressed out about how to make ends meet -- or not if you enjoy the first few years before realizing you're running out of money.

According to the 2016 Retirement Confidence Survey, among those aged 50 or older, only 30% had saved $250,000 or more for retirement, while fully 27% had saved less than $10,000. That suggests that millions are not likely to have enough to retire on at 66 -- or even when they're older. Consider, for example, the conventional rough guide that you might withdraw about 4% of your nest egg annually in retirement (adjusting for inflation over time). With a nest egg of even $400,000, that would give you just $16,000 for the year. Note that the average Social Security income is about $16,000 a year, too.

Image source: Getty Images.

2. You can sock away more in your retirement war chest.

Fortunately, if you delay retiring by a few years, you can work wonders in your financial situation. Imagine, for example, that you have that $400,000 in your retirement account at 66. If you work three more years and retire at 69 and your nest egg grows by an annual average of 8%, it will turn into more than $500,000. That's a big difference -- enough to generate an additional income of about $4,000 if you withdraw 4%.

By working a few more years, you'll be able to save and invest a lot more money, and you'll put off starting to draw down your nest egg, too. Thus, it will have to sustain you for fewer years and it will last longer. In addition, if you're enjoying employer-sponsored health insurance, you can keep doing so for a few more years, helping you save money on the healthcare expense front as well.

Image source: Getty Images.

3. You can collect more from Social Security if you wait.

Delaying retirement can also permit you to put off starting to collect Social Security. For every year between your full retirement age and 70 that you delay, your payout will grow about 8% larger -- up to a total of 32% larger if you delay from age 66 to 70. That can turn an expected monthly check for $1,500 into one for almost $2,000, a meaningful difference.

It's not necessarily quite so valuable to delay starting to collect Social Security, though -- at least not for many people. As the Social Security Administration has explained, "If you live to the average life expectancy for someone your age, you will receive about the same amount in lifetime benefits no matter whether you choose to start receiving benefits at age 62, full retirement age, age 70 or any age in between." In other words, for those who live average lifespans, it's pretty much a wash. After all, the checks you get if you start collecting early might be a lot smaller, but you'll get a lot more of them. And if you delay until age 70, you'll get bigger checks but fewer of them.

You might be in excellent health, though, and perhaps your family tree is full of nonagenarians. If so, aiming for bigger Social Security checks can be smart.

Despite these reasons to not retire early at age 66, it can still make sense to do so -- if you can afford it. Crunch your numbers, perhaps consult a financial advisor, and see what seems best for you.