Image source: Getty Images.

One of the most critical financial decisions that you'll make during your lifetime is when to start taking Social Security benefits. Workers become eligible to take retirement benefits at age 62, and more people choose to take their Social Security then than at any other age. However, even though taking Social Security at your earliest opportunity is a popular choice, there are several situations in which you should consider waiting to take your benefits until a later age. Below, we'll look at three reasons why delaying your Social Security benefits can be your best choice.

1. You're still working and will forfeit your benefits anyway.

If you decide to take Social Security before you retire and you haven't yet reached your full retirement age -- currently 66 for those who turn 62 this year -- then there's a limit to how much you can earn before Social Security will take away some of your benefits. If you are under full retirement age all year, then the maximum you can earn is $15,720. Above that, you'll lose $1 of annual benefits for every $2 you earn over the limit.

A special rule applies to those who reach full retirement age during 2016. For them, the limit is $41,800, and it only applies to your earnings before you hit full retirement age. So even if your total earnings for the year are above the limit, you don't have to forfeit benefits if your earnings prior to turning 66 are less than $41,800. Also, the forfeiture amount is smaller, as you'll lose $1 in annual benefits for every $3 earned above the limit.

If you hit age 62 mid-year, then another special rule applies. Earnings from before your 62nd birthday don't count unless you earn more than $1,310 in any month after you turn 62.

If you're going to forfeit your benefits, then there's little reason to take them in the first place. Instead, delaying your benefits avoids the hassle of dealing with whether your earnings will be too high.

2. You have high levels of taxable income and don't want your Social Security taxed as well.

Another reason to delay Social Security is if your taxable income is high enough to trigger income tax on your benefits. This can happen regardless of whether you're still working.

Whether Social Security benefits are taxable depends on what you receive from various income sources. To start, take your income from compensation, investments, pension income, taxable IRA distributions, and other taxable sources, and add in tax-exempt interest. Then, add one-half of your Social Security benefits. If you're single and the number is above $25,000, then up to half of your Social Security benefits can be subject to tax. If it's above $34,000, then the maximum income taxed rises to 85%. The corresponding numbers for joint filers are $32,000 and $44,000, respectively.

Sometimes, if you expect your income to fall in future years, delaying Social Security will prevent it from ever getting taxed. That's good news for those who are counting on using every penny of their benefits.

3. You don't need the money and would prefer larger monthly payments later.

Many people who start receiving Social Security benefits at age 62 don't have much choice because their finances are stretched to the limit. If you're fortunate enough not to need the money right now, then delaying can give you larger payments down the road when you'll appreciate them more.

By taking Social Security at age 62, your monthly check will be 25% smaller than it would be if you waited until full retirement age of 66. By contrast, if you wait beyond full retirement age, you get 8% per year. That can add up to a 76% swing by waiting, as you can find in the chart below.

Image source: Social Security Administration.

$1,320 divided by $750 is 1.76, representing a 76% increase.

The trade-off is that because you take payments later, you'll get a smaller number of payments over your lifetime. Whether they eventually catch up because of their larger size depends on how things work out. But if you don't really need the money in your early and mid-60s, then delaying ensures that you'll get more money at a time when you'll value it more.

Millions of Americans take Social Security at age 62, figuring that whenever they can start getting money is a good thing. However, in situations like these, delaying your Social Security benefits can be the better choice.