Speaking generally, there is nothing wrong with taking Social Security benefits at the earliest possible time, as the large majority of Americans choose to do.

But there is at least one major exception: You should think twice about doing so if you plan to continue working. Before turning 66, your annual benefits will be reduced by $1 for every $2 you earn above $15,480.

This chart says it all
You can see the impact that continuing to work after 62 (but before 66) has on the typical retiree's benefits in the following chart.

This shows that the average retiree's benefits aren't affected until his or her annual earned income exceeds $15,480. And for every $2 earned above that amount, the annual benefits are reduced by $1.

Now, just to be clear, this assumes that a reduction in benefits will hit a retiree's monthly checks equally. But in reality, your annual benefit is reduced in one fell swoop.

Here's an example from the Social Security Administration that explains how this works (emphasis added):

Let's say you file for Social Security benefits at age 62 in January 2014 and your payment will be $600 per month ($7,200 for the year). During 2014, you plan to work and earn $20,800 ($5,320 above the $15,480 limit). We would withhold $2,660 of your Social Security benefits ($1 for every $2 you earn over the limit). To do this, we would withhold all benefit payments from January 2014 through May 2014. Beginning in June 2014, you would receive your $600 benefit, and this amount would be paid to you each month for the remainder of the year. In 2015, we would pay you the additional $340 we withheld in May 2014.

The good news: Your withheld benefits aren't lost forever
While this sounds ominous, the good news is that you'll eventually recoup any money that's withheld via larger benefits after reaching full retirement at age 66. At that point, your monthly benefit will increase to take into account the months in which benefits were withheld.

Here's a second example from the Social Security Administration that illustrates this point:

Let's say you claim retirement benefits upon turning 62 in 2014 and your payment is $750 per month. Then, you return to work and have 12 months of benefits withheld. We would recalculate your benefit at your full retirement age of 66 and pay you $800 a month (in today's dollars). Or, maybe you earn so much between the ages of 62 and 66 that all benefits in those years are withheld. In that case, we would pay you $1,000 a month starting at age 66.

The point is that your payments won't be lost forever. They're just delayed until full retirement, at which time you're entitled to benefits with no limit on your earnings.

Either way, however, it isn't hard to see why claiming benefits early would be somewhat pointless (unless you're doing so to trigger spousal benefits akin to the claim-and-suspend strategy) if you earn far more than the $15,480 threshold.