One reason people are discouraged from taking Social Security early is because outside income can dramatically reduce their benefits prior to "full retirement" at 66. What's important to keep in mind, however, is that this doesn't apply to investment income.

When the Social Security Administration figures out how much to deduct from your benefits before your 66th birthday, it counts only the wages you earn from a job and/or the net profit if you're self-employed. In other words, it doesn't count pensions, annuities, investment income, interest, or other classes of retirement benefits.

A black calculator on top of a Social Security card and an income statement.

Image source: Getty Images.

The net result is that if you're thinking about retiring early, then there's no reason to be concerned that your Social Security check will be negatively affected by income from your investment or retirement portfolio. To learn more, check out the following video, in which Motley Fool contributor John Maxfield explains the topic further.