Millions rely on Social Security to make ends meet in retirement. But few know that you can actually have to pay tax on your Social Security benefits under certain circumstances.

In the following video, Dan Caplinger, The Motley Fool's director of investment planning, runs through the rules governing whether Social Security benefits are taxable. He notes that the first key is to calculate your investment income and taxable distributions from retirement plans, and then add in half your Social Security benefits. If the resulting figure is above $25,000 for single taxpayers or $32,000 for joint filers, then up to half of your benefits could be taxable. For singles above $34,000 or joint filers above $44,000, up to 85% of benefits can be taxable. Dan concludes by noting that investors in tax-free municipal bonds are often surprised to find that they must add in their income for purposes of the Social Security tax question, making investments in iShares AMT-Free National Muni (MUB 0.16%) and similar tax-free vehicles lose some of their appeal.