Social Security provides tens of millions of Americans with valuable retirement and disability benefits. The revenue for those benefits comes from taxation, and there are two completely separate sets of Social Security taxes that you might have to pay over the course of your lifetime. One is much more common than the other, but both can have a big impact on how much of your paycheck you take home and how much of your benefits you end up keeping.

Social Security payroll taxes

The Social Security tax that most people are familiar with is the payroll tax that gets withheld from your paycheck. Employees have to have 6.2% of their paychecks withheld to go toward Social Security taxes, with total annual wages up to $118,500 being subject to the tax. To that amount, your employer ends up adding a matching 6.2% amount of its own on top of the money it takes out of your paycheck. Those who are self-employed have to come up with the total 12.4% on their own.

You might not see a line item on your paycheck that explicitly says Social Security payroll tax. Instead, some employers refer to FICA, which is the federal law that forces employers to withhold both Social Security and Medicare payroll taxes from their employees' pay. With Medicare taxes amounting to 1.45% and not having a wage limit, your total withholding will amount to up to 7.65% of your earnings.

Income taxes on Social Security benefits

The other tax related to Social Security often comes as a shock to people, because it effectively takes away some of your benefits. In the early days of Social Security, benefits were always tax-free, but in the early 1980s, that law was changed to force some taxpayers to have a certain amount of their benefits included in their taxable income for federal tax purposes.

To figure out how much of your Social Security benefits might be taxable, the first step is to add up all the income that the IRS makes you count toward a given threshold. That includes your earnings from work, investment income, and other sources of taxable income, as well as municipal bond income that is treated as tax-exempt. To that total, add one-half of your Social Security benefits to get what the IRS calls combined income.

What you do with the resulting number depends on your filing status. If you're single and the combined income number is less than $25,000, then you won't have to include any of your Social Security benefits as taxable income. Between $25,000 and $34,000, up to one-half of your benefits get included, and above $34,000, the maximum included rises to 85% of your benefits.

For joint filers, the threshold amounts are slightly different. Your combined income can be up to $32,000 without having to include anything in taxable income. From $32,000 to $44,000, there's a 50% limit, and above $44,000, up to 85% of your benefits can be taxed.

Note, however, that just because you can pay up to those percentages doesn't mean that you will. The IRS worksheet that applies to Social Security taxation is very complicated, but what you'll include as taxable income also depends on how much more than the threshold amounts you earn. For instance, if you're single and have combined income of $25,002, you'll only have $1 of your benefits included in taxable income -- half the $2 by which your combined income exceeds the $25,000 threshold.

Fortunately, you don't have to do the calculations all on your own. This Social Security tax calculator can help do the calculations for you.

Having to pay two different types of taxes related to Social Security strikes many people as unfair. Nevertheless, you need to know about both taxes in order to make sure you don't get surprised at the amount of money taken out of your paycheck -- or that you have to pay the IRS with your annual tax returns after you retire.