We all know taxes are unavoidable, but you may not realize how much of your money the IRS gets its hands on. Here are five things you may be surprised to learn are taxable.

IMAGE SOURCE: GETTY IMAGES.

1. Social Security benefits

While some people don't have to pay taxes on their Social Security benefits, yours might be taxed if you have additional income coming in. To see if you'll pay taxes on Social Security, you'll need to figure your provisional income as follows:

  • Start by taking your total income not including Social Security
  • Next, add any tax-free interest you receive (like what you'd get from municipal bonds)
  • Finally, add in 50% of your Social Security benefit amount

If your total falls between $25,000 and $34,000 and you're a single filer, you could be taxed on up to 50% of your benefits. The same applies if that number falls between $32,000 and $44,000 and you file taxes jointly. Furthermore, if your provisional income is greater than $34,000 as a single filer or $44,000 as a couple filing jointly, then you could be taxed on up to 85% of your benefits.

Not only can the federal government tax Social Security benefits, but the following 13 states tax Social Security benefits to some degree:

  • Colorado
  • Connecticut
  • Kansas
  • Minnesota
  • Missouri
  • Montana
  • Nebraska
  • New Mexico
  • North Dakota
  • Rhode Island
  • Utah
  • Vermont
  • West Virginia

If your state is on this list, expect to fork over a portion of your benefit payments.

2. Retirement plan withdrawals

Be prepared to pay taxes on whatever income you withdraw from your traditional 401(k), traditional IRA, or pension plan. Even annuity income is taxable to a certain extent. Your retirement plan withdrawals are taxed at your ordinary income-tax rate, and unless you're dealing with a Roth account, you're required to start taking distributions once you turn 70-1/2.

Note that withdrawals from Roth IRAs and Roth 401(k)s are tax-free.

3. Gambling winnings and prizes

Had a good day at the track or the slots? Don't spend your winnings just yet, because the IRS will want its share. In fact, you're required to pay taxes on both cash and non-cash winnings.

Cash winnings can stem from:

  • Lottery payouts
  • Poker
  • Slot machines
  • Horse races

Whoever pays your winnings is required to provide you with a Form W-2G, provided you come away with:

  • $600 or more at a track if that amount is at least 300 times what you bet
  • $1,200 or more from a bingo game or slot machine
  • $1,500 or more in winnings from keno
  • $5,000 or more from a poker tournament 

Non-cash winnings, like a car or vacation, must be reported on your taxes as well. To do so, take the fair market value of your prize when calculating your income for the year.

4. Unemployment income

Unemployment benefits help countless people who lose their jobs stay afloat financially as they look for work. But if you're receiving benefits, get ready to lose a portion to taxes. Generally, you're given the option to withhold taxes up front or down the line. Choosing the latter might put more cash in your pocket while you're out of work, but be prepared to owe that amount when you file your tax return.

5. Forgiven debt

Many people rack up debt to the point where they can't repay it in full. When that happens, a lender will sometimes be willing to settle for a smaller amount and forgive a portion of the total debt owed. If this happens to you, then you may be in for an unpleasant surprise come tax season. Although forgiven debt never shows up as actual cash in your pocket, in many cases, you're still required to pay taxes on the amount you no longer owe. For example, if you have a $20,000 credit card balance and your lender forgives half that amount, you'll end up owing taxes on $10,000 unless you qualify for an exception, like bankruptcy.

Countless tax filers get caught off-guard each year when they wind up owing the IRS money for things they never knew were taxable. Now that you're aware of the income sources that fall within the IRS' reach, you can take steps to plan accordingly and avoid trouble when the time comes to file your taxes.