Most people dread doing their taxes every year, whether or not they owe the IRS money. Thanks to the insane complications of the federal tax code, just filling out the return can be an enormous hassle. But if you're tempted to skip filing this year, think again -- not filing your tax return can put you in hot water.

Failure-to-file penalty

The IRS considers not filing a return to be a more heinous offense than filing and not paying. The late-payment penalty is 0.5% of the unpaid taxes per month; the failure-to-file penalty is 10 times greater at 5% of the unpaid taxes per month.

Both penalties are capped at 25% of the unpaid taxes -- each. So if you owe $1,000 in federal income taxes for the year, and you don't file your return or pay, you could end up owing an extra $500 in penalties. If you file the return but don't pay, the most you could owe for these penalties is $250 (plus interest).

IRS headquarters

Image source: Getty Images.

Substitute returns stink

Once the IRS figures out that you should have filed a tax return but didn't -- which may take them several years -- it will make a tax return up for you. This IRS-generated tax return, called a substitute return, will include all sources of income that the IRS can find for you. However, the agency won't exactly strain itself to identify deductions or tax credits that you might qualify for.

In other words, the substitute return will probably result in a much higher tax bill than a return you created yourself for the year. And because interest and penalties are based on a percentage of the tax bill, those charges will be higher as well.

Losing out on refunds

If you finally get around to filling out your tax return, you may be pleasantly surprised to find that you actually have a refund coming to you. However, there's a three-year time limit on claiming those refunds. If you wait too long to file your tax return for that year, you could miss out on a whole lot of money.

Indeed, the IRS says it's sitting on more than $1 billion in unclaimed refunds. Is some of that money yours?

Losing Social Security credits

For the self-employed, there's an extra risk associated with not filing your taxes. Your tax return is the only way that the federal government will know exactly how much income you made for the year and, therefore, how many Social Security credits you accrued. If you don't file your taxes, you won't get any income reported to Social Security for the year -- which could cost you big-time once you retire.

Damaging your credit (and your reputation)

If you continue not to pay your taxes, the IRS will eventually start taking really nasty measures to get its hands on that money. The agency might slap a levy on your income, meaning it will siphon money right out of your paycheck before you can even touch it.

The IRS will contact your employer to set up such a levy, meaning you will likely end up having an embarrassing conversation with your boss. The IRS can also file a notice of federal tax lien, which will appear on your credit report and almost certainly cause a dramatic drop in your score.

Going to jail

In extreme circumstances, the IRS may actually charge you with tax evasion and haul you off to prison. This isn't a common turn of events: If the federal government imprisoned everyone who didn't file their taxes, the U.S. prisons would fill up awfully fast.

Usually, the agency only pursues criminal charges against tax evaders who are either very rich, very famous, or both, like Wesley Snipes. The IRS is more likely to pursue civil charges for tax-evasion cases, which won't involve jail time, but will involve enormous fines and penalties.

Getting those returns filed

Now that you know all the horrible ramifications of not filing your tax return, you're likely eager to get any outstanding returns taken care of. The IRS offers a Free File software program for any taxpayer who meets certain income limits, but you can only use it for the most recent tax year. Older returns can't be e-filed at all -- you'll have to use the old-school approach of sending them in by snail mail.

If you have several years' worth of tax returns outstanding, or you owe a great deal, you could save a lot of money by hiring an experienced, credentialed tax advisor (an Enrolled Agent or CPA). Plus, your advisor can deal with the IRS and its representatives for you. Now that's a service that's definitely worth the money.