Now more than ever, millions of workers are afraid that they'll never be able to retire. With a huge chunk of those workers having put next to nothing toward their retirement savings, they'll be counting on Social Security and any other income sources they may have to make ends meet -- or earning a paycheck as long as possible.

But even those who have already saved up a considerable nest egg can't afford just to assume that everything's going to work out OK. The federal budget looks ugly, and you can count on Uncle Sam to make sure he gets his cut of your savings any way he can.

The flip side of the best tax break ever
During your career, IRAs and 401(k) plans seem like the best deal ever. You get to put your money into a tax-sheltered account where you don't have to worry year in and year out about how much in interest, dividends, and capital gains your investments generated. What happens in your IRA stays in your IRA -- at least until you retire. And even better, the government gives you a tax break for doing so, letting you write off traditional IRA and 401(k) contributions entirely against your regular taxable income.

But eventually, you have to pay the piper, and the cut can be huge. When you take money out of your IRAs, you have to pay income tax on the withdrawals. As a result, assuming you don't have money elsewhere to pay the taxes, you have to pull out more money than you need to spend just to cover the tax bill. For example, if you're in the 25% tax bracket and need to spend $45,000 from your retirement accounts every year, you'll have to take out $60,000 -- with $15,000 of that going straight to the IRS.

What's especially egregious is that you have to pay your higher ordinary tax rate, even if the income within your retirement accounts came from capital gains, dividends, or other income that enjoys preferential treatment in taxable accounts. That makes holding high-growth stocks, as well as high-dividend stocks like Frontier Communications (NYSE: FTR) or Telefonica (NYSE: TEF) inside an IRA less attractive, because you effectively lose the tax breaks that their income would ordinarily give you. Even Apple (Nasdaq: AAPL) shareholders can get the worst of both worlds by having kept long-held shares in a traditional IRA versus a taxable account.

How to solve the problem
Fortunately, there are some smart tactics you can use to cut your tax bill. First and foremost, be smart about which investments you make in your various accounts. For instance, mortgage REITs like Annaly Capital (NYSE: NLY) and American Capital Agency (Nasdaq: AGNC) make great IRA holdings because their dividends don't typically qualify for lower tax rates. So you aren't losing anything by holding them in your regular IRA. By contrast, consider putting true growth plays, such as biotech upstart Inhibitex (Nasdaq: INHX) and video game maker Majesco Entertainment (Nasdaq: COOL), into either a Roth IRA (where their income can be tax-free) or a taxable account (where they could qualify for 15% long-term capital gains taxes).

Second, once you've retired, realize that paying some tax is inevitable -- so take advantage of opportunities to pay as little as possible. For instance, if you start out retirement in a low tax bracket, take money out of your retirement accounts even if you don't need to spend it. By using up your low 10% and 15% brackets now, you could save yourself from bigger taxes of 25% or more down the road, when the IRS forces you to start taking money out of your retirement accounts. Just stick the money in a taxable account -- or even better, convert to a Roth IRA, where you can enjoy tax-free growth as long as you keep the money in the account.

Last, keep control of your expenses as well as you can. Retirement is more expensive than you may think, so look into what you'll want to pursue before you retire and build a realistic budget for your own dreams.

Keep your money
After a career of working hard, you deserve to keep as much of your hard-earned money as you can. By being tax-smart, you can stop the IRS from taking more than its fair share.

Once you've got your taxes figured out, though, you're not done. Our latest special free report reveals the shocking can't-miss truth about your retirement -- and what you can do about it. Grab a copy today and find out everything you need to know.