The panic has wiped out a huge part of many people's life savings. But plenty of folks haven't batted an eyelash at falling stocks -- because they don't own any.

National Save for Retirement Week runs through this Saturday. Observed by Congress, state and local governments, and private employers, the event aims to encourage workers to make sure they're putting enough aside for their long-term financial needs.

You've already seen the dire statistics. Half of all workers have less than $25,000 saved toward retirement, and around 20% have no savings at all. But I'd like to focus on a different group of retirement savers: those who were smart enough to start gathering a nest egg, but who are now wondering whether the current market will deny them their hopes of retiring successfully.

Second-guessing your retirement planning
Over the past several weeks, I've heard countless stories from people about their frustrations with investing. They did everything they were supposed to -- regularly setting aside money from their paychecks toward long-term goals like retirement, college for their kids, or a down payment on a home. Many of them invested in fairly conservative stocks, blue chips like Johnson & Johnson (NYSE:JNJ) and IBM (NYSE:IBM), and kept their exposure to small-cap and international stocks low.

Yet after all the effort they've made to set aside their hard-earned money, many of them have little to show for it right now. One person I talked to had set aside a few hundred dollars toward his 9-year-old daughter's college fund each month since she was born -- and now, the fund is worth less than the total of all the money he put in all those years. My own 529 plan account balance for my 3-year-old is 20% less than my total contributions.

It's enough to make you think about giving up on the whole thing.

Keeping the faith
Of course, I know better. There've been countless historical examples of times when everyone thought stock investing was dead and buried -- only to see the market rise from the ashes and leave them on the sidelines, having sold at the lows. I watched Coca-Cola (NYSE:KO) go from a quiet, conservative stock to a growth powerhouse during the 1990s. I profited from Altria Group's (NYSE:MO) recovery from overblown litigation risks earlier this decade. I didn't have the guts to buy (NASDAQ:AMZN) back in 2001 at less than an eighth of its current, beaten-down price -- but many people did.

Believe me, I know how hard it is to watch your account balances drop quarter after quarter, with no apparent end in sight. One of the scariest things about the current panic is how no one seems to have any confidence that things will get better.

But that's always how things look during stormy times. When energy prices spiked after the invasion of Kuwait in 1990, many thought higher prices would be permanent -- yet they weren't. When the tech bubble burst, few believed that companies like Amazon would have the second chance at glory they later enjoyed. And now, few are comfortable mulling the long-term prospects of homebuilders like Hovnanian (NYSE:HOV) or banks like Wells Fargo (NYSE:WFC), but they may well come out of this crisis stronger than ever.

Looking back, you'll wonder why you ever doubted it.

Stay on track
As unfortunate as it is that many people never learn to save, it's a far worse calamity when those who have taken responsibility for their own financial well-being just stop trying. There's no denying the difficulty of our current situation is for those who've spent decades accumulating their life savings, only to see large portions of it vanish.

But the answer isn't to give up. Although it may take longer than you'd like, the economy will recover. Stocks of companies that can capitalize on the current weakness will rebound. And those who stick with their investing plans will keep buying shares at extraordinarily low prices -- prices that will help boost their overall returns.

If you're feeling like a sucker for saving for retirement, you're not alone. But you're not a sucker. On the contrary, you're among the rare few who have what it takes to succeed with your money. Don't let the panic push you off your path to financial security -- fight back.

More on retirement saving in tough markets: