The past few years have been tough on a lot of investors with many tapping into their nest eggs early or shying away from the stock market for fear of loss. Even young workers are making these moves.

About 45% of workers between the ages of 18 and 35 say they don't see a point in saving for their future until things return to normal, according to a Fidelity survey. But there are serious flaws in that logic.

Serious person staring at laptop.

Image source: Getty Images.

What's normal anyway?

The Fidelity survey didn't give a clear definition of what "back to normal" meant. It did mention the COVID-19 pandemic and record-high inflation as two of the major hurdles savers are currently dealing with. However, it didn't specify what needed to happen for workers to feel that things were back to normal again.

Each person likely has a little bit different criteria. But if you're waiting for prices to drop to pre-pandemic levels and the stock market to recover before you start saving for your future again, you could be waiting a very long time.

Even in good times, there's always a risk of loss with investing. You might think you're avoiding that by keeping your money out of the stock market, but there's another risk associated with this too.

How delaying retirement savings costs you

You can't beat the potential returns of the stock market over the long term with any other savings strategy. No savings account or certificate of deposit (CD) can offer you anything close. If you choose to use these accounts rather than investing, you'll have to save a lot more of your own money each month to reach your retirement goals. The same goes for anyone who puts off retirement savings whenever the stock market is experiencing a lot of volatility.

For example, if your goal is to save $1 million of your own money by 65, you'd only have to save about $403 per month if you began saving at 25 and earned a 7% average annual rate of return on your investments. But if you put off saving for just five years, you'd have to save $582 per month, assuming you earned the same average return. That would cost you $51,000 more over the course of your working years, and that number will only grow the longer you wait to begin saving.

Even if you're facing stock market volatility or an impending recession, investing regularly for retirement is still the best move for young workers. You may lose money in the short term, but when you're decades away from retirement, these losses aren't as significant. The shares you buy now when prices are low can increase in value quickly when the market recovers.

It's also still a good idea to diversify your portfolio to help reduce your risk of loss -- but avoid investing too conservatively. A general rule of thumb is to keep 110 minus your age invested in stocks. Aim to have at least 25 different stocks in your portfolio as well, so no single stock affects your nest egg too much. Or you could try investing in an index fund. This is an affordable way to get an ownership stake in hundreds of top companies.

We may never return to the normal we knew before the pandemic, so we just have to do our best from where we're at right now. Try not to let your short-term concerns drive all your decision-making. Focus on the long term and do your best to contribute money to your retirement account every month.