Millions of Americans have been impacted financially during the coronavirus pandemic. Some have seen their savings increase as a result of work and lifestyle changes, while others have seen their income decline to a dangerous extent.

If you landed in the latter camp, you may have needed to take an early withdrawal from your retirement plan to make ends meet. The CARES Act, which was signed into law in late March, allowed for up to a $100,000 penalty-free withdrawal from an IRA or 401(k) for those impacted by the pandemic. And new data from Vanguard shows that some savers took advantage of that option.

In fact, an estimated 4.5% of Vanguard 401(k) savers took a coronavirus withdrawal between March and September, and among them, the median withdrawal amount was $12,000. On the one hand, a $12,000 withdrawal isn't awful if you consider that savers have the option to remove up to $100,000. On the other hand, even a $12,000 withdrawal could have negative impacts on retirement down the line.

Man at laptop with serious expression

Image source: Getty Images.

Why even modest withdrawals can hurt retirement savers in the long run

If you took something in the ballpark of $12,000 out of your retirement plan to cope with the coronavirus pandemic, you might assume that it's not such a huge deal. While $12,000 or anything in that vicinity certainly isn't nothing, it's also a far cry from $100,000.

But remember, the money in your retirement plan doesn't just sit there doing nothing. Rather, you can invest it for added growth.

Imagine you go heavy on stocks in your retirement plan so that your investments manage to generate an average annual 7% return. Let's also imagine you've taken a $12,000 retirement plan withdrawal at age 35 but don't intend to retire until age 67. This means that your missing $12,000 will actually leave you about $104,600 short in retirement when you factor in lost investment growth.

If you had to take a CARES Act withdrawal to cover immediate expenses, there's no need to beat yourself up over it. But what you should do is aim to replace that money once your financial circumstances improve. If you boost your savings rate once you're back on your feet, you may be able to make up for any withdrawal you took -- even a sum much larger than $12,000.

Don't short yourself on savings

Once you retire, you should expect to need at least 70% of your former annual income to live comfortably. Social Security may replace up to 40% of your previous wages if you're an average earner and benefits aren't cut in the future.

But that means you'll still need a substantial amount of money from savings, so if you took a CARES Act withdrawal -- even a modest one -- do your best to pay yourself back over time. It may take a while for you to recover from the effects of the pandemic, but once that happens, focusing on boosting your savings could help you avoid financial difficulties later in life.