Even if you expect to live a fairly modest lifestyle in retirement, you'll still need money to pay for it. In fact, Social Security will generally do a good job of replacing 40% of your former income, more or less, if you're an average wage-earner. But most seniors need 70% of their former income or more to live comfortably.

Even if you're willing to cut corners as a senior and move to a part of the country where the cost of living is cheap, there's a good chance you still won't manage to get by on Social Security alone. Those benefits pay the average recipient around $18,000 a year today, so even if you're amenable to living in a small home and sticking to local entertainment, you may find that $1,500 a month just won't cut it when you factor in essential expenses like transportation, healthcare, and food.

It's for this reason that saving for retirement independently is crucial; you need some amount of extra money to supplement your Social Security income. Yet 21% of Americans have not yet started saving for their senior years, according to a new FinanceBuzz survey, and alarmingly, that includes almost 16% of Gen Xers and more than 12% of baby boomers.

Man at laptop with serious expression

Image source: Getty Images.

If you've yet to start building your nest egg, you should know that the longer you wait, the harder it will be to grow wealth for retirement. On the other hand, if you start making modest contributions to a savings plan today, you stand a better chance of accumulating enough of a balance to avoid financial struggles later in life.

Why it pays to start saving for retirement immediately

When it comes to growing retirement wealth, your greatest weapon is none other than time. The more years you have to invest your savings, the more money you stand to end up with. That's why delaying your savings is a bad idea. Even if you're only in a position to start socking away $50 or $100 a month right now, over time, it will make a big difference.

In fact, let's say you're 27 years old and think you'll retire at 67, which is full retirement age for Social Security purposes. If you start setting aside $100 a month in an IRA or 401(k) now, and continue doing so for 40 years, you'll wind up with about $240,000, assuming your invested savings deliver an average annual 7% return during that time. That return is a few percentage points lower than what the stock market has averaged over time, and it's a reasonable assumption for someone with a 40-year savings window.

Of course, if you're no longer in your 20s, you may need to do a lot better on the retirement savings front than $100 a month. Say you're 42, with 25 years left until retirement. In that case, it will take a little over $300 a month in contributions to accumulate around $240,000 in savings, assuming that same 7% return. The point, however, is that the sooner you begin funding an IRA or 401(k), the less money you'll need to contribute to build wealth, because once that money is in your retirement plan, you'll get to put it to work immediately.

Finally, remember that there's nothing wrong with starting really small on the retirement savings front and working your way up as your income increases. If you can only swing $20 a month right now, go for it. You may be in a position to do much better in a couple of years from now, but that way, you'll at least get the ball rolling.