Many people who don't invest make that choice because they don't see much point in making a financial sacrifice here and now. Their mindset is, that little bit of money invested now could never grow into a meaningful nest egg later.

In truth, however, the potential net returns on even the smallest of annual contributions can be surprising. The key is getting started as soon as possible and letting time do the bulk of the work.

Just don't get crazy with your expectations.

Compounding is the key

Most people starting out in life don't have a lot of extra money to pour into long-term investments. Student loans, first houses, and entry-level incomes just all work against saving for retirement.

The thing is, you don't have to contribute a fortune right now to become relatively rich later in life. Even the smallest yearly contributions to a retirement fund can grow to a small fortune over time.

As an example, check out the growth of a yearly investment of $5,000 in an S&P 500 (^GSPC 1.03%) index fund within a tax-deferring IRA. Assuming it continues to log the index's average annual gain of 10% and that you reinvest any dividends the fund pays, after three decades you'd be sitting on a stash worth a little over $900,000.

Notice how most of the growth takes shape in just the last few years of the 30-year stretch. That's the power of compounding, or earning money on your previous, cumulative investment earnings.

Chart showing the growth of a $5,000 annual investment in an S&P 500 index fund earning 10% for 30 years.

Data source: Calculator.net. Chart by author.

Not bad, except for one thing: This model doesn't actually multiply $150,000 worth of net contributions tenfold. That figure of $900,000 is only around six times what this hypothetical investor has put into the market. How do you get 10 times your initial inputs?

Well, you can't -- at least not anywhere close to the current scenario. You'd need to work and contribute $5,000 per year for 38 years to multiply your money by 10; $190,000 worth of contributions would be worth $2 million at the end of that 38 years.

However, that's a long time. Not everybody wants to work that much of their lives. Heck, not everybody can work that much of their lives.

And curiously, regularly upping your annual contribution over time doesn't do the trick either (although it certainly doesn't hurt). Raising your yearly inputs merely raises the baseline comparison, with most of that increase also taking shape in the latter portion of the time frame in question. These increased inputs don't do you nearly as much good then. You need these bumped-up contributions earlier to maximize the impact of compounding later.

This means the only realistic means of driving tenfold growth of steady, annual contributions into a tax-deferring retirement is by improving your overall average return. Simply bumping it up to 12.5% per year would do the trick, turning $150,000 worth of cash inputs into $1.5 million within 30 years. Again, the chart shows how most of the growth came at the very back end of the time frame when compounding really kicks in.

Chart showing the growth of a $5,000 annual investment in an S&P 500 index fund earning 12.5% for 30 years.

Data source: Calculator.net. Chart by author.

Problem? As experienced investors can attest, the quest to squeeze out that last 2.5 percentage points worth of average annual gain comes at great risk -- and often, regret.

It's just plain tough to beat the market

It's a conundrum, to be sure. An average return of 12.5% doesn't feel dramatically bigger than the S&P 500's average yearly gain of 10%. Surely a little more savvy and finesse will put such returns within reach, right?

Not really. More often than not, the effort to beat the market -- no matter how marginally -- leads to subpar results.

Don't believe it? Chew on this: Not even the professionals can consistently do it. Standard & Poor's regularly updated report card on the matter indicates that a little less than half of the mutual funds offered to U.S. investors actually outperformed the S&P 500 last year. And that was a year they had help, by virtue of performing less badly.

Remember, the index lost ground last year. A fund manager only had to be mostly on the sidelines to fare better than the benchmark. Stretching the performance comparison out to five, 10, and 20 years consistently leaves less than 20% of funds doing better than the S&P 500.

If the pros can't do it, imagine how tough it is for amateur investors to do it.

Make realistic retirement projections

None of this is to suggest it's impossible to beat the market. People do it. Never say never.

Rather, the takeaway here is that proper planning and establishment of realistic expectations is a key part of any investing plan. Don't be so motivated to multiply your net retirement contributions by a specific number (like 10) that you end up missing the mark by a mile. If you can grow your total net inputs between five and eight times over a three-decade span, you're actually doing pretty well -- better than many, in fact.

But it's just not in your blood to at least try? Here's an alternative plan: Commit half of your retirement savings to an index fund that reflects the market's long-term bullish tide, and put the other half into quality growth stocks for the long haul.

At least this strategy circumvents the biggest pitfall of trying to beat the market, which is trading too often. See, even veteran investors have a penchant for getting in and getting out of individual stocks at the worst possible time.

Mostly, though, keep your expectations in check. There aren't too many people who will multiply their money by a factor of 10 by making the yearly contributions most of us are able to make. You'd need to start out with a big chunk of money to have a great shot at making that happen.

Looking to start doing just that? Here's a look at how to calculate how much you'll need to retire.