You'll often hear that if you were to give up store-bought coffee, you'd bank enough money for your dream retirement. Well, that's just not true.

Cutting back on store-bought coffee might save you somewhere in the ballpark of $5 a day -- maybe less, if you opt for simper drinks over fancy lattes -- but in the course of a year, that's $1,825. Save that much annually over 30 years, and even if your investments do fairly well, generating an average annual 7% to almost mimic the stock market's average, you'll still wind up with just $172,000 and change -- a decent chunk of cash, but hardly enough for the comfortable retirement you're probably hoping for.

Fancy cup of coffee

IMAGE SOURCE: GETTY IMAGES.

Still, GOBankingRates data shows that Americans are probably spending more money on coffee than they should be. Currently, folks who love their caffeine spend an average of $1,100 on it per year. Yet on average, Americans only contribute around $2,000 a year to their IRAs or 401(k)s.

Furthermore, an estimated 34% of U.S. adults have absolutely no money in savings. That's troubling, considering that everyone, from recent college graduates to retirees, needs emergency savings to the tune of at least three months' worth of living expenses.

If you tend to spend upward of $1,000 on coffee per year, it may be time to rethink that habit -- or find another way to boost your savings without having to give up the one luxury you're probably loathe to forgo.

Is coffee really killing your finances?

Let's be clear: Giving up store-bought coffee won't necessarily spell the difference between paying the bills in retirement or not. But it could help.

Imagine you currently sock away $2,000 a year like the typical American in your IRA or 401(k). Over 30 years at an average annual 7% return, you're looking at about $189,000. That's not a ton of money to retire with. But when you add in the $172,000 you could amass by giving up store-bought coffee for 30 years and brewing it at home for pennies instead, your total rises to $361,000. That's a far more comforting sum. As such, you may want to consider cutting back on your coffee-buying habit if your nest egg needs a boost, and you can't easily identify other expenses to cut to make that happen.

Furthermore, if you're without an emergency fund, then you should absolutely cut back on not just coffee, but pretty much every non-essential you spend money on until you have three months of living expenses tucked away in the bank. Without that safety net, you'll generally have no choice but to rack up costly, credit-damaging debt when unplanned bills land in your lap, and that's a good way to wreck your finances for years.

Of course, if you're consistently socking away a good 15% of your income or more for the future, and you also have a healthy emergency fund, then there's no need to change your coffee habit whatsoever. Heck, go ahead and spring for a second latte on those days when work seems to drag on indefinitely and you need that boost. But if you're without emergency savings, store-bought coffee is a fairly easy thing to cut back on. And if you're doing OK on emergency savings but are lacking on the retirement savings front, you should consider scaling back on the lattes as well.

That said, there are other expenses in your budget you can trim to keep those daily $5 coffees coming. You might, for example, cancel the cable plan you don't use often, stop paying for a gym when you can jog outdoors for free, or cook dinner at home every night rather than treat yourself to takeout twice a week. You can even consider making bigger changes, like finding a cheaper apartment to rent, if you enjoy life's little luxuries and can't bear to give them up. But don't make the mistake of neglecting your near-term or long-term savings. It may not be your coffee's fault per se, but if something doesn't give, there's a good chance you'll wind up struggling financially in one way or another at some point in time.