When you're saving for a truly long-term goal, investing in stocks makes a lot of sense. But what you choose to do with the money that you don't put into stocks can be just as important as picking winning stocks. Unfortunately, many investors aren't being very rational about what they do with their spare cash, and their bad decisions may end up costing them more than they think.

The bond buying spree
If you look at performance over the past few years, you'd conclude that the best place to invest spare cash is the bond market. The combination of weak economic activity and loose fiscal and monetary policy has pushed interest rates lower, making bonds that investors already owned more valuable.

The problem, though, is that those falling rates and higher returns have pushed the cost of bonds so high that even their best-case returns aren't all that attractive. Five-year Treasury bonds, for instance, yield less than 1.25%. And even if you go into the corporate arena, rates aren't that much better. Wal-Mart (NYSE: WMT) bonds yield 1.5% currently, while Home Depot (NYSE: HD) debt pays around 2.2%. Only by taking on a lot of default risk can you find outstanding yields; MGM Resorts (NYSE: MGM) issued six-year notes paying more than 10% earlier this week, but with the debt expected to carry a junk bond rating of CCC+, investors can hardly call that a risk-free return.

The typical choice investors make when yields are this low is to buy longer-term bonds. But that raises two more concerns. First, even long-bond yields aren't all that high. Travelers (NYSE: TRV) issued 10-year bonds yesterday that paid less than 4%. Johnson & Johnson's (NYSE: JNJ) 10-years paid less than 3%, and even its 30-year debt carries a yield of just 4.5%. Moreover, those bonds are more sensitive to interest rate increases, which many consider imminent.

Where to find better yields
Even with bond yields so low, many investors see little choice but to take what they can get. After all, a typical asset allocation strategy requires you to keep a substantial portion of your money in bonds, and the older and more conservative you are with your investments, the more you need to keep in bonds.

What many investors forget is that some investments act pretty much exactly like bonds, even if they have another name. One example is the simple bank certificate of deposit.

If you think about it, you'll realize that the lowly bank CD is, at least for small investors, just as safe as a Treasury bond. With FDIC insurance on CDs up to $250,000, you don't have to worry about the creditworthiness of the bank you choose. If the worst happens and your bank fails, then the government will step in and reimburse your lost money.

Meanwhile, the CD has almost identical traits to a bond. You can choose to take interest payments just like a bond pays. When the CD matures, you get your money back. In fact, in some ways, the CD is actually better than a bond, because if you need your money early, you can get it just by paying an early withdrawal penalty. With regular bonds, you'd have to sell them into the secondary bond market, at whatever price a buyer was willing to pay -- which could be much more of a hit than what the bank would charge.

When it comes to returns, though, banks beat Treasuries by a substantial margin. Discover Financial's (NYSE: DFS) Discover Bank pays a 2.6% rate on a five-year CD, while AIG's (NYSE: AIG) AIG Bank pays 2.2%. It's astounding that you can get an extra percentage point or more of yield over comparable Treasuries. In some cases, it's even more than what high-quality corporate bonds pay -- without any default risk at all.

A bond by any other name
So if you're wondering where the best place is to put your cash, but are worried about a potential bubble in the bond market, take a closer look at bank CDs. They aren't glamorous, but with their yields higher than riskier alternatives, you could certainly do worse.

Many investors are turning to dividend-paying stocks for income. Find the best ones by clicking here to get the Motley Fool's free report, 13 High-Yielding Stocks to Buy Today.