When President Barack Obama announced plans to help struggling homeowners in 2009, he made it clear whom the government was not out to help: "I want to be very clear about what this plan will not do: It will not rescue the unscrupulous or irresponsible by throwing good taxpayer money after bad loans. It will not help speculators who took risky bets on a rising market and bought homes not to live in but to sell."

Almost three years later, the plan has been almost universally panned as a failure, barely getting off the ground. There are several reasons for this, but one may be that the number of homeowners who weren't being unscrupulous or irresponsible by purchasing multiple homes is smaller than you think.

The Federal Reserve Bank of New York recently asked a simple question: What percentage of mortgages lent during the bubble went to people who owned more than one home?

Its answer: nationwide, over a third. In Arizona, California, Florida, and Nevada, it was nearly half. Astoundingly, about 1 in 10 mortgages lent in those states during 2006 and 2007 went to those who owned more than four homes. All figures are about double the rate seen before the bubble.

The numbers might seem reckless in hindsight, but try to remember how rational owning a second home seemed at the time. In April 2004, an article in Florida's Ocala Star-Banner explained:

Second homes aren't only for rich people. They have become mainstream whether they're used for vacations or for rental income ... You don't have to have a pile of cash at hand to buy a vacation house. You can use the equity in your primary residence to help for a second home (or third).

In 2006, an executive of mortgage lender GMAC (later bailed out by taxpayers) described the thought process when deciding to own a second home: "The number one factor that any homeowner should consider is the memories that a vacation home can provide." Whether you could afford one didn't seem particularly relevant.

In 2007, the Los Angeles Times laid out the rationale for buying a second home as an investment:

Real estate is a leveraged investment. One can own a second home with an equity investment (down payment) of no more than 20%. In fact, there are many programs that let people buy with a lot less. Most investors can't do this with stock. They need to pay the entire price of the stock.

It continued:

An investor can live in real estate. Stock certificates are pretty, with great colors, cool writing and embossed letters. Unfortunately, the stock investor can't go to sleep in them or stand on them to watch the sunset over the lake, or hold a party for your friends and family in them. They just -- hopefully -- make money. Real estate provides many kinds of satisfaction that money can't.

A second home has long-term wealth-building powers. In a nutshell, if the owner thinks a house is good enough to live in and enjoy, someone else will too, and they'll pay for the privilege to rent it.

And then came the money line: "Real estate markets don't crash the way stocks do when the bull runs out of steam. In short, it's a less risky investment."

Think about that mentality, and it's safe to say that most of those leveraging up with multiple homes didn't even realize they were speculating. They thought they were doing the responsible thing, investing their money in a safe asset, building wealth for the future.

But regardless of intention, the dynamics of buying a home as an investment are far different than those of purchasing a primary residence. As the Fed explains: "Because investors don't plan to own properties for long, they care much more about reducing their down-payments than reducing their interest rates." In fact, the more homes you owned, the higher the likelihood you were using a subprime mortgage. That alone caused overbidding and pushed up prices to crazy levels. If you want to blame one group for the housing bubble, the crowd of Americans who owned more than one home is a good place to start.

None of this is breaking news -- of course speculators helped fuel the bubble. But it helps paint a better picture of who is losing their homes today.

About a third of all mortgages "seriously delinquent" early in the recession belonged to those who owned more than one home. More recently, it was about one-quarter. Most of those ended up in foreclosure. In other words, between one-quarter and one-third of those who have succumbed to foreclosure during the housing bust haven't necessarily lost their home, but their second, third, or fourth homes -- their investments, in other words.

That should alter the conversation about the current foreclosure crisis. No one likes to see anyone lose their home, and many homeowners have truly suffered a raw deal. But so much of the angst over the foreclosure crisis has overlooked an important truth: A large number of those who have lost their homes had no business owning them in the first place.

What do you think? Share your thoughts in the comments section below.