The number of mortgage lenders willing to touch mortgages that aren't plain vanilla prime mortgages for under $417,000 continues to dwindle.

After the market closed last night, Capital One (NYSE:COF) announced it was shutting down its GreenPoint wholesale mortgage business, which focused on jumbo loans (above $417,000) and Alt-A loans, which are given to borrowers who have high credit scores, but who are often self-employed and lack sufficient income documentation. As a wholesale mortgage business, Greenpoint would originate and then sell the loans into the market.

In the last two weeks, any mortgage that cannot be sold to Fannie Mae (NYSE:FNM) or Freddie Mac (NYSE:FRE) -- known as conforming mortgages -- joined subprime mortgages as unpalatable to investors. Originators are quickly finding that they can buckle down on what they originate and charge higher rates for these loans, as Wells Fargo (NYSE:WFC) began doing a couple of weeks ago, or they can exit the business. What they can't do is operate the way they always have.

Capital One decided to cut its losses and move on, and I think it's a move that makes plenty of sense. The company has a very strong credit card business and is still working to get its acquisition of North Fork Bank under control. The company will take a $2.15-per-share charge to exit the business, but expects that it will still earn $5.00 a share in profits this year. That's a decline from last year's $7.62, but a clear indication that the rest of the business is still sound. It isn't worth damaging the franchise for a cyclical, noncore business that is bleeding money.

Capital One focusing on the core of its business makes sense. But it does make me wonder how many will be left standing in the nonconforming mortgage business, and what the rules of the mortgage game will be after the dust settles.

Fannie Mae is an Inside Value recommendation. Take a free trial today.

Nathan Parmelee thinks the general sell-off in financial services is creating some great opportunities and has made a couple of  purchases, but he has no ownership stake in any of the companies mentioned here. The Motley Fool has an ironclad disclosure policy.