The housing market appears to be on an upswing. Year-over-year home prices, as well as sales of new and existing homes, have all improved meaningfully. But a huge threat -- one that could potentially derail a sustained housing recovery -- remains.

A mortgage finance mystery
By many measures, owning a home right now is more affordable than it has been in a long, long time. Home prices are down a third from their 2006 peak, and mortgage rates are enticingly low. The average rate on a 30-year fixed-rate mortgage is the lowest it's been in at least 50 years.

So then why are so few mortgage applications approved?

The short answer is the lack of mortgage credit availability for a huge chunk of the population. Indeed, Morgan Stanley analysts view this as one of the biggest constraints on a housing rebound, writing in a June Housing Market Insights report, "Mortgage-credit-dependent home buying -- predominantly non-distressed ownership home buying -- will remain under pressure for an extended period until mortgage financing challenges are addressed and there is a sustained increase in employment."

Why no mortgage credit?
Fannie and Freddie are partly to blame. The two government-sponsored enterprises have been requiring banks to repurchase a ton of loans made before 2006. Largely as a result of this so-called putback risk, banks have adopted much tighter credit and documentation standards for new loans. Even the Federal Reserve, in an April survey, suggested that putbacks were the primary reason underwriting standards have been so remarkably stringent.

Putbacks have long drawn the ire of mortgage lenders, including Bank of America (BAC 1.53%), which suggested that Fannie and Freddie's policies have caused the bank to deny many otherwise credible loan applicants. Bank of America isn't alone in its charges against the government-sponsored mortgage giants.

Other banks and mortgage originators have voiced similar concerns, arguing that they've had to deny a large number of loans due to concerns that Fannie and Freddie may later force them to repurchase loans that failed to meet stringent underwriting standards.

According to data from Inside Mortgage Finance, Fannie and Freddie have requested banks to buy back a whopping $66 billion worth of mortgages originated between 2006 and 2008. Since a large portion of these mortgages has gone sour, banks will be faced with major losses once the loans are repurchased.

Have Fannie and Freddie gone too far?
Fannie and Freddie, along with the FHA, back a whopping 94% of all new home mortgage loans. Saying that the mortgage market is almost entirely dependent on these government-sponsored enterprises would not be an overstatement.

It is now well-documented that one of the reasons why home prices became so enormously inflated in the lead-up to the housing bust was a significant deterioration in mortgage underwriting standards. Because of badly misaligned incentives for mortgage lenders and the entities that securitized the mortgages, millions of people who lacked the financial means to afford a home were offered loans to own one.

But now, it appears we're having the opposite problem. Current underwriting standards, some argue, are far too conservative. Many mortgage bankers and industry commentators think Fannie and Freddie have gone too far.

Whereas lenders' main priority should be to lend to those borrowers with a reasonable likelihood of repaying the mortgage, it appears this is no longer the case. Now, some argue, lenders are far more concerned with avoiding a putback. This makes it very difficult for otherwise credible borrowers to obtain a loan. Many are subjected to overwhelming documentation requirements, some of which seem outright unnecessary.

On top of higher FICO score requirements, there are other factors further constraining would-be homebuyers. For instance, documentation standards and down payment requirements are also much more stringent these days.

According to Ellie Mae, a provider of software to the mortgage industry, the average borrower who was denied a conforming loan in August had a FICO score of 734 and was willing to put 19% down. At almost any other time in the past, borrowers with similar credit profiles could have easily obtained a loan.

What about QE3?
This highly conservative approach to lending may mitigate the positive impact of the Federal Reserve's latest round of quantitative easing, aimed at rejuvenating what Chairman Ben Bernanke called "a missing piston" in the nascent economic recovery. But it's unclear whether QE3 will have a meaningful impact on the housing market.

For one, it may actually end up helping the banks that originate mortgages the most, while not helping homeowners by much, and actually hurting certain types of companies who engage in purchasing mortgage-backed securities, such as mortgage REITs. While mortgage REITs have benefited tremendously over the past few years thanks in large part to the Federal Reserve's largesse, they now face a major threat from QE3.

Mortgage REITs, which sport some pretty phenomenal yields, have become increasingly popular in recent years. Some of the more popular ones, like Annaly Capital (NLY 0.61%), Chimera Investment (CIM 1.26%), and Anworth Mortgage Asset (ANH), all yield above 9%, with Annaly and Chimera yielding in the 12%-14% range.

And another, American Capital Agency (AGNC 0.22%), boasts a staggering 15.3% yield. But with QE3 pushing down longer-term rates and lowering the interest rate spread that determines these companies' profits, it appears that some of these payouts may be in serious jeopardy.