The real estate market has performed pretty well over the past couple of years, with the average U.S. home price about 27% higher than the 2012 lows. So, this means we're out of the "danger zone", right?

Not so fast! There are still a few things that could very easily end the housing rally that we need to keep an eye on, and one of the biggest hazards is first-time homebuyers staying out of the market, which is already happening.

With the recent data indicating the market is slowing down, it is more important than ever to watch for signs of trouble.

First-timers are an important part of a healthy housing market
Historically, first-time home buyers have made up about 40% of the market and this has dropped recently. According to the National Association of Realtors, 38% of 2013 U.S. home buyers were first-timers, and the percentage declined throughout the year, dropping to as low as 27% in December, the lowest percentage since they began tracking the percentage in 2008.

There are several reasons first-timers might be struggling to afford homes. Higher prices, tight credit, poor employment prospects, and high debt could all be keeping first-timers on the sidelines.

Mortgages are either tough to get, or unaffordable
Up until a few years ago, the traditional route for first timers, FHA loans, were much more affordable.

Not only has the upfront mortgage insurance premium nearly doubled from 1% to 1.75% of the loan amount, but the annual mortgage insurance must be paid for the entire life of the loan. With an FHA loan, an interest rate of 4.5% actually works out to an effective APR of closer to 6%. For a comprehensive discussion of how much more expensive an FHA loan is, here is a thorough discussion

Conventional loans typically require a 20% down payment, which is getting tougher for first-timers to come up with as prices rise, and credit requirements are very high. The average FICO score for a conventional purchase loan is currently 755, and the average rejected credit score is 724, which is well in the realm of "excellent" credit.

Poor job market for younger people
Adding to the unaffordability of homes for first-timers is the difficulty of finding a good job. The unemployment rate for recent college grads is 8.5%, well above the national average, and another 16.8% are underemployed. The situation is even worse for younger adults who didn't graduate college, with some reports putting the unemployment rate for those under 25 with just a high school diploma at a staggering 30%.

So, even if young, potential first-time buyers have the credit and down payment, they won't have much luck obtaining a mortgage without the income to justify the loan.

As debts rise, less younger people will be able to buy
It's no secret that student loan debt has grown tremendously over the past couple of decades. The debt may or may not be worth it to obtain a degree and (theoretically) a higher paying job, but what's certain is more debts make it difficult to buy a house.

However, it's not just student loan debt we should be concerned about. The average "millennial" (ages 22-33) spends 12% of their take-home pay on student loans according to a recent survey, which is surprisingly not the biggest debt burden they have.

Credit card debt is often the biggest problem for younger adults, and eats up about 16% of the millennials' take home pay, on average.

Most mortgage companies shoot for total debt-to-income ratios below 36% for conventional loans, so if 28% of millennials' income goes to paying off student loans and credit cards, it doesn't leave much room to take on a mortgage.

To sum it up, home sales are still below what's considered a "healthy" level, so the market can't afford to see this huge group of buyers disappear. Any further drop in the percentage of first-time homebuyers in the market could be a warning sign for the housing market.

Should you buy these top dividend stocks instead of real estate?
The smartest investors know that dividend stocks simply crush their non-dividend paying counterparts over the long term. That's beyond dispute. They also know that a well-constructed dividend portfolio creates wealth steadily, while still allowing you to sleep like a baby. Knowing how valuable such a portfolio might be, our top analysts put together a report on a group of high-yielding stocks that should be in any income investor's portfolio. To see our free report on these stocks, just click here now.

Try any of our Foolish newsletter services free for 30 days. We Fools may not all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors. The Motley Fool has a disclosure policy.