For more crisp and insightful business and economic news, subscribe to The Daily Upside newsletter. It's completely free and we guarantee you'll learn something new every day.

The United States hasn't built enough homes over the past 15 years.

As housing prices reach record highs nationwide – against quite a few forecasts at the start of the year – buying a home is now completely unaffordable in most parts of the country for the average American.

According to a recent report from property data provider ATTOM, home prices in 99 percent of the key counties it examined shot well past the purchasing power of the average income earner making just above $70,000 a year.

"Buyer demand continues to outmatch housing supply, creating upward pressure on home prices, despite the fact that home purchase costs are taking up an outsized share of household incomes," noted Realtor.com's chief economist Danielle Hale in a statement.

As a rule of thumb, financial planners typically recommend individuals not spend more than 30 percent of their gross monthly income on housing costs (this includes not just rent or mortgage payments, but also utilities, taxes and insurance). But according to data from lending marketplace LendingTree, a growing number of Americans are spending more than that – with Hawaii, California and New Jersey the biggest offenders, in that order. Other states predictably struggling include New York, Connecticut and Florida, but also Rhode Island, Oregon, Massachusetts and Vermont. These all rank in the top 10

This week, Morgan Stanley, which had forecast housing prices would tick down this year, switched its projections to an increase in housing prices of as much as 5 percent for 2023. And in some places, it is far more than that, with the greater Los Angeles area leading the way with prices ballooning nearly 24 percent in the 12 months ended in September. Other top cities with outsized price increases include San Diego, Richmond, Cincinnati, Boston, Rochester and Pittsburgh (the list is actually quite surprising, with a lot of cities you wouldn't expect).

Exacerbating this, of course, has been sky-high inflation and rising interest rates, making the cost of building or buying a home – or relocating – far more expensive than in the past. This will be incredibly frustrating for Americans who didn't take advantage of America's prolonged period of historically low interest rates following the 2008 global financial crisis, which deepened during the pandemic, as mortgage rates fell to a record low of 2.65 percent in 2021.

By contrast, the average rate on 30-year mortgages is now at nearly 8 percent, the highest level in more than 20 years. While the U.S. Federal Reserve has, for the moment, paused its aggressive campaign of rate hikes, borrowing costs aren't expected to come down anytime soon with the central bank's "higher for longer" mantra firmly in place and inflation levels not expected to cool to meet its target until 2026.

The origins of the current housing supply crisis are well known, as construction in the U.S. dropped like a stone during the global financial crisis 15 years ago, which led to the supply of new homes coming onto the market falling lower than any time since the 1960s.

Specifically, what does that mean for U.S. housing supply now? According to the National Association of Realtors, the country is experiencing an estimated housing shortage of between 5.5 million and 6.8 million units – with the gap between supply and demand widening every year. (See the association's housing shortage tracker here.)

The crisis has reached a fever pitch, as demand for housing is unlikely to decline anytime soon, with America's population on track to hit nearly 390 million by 2050, according to the U.S. Census Bureau. In other words, there has never been more Americans in need of housing.

According to data from the U.S. Department of Housing and Urban Development, nearly 600,000 people across the country are experiencing homelessness, or around 18 out of every 10,000 people. It also estimated around 28 percent of those homeless are living in families with children.

Homelessness by state, from U.S. Department of Housing and Urban Development.

Further pushing prices up are institutional investors, such as large banks and investment firms, which are hoovering up huge quantities of available housing inventory for profit. These investors have accounted for more than 13 percent of all residential real estate purchases in recent years, according to a National Association of Realtors study.

While home prices and rent likely will stabilize or even ease over the next year, according to Fannie Mae, this may stem more from an economic slowdown leading to reduced demand than a genuine decrease in need among Americans looking for housing. Overall, Fannie Mae says, "the supply problem will remain, and the pain it causes will likely continue to be felt mostly by low- and moderate-income families."

On the plus side, higher prices and scorching demand likely will lead to innovations in U.S. housing over the longer term – whether in the form of much-needed improvements to zoning, more flexible rules and conditions for home-building and investment in underdeveloped areas and opportunity zones.