One of the most out-of-favor sectors in the stock market this year is the mortgage space. We saw major lenders go bankrupt or close up shop as volume fell off a cliff. Mortgage real estate investment trusts (REITs), which invest in debt, not property, have also been sold off as the Federal Reserve has raised interest rates. The damage is widespread.

However, investors are starting to ask whether REITs (especially mortgage REITs) have been beaten up enough that they are now a buy.

My first thought is, not yet. Third-quarter earnings will be awful for the sector and there is a risk of dividend cuts. That said, here are three names for investors to keep an eye on and consider down the road a few quarters from now when the Fed is done hiking rates.

Picture of the Federal Reserve Building.

Image source: Getty Images.

1. AGNC Investment

AGNC Investment (AGNC -1.18%) is a mortgage REIT that focuses primarily on investing in mortgage-backed securities (MBS) that are guaranteed by the U.S. government. These sorts of mortgage-backed securities contain no real credit risk, but they do have a lot of interest rate risk. AGNC Investment is probably the safest stock in the mortgage REIT space given that a potential recession would impact it less than most other mortgage REITs. 

Mortgage REITs are sensitive to interest rates, but they are also sensitive to the performance difference between mortgage-backed securities and Treasuries. Mortgage REITs generally hedge their interest rate risk using products linked to Treasuries or other short-term interest rates. The problem occurs when mortgage-backed securities underperform their hedges. In trader parlance, this means that "spreads widen." We saw a lot of widening over the past month as investors fret that the Fed might be forced to sell some of its mortgage-backed securities portfolio. 

The market clearly is worried about third-quarter earnings, as the stock trades with a dividend yield close to 20%. This indicates that investors are assigning a pretty hefty chance of a dividend cut. The stock may look cheap, but I would avoid it until the Fed is done with rate hikes. 

2. Annaly Capital Management

Annaly Capital Management (NLY 1.35%) is another major mortgage REIT with similar headaches as AGNC Investment. Annaly is somewhat different in that it has a more diversified portfolio than AGNC. While Annaly invests in agency MBS like AGNC, it also holds mortgages that are not guaranteed by the U.S. government.

These loans are commonly referred to as "Non-QM," which means they fall outside of the parameters that the government will ensure. These loans have nothing in common with subprime loans from the bad old days; they are commonly made to professional real estate investors who intend to use rental income to pay the mortgage. These loans have less interest rate risk than agency MBS, but they do contain credit risk. 

Annaly also holds mortgage servicing rights, which are one of the few assets that increase in value as interest rates rise. If interest rate hikes push the economy into a recession, defaults should increase, which will increase servicing costs and make the asset worth less. We are already seeing mortgage bankers raise capital by selling mortgage servicing rights, which is impacting prices. Like AGNC, Annaly has an exceptionally high dividend yield at over 20%, which is a warning sign for a dividend cut. Tread carefully. 

3. Rithm Capital Management

Rithm Capital Management (RITM 0.04%) is a mortgage REIT that also has a substantial mortgage origination business. Rithm holds a large portfolio of mortgage servicing rights, which helped offset declines in origination volume. Rithm is another mortgage REIT with a sizable dividend yield of 13%.

If the mortgage origination business turns around next year, the company could see gains on its own origination business as well as its holdings of mortgage-backed securities. That said, investors should wait until the Fed signals it is through with this interest rate hiking cycle before jumping in.