Over the past year, times have been tough in the mortgage industry. Mortgage originators have suffered from declining origination volume, while mortgage real estate investment trusts (REITs) have seen mortgage-backed securities underperform Treasuries, which led to declining book value per share.

Most mortgage REITs cut their dividends last year, but two companies, AGNC Investment (AGNC 0.58%) and Rithm Capital (RITM 1.13%), have maintained their dividends over the past year. Which one is the better buy? 

A person holds a tablet while surrounded by digital figures and graphs.

Image source: Getty Images.

AGNC Investment is the pure-play agency mortgage REIT

AGNC Investment is the classic agency mortgage REIT. It invests almost exclusively in mortgage-backed securities, which are guaranteed by the U.S. government. This means that if the borrower doesn't make a mortgage payment, the government ensures that the investor will still get paid the principal and interest they are owed. These securities don't have credit risk, but they have a lot of interest rate risk. Unhedged mortgage-backed securities contributed to the failure of Silicon Valley Bank, so an absence of credit does not equate to no risk at all. 

AGNC has been battered by rising interest rates as its portfolio of mortgage-backed securities underperformed Treasuries. Book value per share fell from $16.76 per share to $10.76 over the course of 2022. AGNC's portfolio of mortgage-backed securities declined dramatically as rates rose; however, the gains on the hedges were insufficient to cover the losses on the portfolio. At this stage, the underperformance is the worst in five years. If mortgage-backed security spreads return to historic levels, AGNC Investment should see increases in book value per share. 

Rithm Capital has a diversified business model 

Rithm Capital is more of a hybrid REIT. It owns portfolios of mortgage-backed securities like AGNC, but it also originates mortgages and does mortgage servicing. The mortgage origination arm has struggled over the past year as interest rates have picked up and refinancing activity has disappeared. Mortgage servicing, on the other hand, performed well in this environment. Mortgage servicers administer the mortgage on behalf of the ultimate investor. The servicer sends out the monthly bills, collects payments, ensures property taxes and insurance are paid, and works with the borrower in the event of a default. The servicer is paid 0.25% of the mortgage balance outstanding per year. The right to perform this service has value and is capitalized on the balance sheet as an asset. Mortgage servicing rights increase in value as interest rates rise. 

Rithm Capital's strategy allowed the company to report an increase in book value per share in 2022, from $11.44 per share the year before to $12. Rithm Capital's diversified business stream and its reliance on fee-based income helped the company perform in one of the most challenging environments. 

The economic backdrop is becoming more challenging for Rithm Capital 

Rithm Capital might struggle more than AGNC in the future. While interest rates are still increasing, the biggest hikes are in the past, and mortgage delinquencies will start rising if the economy hits a recession. Rising delinquencies hurt the value of servicing in general, and raise the company's costs. If the Federal Reserve starts cutting rates, there will be a period where servicing values fall, but refinancing opportunities still remain elusive. 

AGNC's pure-play agency REIT status will be largely immune to a recession since it will get paid regardless of delinquency rates. Mortgage-backed securities underperformed Treasuries last year due partially to interest rate volatility, which is a function of uncertainty. Once the Fed gives the all-clear signal, interest rate volatility should dissipate, which will help both companies' mortgage-backed securities portfolios. AGNC's 14.1% dividend yield is superior to Rithm's 10.8% yield. Rithm does trade at a 20% discount to book value compared to AGNC, which trades right about at book value. This is because many of Rithm's assets are illiquid and hard to value so the market assigns a discount. 

Both AGNC Investment and Rithm Capital are great mortgage companies. The bottom line is that a lot more can go wrong with Rithm than with AGNC Investment.