Can mortgage REITs convince in the third quarter? The market has sent mixed signals so far. While American Capital Agency (NASDAQ:AGNC), for instance, reported a third-quarter sequential book value decline of 2.7%, smaller players in the sector like Hatteras Financial (UNKNOWN:HTS.DL) convince with a solid performance.

Putting a smaller mortgage REIT like Hatteras Financial onto the radar can make a lot of sense: Though the company only has a market capitalization of $1.8 billion, less than a quarter the size of American Capital Agency Corporation, for instance, the mortgage REIT achieved respectable performance results, not only in the third quarter, but over the course of last year.

Hatteras Financial presented investors with solid book value growth as other mortgage REITs struggled in the sector in the second half of last year, and its core earnings per share are covering its dividend.

In addition, the dividends paid to investors now look sustainable, which makes an investment in this mortgage REIT an interesting alternative to larger industry rivals such as AGNC or Annaly Capital Management (NYSE:NLY).

Probably the biggest difference between Hatteras Financial and its much larger mortgage REIT cousin Annaly relates to Hatteras Financial's much shorter length of its performance record.

When it comes to reliable dividend payments, one of the most distinguishing characteristics is the dividend record: And Annaly is clearly standing out from the crowd here.

The biggest mortgage REIT pays investors regular quarterly dividends since the late 1990s and throughout business cycles, whereas Hatteras Financial has a significantly shorter remuneration record as it forks over cash to shareholders only since it went public in 2008.

However, just because Hatteras Financial is a younger REIT with a less mortgageable dividend history than Annaly, and much smaller than both Annaly and AGNC, doesn't mean investors should pass the company up as an income investment.

Book value
This is where the music is being played in the mortgage REIT sector. Investors can quickly judge the success of a mortgage REIT by looking at the latest development of its underlying net asset value.

Positive book value growth speaks volumes about the mortgage REITs ability to profitably invest in residential mortgage-backed securities.

And Hatteras Financial has had some success in demonstrating its ability to deliver book value growth for investors: Hatteras Financial's book value has now grown four quarters in a row and its third-quarter net asset value was reported 5% higher than last year's.

Core earnings covering dividend payments
As usual in the residential mortgage-backed securities investing business, earnings can be extremely volatile due to a variety of factors influencing interest rates, demand for mortgage securities, the amount of leverage utilized etc.

Despite noteworthy volatility in Hatteras Financial's core earnings, its core EPS has fluctuated between $0.51 and $0.64 per share over the last year, the mortgage REIT covered its shareholder distributions in each of the last four quarters; and sometimes, like in Q1 and Q2, even with a significant margin of safety.

Hatteras Financial's high average core EPS of $0.57 has also allowed the mortgage REIT to keep its dividend payout steady at $0.50 per share -- also for four consecutive quarters.

Dividend yields
Hatteras Financial compares well against other mortgage REITs in terms of dividend yield. Shares of the company now yield 10.52% (data courtesy of S&P Capital IQ), which is on the same competitive level than the forward dividend yields of Annaly (10.53%) and American Capital Agency Corporation (11.66%).

The Foolish Takeaway
Hatteras Financial seems to be a proper alternative to its larger mortgage REIT rivals. Over the course of last year, Hatteras Financial has convinced with slow, but steady book value growth and a dividend that has been covered by its core earnings, despite the fact that earnings have been volatile.

In addition, Hatteras Financial's dividends have been held steady for four quarter in a row, which suggests that the dividend adjustment process has come to an end and investors are seeing sustainable dividend levels going forward.