For investors looking to live off the income they generate from their portfolios, dividend stocks with high yields tend to be extremely attractive. But investors need to tread with caution because a big yield alone doesn't make a stock a good investment. In fact, that high yield can even be a sign that there's potential trouble ahead for the dividend.

Here's why ultra-high-yielding Realty Income (O -0.17%) is a buy but AGNC Investment (AGNC 0.97%), with its even higher yield, is a stock to avoid.

Realty Income is big, high-yield, and boring

Realty Income has trademarked the nickname "The Monthly Dividend Company." While the motto references how frequently the dividend is paid, it's also a statement about how important the dividend is to management and the board of directors. Notably, the dividend of this real estate investment trust (REIT) has increased annually for 29 consecutive years. That doesn't happen by accident. It requires a strong business and an ingrained management ethos that transcends any individual CEO or employee.

O Chart

O data by YCharts

To be fair, the dividend grew fairly slowly over that span, at roughly 4.3% per year. But that's more than enough to keep up with the historical rate of inflation growth. Although inflation is pretty high right now, the buying power of Realty Income's dividend has grown over time. What's most interesting, however, is that the stock's dividend yield of around 5.7% is near the highest levels of the past decade. So the stock looks historically cheap. That yield is also well above the roughly 1.6% you would collect from an S&P 500 index fund or the 4.9% yield of the average REIT, using the Vanguard Real Estate Index ETF (VNQ 0.05%) as a proxy.

And Realty Income offers this high yield backed by a very boring business. It's the largest net-lease REIT, with a market cap of $39 billion and a portfolio of more than 13,000 properties. (A net lease requires the tenant to pay for most property-level operating costs.) Around three-quarters of the portfolio is in retail properties, which are fairly generic and easy to buy, sell, and release. The rest of the portfolio is in industrial assets and an "other" category that includes casinos and vineyards. Add in a healthy dose of European properties, and you have a fairly well-diversified and low-risk portfolio. The company's balance sheet is also investment-grade-rated.

Slow and steady is the name of the game, though the past year or so has been rough on the stock because of rising interest rates. Higher rates make alternate investment options like CDs more attractive to income investors. Rising interest rates also create dislocations in the real estate sector, which has yet to find a settling point. That's why Realty Income's stock is down and the yield has risen to what appear to be historically attractive levels. Given the history here and the investment approach, it seems highly likely that Realty Income muddles through just fine.

AGNC is a poor fit for conservative income investors

AGNC's 16.5% yield is likely to turn out to be too good to be true. For starters, it's a mortgage REIT, and mortgage REITs represent a complicated subsector of the real estate sector. It's fairly easy to understand buying a property and renting it out, like what Realty Income does. AGNC's business is to buy portfolios of mortgages using leverage, often backed by the portfolio of mortgages it owns, in an effort to earn the spread between the REIT's interest costs and the interest it collects from the portfolio. Unless you're willing to really dig in to understand the mortgage REIT niche, you should probably avoid the sector and, thus, this stock.

AGNC Chart

AGNC data by YCharts

That said, the graph tells the real story. Over the past decade, AGNC's dividend has headed steadily lower, which has sent the stock heading steadily lower. The yield has remained high, but investors have ended up with less dividend income and a capital loss. That's the worst possible outcome for investors trying to live off the income they generate from their portfolios. If that graph doesn't dissuade you from buying this high-yield stock, there's a good chance nothing will.

Lower and reliable is better

Realty Income is unlikely to excite you very much, but that's by design. It wants to make sure the dividend gets paid and increased regularly. Add in an ultra-high 5.7% dividend yield, and conservative dividend investors should be looking closely at the stock today. AGNC's yield is more than twice as high, but you just can't rely on it given the terrible dividend history. It's a unique investment that isn't appropriate for most investors, and in any event, it's really meant for institutional investors focused on diversification goals, like insurance companies and pension funds. If you're trying to live off of your dividends, there's a very clear winner here.