With investors searching desperately for ways to increase their income, dividend stocks have become more popular than ever. But smart investors always look at the other side of any trade to see whether there's a potential downside they're missing -- and in many cases, heavy-hitting institutional investors are making some sizable bets against high-yielding dividend stocks.

Learning from the pros
As fellow Fool Morgan Housel pointed out earlier this week, you can learn a lot by looking at what professional institutional investors do. Even though small individual investors have some advantages that the big players can't take advantage of, institutions do have a wealth of resources, knowledge, and experience on their side.

So with that in mind, I used data from the tracking firm AlphaClone to look into what strategies nearly 300 hedge funds and other institutional investors were using to invest in some of the highest-yielding dividend stocks in their respective sectors. Here's what I found:

Stock

Dividend Yield

% of Funds Holding Stock

Change in Market Value Held by Funds vs. Year-Ago Quarter

Change in Share Holdings Since December 2008

Annaly Capital (NYSE: NLY)

15.2%

21%

(20%)

(41%)

Nucor

3.7%

19%

(20%)

0%

Otter Tail (Nasdaq: OTTR)

6.0%

7%

(40%)

(12%)

PDL Biopharma (Nasdaq: PDLI)

19.2%*

10%

(14%)

(5%)

Hudson City Bancorp (Nasdaq: HCBK)

4.9%

17%

(8%)

(10%)

Capstead Mortgage (NYSE: CMO)

9.4%

7%

(12%)

(35%)

Sources: AlphaClone, Yahoo! Finance. *Yield based on two most recent $0.50 special dividends.

In light of how much interest average investors have paid to most of these stocks, I found these results pretty shocking. Not only have most institutional investors largely ignored these strong dividend payers, but many of those that have shown past interest have started to head for the exits.

Don't panic
Before you become convinced that these stocks are all doomed to fail after being chased by dumb money, though, you need to look more carefully for a reasonable explanation. With Annaly, for instance, share holdings among these institutions had more than doubled between 2006 and 2008 as conditions became more favorable for mortgage real estate investment trusts. Capstead enjoyed an even greater amount of interest. So taking some profits doesn't necessarily mean that these funds are giving up entirely on the sector. The same big spreads across the yield curve continue to support the bull case for these REITs.

In several other cases, institutions timed their exits and entrances from these stocks fairly well to coincide with the market meltdown in 2008 and early 2009. Otter Tail and PDL Biopharma saw drawdowns in holdings in 2008, but then institutions started to reload in 2009 and took advantage of the stock market's rally. Although their share prices haven't exactly skyrocketed, their dividend payments have provided a significant kick for shareholders. Certainly, PDL Biopharma's huge dividend makes it an outlier among biotech stocks, many of which have to wait for years burning cash before getting a single drug through their pipelines to produce any substantial revenue -- let alone profits. Like other utilities, Otter Tail has faced many challenges as energy prices have gyrated wildly in recent years.

Finally, in the case of Nucor and Hudson City, institutional investors have kept their exposure relatively flat over the past two years, and they likely haven't seen the capital gains they wanted to get from these stocks. Even if they're exiting their positions, they're doing so in a measured way that's more suggestive of raising liquidity for investor distributions or reinvestment into new ideas rather than giving up on the space.

Most encouragingly, there are plenty of dividend stocks that have drawn major increases in institutional interest lately. North American Tanker (NYSE: NAT) has seen share holdings among funds rise fivefold since late 2008, while institutions own two-thirds more shares of Frontline (NYSE: FRO) than they did a year and a half ago.

Stay strong
Few things are scarier than when the general public starts jumping in on a trade you were early in making. But even though mass hysteria may bring on unsustainably high prices for dividend stocks, there's at least some evidence that the smart money isn't getting out entirely just yet.

Stocks are scary. Find out from Alex Dumortier why you should prepare for a 60% stock market crash.

True to its name, The Motley Fool is made up of a motley assortment of writers and analysts, each with a unique perspective; sometimes we agree, sometimes we disagree, but we all believe in the power of learning from each other through our Foolish community.

Fool contributor Dan Caplinger has a love-hate relationship with big financial institutions. He doesn't own shares of the companies mentioned in this article. Nucor is a Motley Fool Stock Advisor choice. Otter Tail is a Motley Fool Hidden Gems selection. The Fool owns shares of Annaly Capital Management. Try any of our Foolish newsletters today, free for 30 days. The Fool's disclosure policy keeps you out of the dumps.