Many investors dream of building a portfolio that delivers significant passive income. If you invest your funds carefully, that dream is always within reach, and now's a very tempting time to make progress toward it. 

With a bear market in full swing, there are undoubtedly more buying opportunities as stocks look seemingly cheaper than they were a few months ago. But not every dividend stock will make for a solid passive cash-flow source a decade from now. Let's delve into a few arguments in favor of loading up on passive income stocks right now and a few in favor of holding off. That way you'll get a feel for where to allocate your money that'll deliver optimum, risk-adjusted returns. 

Yields are high and might rise further

One big argument in favor of buying passive income stocks right now is investors will be able to lock in more income for each invested dollar than before. As share prices fall, dividend yields rise. It's simple math: Unless the per-share value of the dividend is cut, falling stock prices mean a higher payout in proportion to the share price. 

Take the cannabis real estate investment trust (REIT) AFC Gamma (AFCG 0.62%) for example. It writes real estate-backed loans to marijuana businesses that need to raise cash but can't do so through traditional financial institutions due to legal constraints. The REIT then collects interest payments on these long-term loans, finding its margin from the difference between the rate it can borrow at and the rate at which it lends to its clients. 

At the moment, its forward dividend yield is above 13.3%, which is 28% higher compared to the start of 2022. Of course, its share price is down by around 28% in that period, and it could fall further depending on the market's sentiment about the cannabis industry, REITs, and the company's ability to raise and deploy capital given rising costs of borrowing. And that's not even considering the possible impact of the business's critical internal processes, like picking which cannabis operators to write loans to.

If you purchase a bunch of AFC Gamma shares today, you'll be getting an exquisitely high yield even if its share price falls further. It'd only take an investment of just over $7,500 to generate $1,000 in annual passive income, which is hard to beat. Plus, if you're concerned about missing out on even higher yields as the bear market continues, you could always stagger your purchases over a few months. So on the surface, it looks like investors should be getting shares of high-yield passive income stocks while the getting's good.

Economic and monetary headwinds are significant risks

Despite the attractiveness of high yields right now, getting higher yields compared to yesterday isn't very helpful if your companies end up slashing their dividends for whatever reason. Rather than prioritizing investments based on their dividend yield alone, investors should instead consider whether a company's dividend payments are going to be sustainable over time. And in today's economy, there are a plethora of threats to dividends that should give investors pause. 

First, interest rates are rising, so businesses will incur higher interest expenses when they borrow money. That could be especially burdensome to REITs, thereby making an entire category of traditional passive income stocks far more hazardous than under normal conditions regardless of their high dividend yields. Second, there are swirling winds of a global recession brewing. If such a recession ends up happening and it dents demand for products, it could jeopardize dividends from consumer discretionary and industrial businesses too. Third, there's a lot of bizarre activity in the commodities markets, which also confers additional risk to the payouts of many manufacturing and manufacturing-adjacent companies. 

Thankfully not all dividend payers are exposed to higher-than-normal levels of risk. In particular, profitable and growing pharmaceutical businesses like AbbVie (ABBV 0.23%) are likely to be resilient in the face of the current set of headwinds. As AbbVie generated nearly $22 billion in free cash flow (FCF) in 2021 while paying only around $9.2 billion in dividends, it's clear that there's plenty of room to keep distributing capital to shareholders even if it continues to hike its payout on a yearly basis. It doesn't need to borrow money to fund its development of new medicines, nor will demand for its medicines contract if the economy goes into a recession, as medicine is a high priority for people to buy even if they're facing financial difficulty; nor is it exposed to significant risk from commodities. 

The catch is that AbbVie's forward dividend yield is "only" around 4%. That means you'd need to invest a cool $25,000 to get the same $1,000 of annual dividend income as you'd get with a much smaller investment in AFC Gamma. Nonetheless, the chances of the company cutting its dividend in the near term are low, and that makes it a safer option

Therefore, investors should seriously consider buying shares of high-quality passive income stocks like AbbVie whose distributions are secure in light of the economic headwinds that they can reasonably expect to encounter. When it comes to higher-yielding but higher-risk stocks like AFC Gamma, it could still be worth picking up a few shares too. Just don't bet the farm on them right now.