Image source: Flickr user Elvis Fool.

"Sometimes what goes into your shopping bag should find its way into your portfolio," I argued last month, discussing the many reasons to love investing in publicly traded retailers. Malls are a fun place to bone up on field research. It's simple enough to see if the retailer is threatened by online competition. Retailers typically operating on different fiscal calendars, making it easier to avoid a deluge of earnings reports. Retailer financials are also reasonably easy to decipher.

However, there are also downsides to snapping up the stocks of mall merchants and other retailers. Let's go over a few of the reasons it could be dangerous to shop for stocks where you shop for everything else.

1. Buy and hold is a challenge

There's no shortage of once publicly traded chains that have faltered. There's a good chance you remember where Borders, Blockbuster, and Circuit City used to set up camp near your town. However, there aren't too many retailers that you can just buy and forget. 

Circuit City turned down a buyout offer at $17 a share in 2005. Three years later it was liquidating itself in bankruptcy. Shoppers are fickle, and trendy chains fall out of favor. Buying into a retailer entails staying on top of the investment with every passing quarter. 

2. There is no all-weather chain

It may seem easy to sidestep the hot concepts, but even the blue chips of retail are vulnerable. Wal-Mart (WMT 1.02%) is the world's largest retailer. It's often seen as an all-weather outfit. When the economy's rough, folks turn to Wal-Mart as a discounter to get more bang for the buck. When the economy's humming along, consumers are shopping more in general.

It's against this backdrop that Wal-Mart suffered six consecutive quarters of negative comps, and this didn't happen during a dark recessionary stretch. This happened through most of 2013 and into late 2014.

3. Amazon is a serial disruptor

The leading online retailer is a growing threat across most retail categories. Amazon.com (AMZN 1.49%) began selling books, and we saw how well that played out for Borders. It moved on to CDs and DVDs, and while digital delivery was the biggest culprit in the slow death of most media retailers, Amazon's arrival didn't help.

Amazon disrupts by selling a growing number of categories, and it can do so on razor-thin margins. It also only helps Amazon -- and hurts traditional chains -- that the leading e-tailer doesn't have the overhead related to manning physical stores. 

Amazon keeps gaining market share at the expense of offline and even many online specialists. Its net sales soared 20% last year. If you're buying into a retailer in a niche that Amazon is or will be participating in, you'll have to be careful you don't get disrupted. 

4. Everyone is susceptible to declining mall trends

If you're finding that malls aren't as crowded as they used to be, you may be on to something. Retail real estate researcher Green Street Advisors reported earlier this year that the occupancy rate at malls was declining for the first time since the recession. Rental rates also appear to be slowing down.

It's not just retail stores feeling the pinch. Some restaurant chains have noticed that mall-based locations aren't holding up as well as other eateries. 

The news, however, may not be all bad. Some mall operators say folks are spending more despite flat traffic. The internet has helped enlighten shoppers, and that could be leading to more focus when they arrive with an intent to buy. It will be hard for mall traffic to bounce back, so make sure the retailers that you do buy have a viable online platform.