For years, the retail world has seen a huge divide between companies catering to the rich and those with greater mass appeal. Now, though, that divide appears to be closing as those who had previously ignored economic uncertainties finally start to rein in their spending.

A recent survey from the American Affluence Research Center looked at how wealthy consumers plan to spend their money, with an emphasis on holiday spending. The results are bad news for investors who had hoped that high-end retail chains would be able to avoid any potential downturn from a slowing global economy or political concerns here at home.

Figuring out the rich
According to the survey, which targets a cross-section of the 11.4 million households that make up the wealthiest 10% of U.S. households based on net worth, the rich will spend about $50 less on holiday gifts this year, with the average estimated at $2,102 per household. Combine that with the 7% of rich households that aren't planning to buy holiday gifts at all, and you get a 5% drop in total spending for the season to $22.3 billion.

The demographics also show a shift toward frugality among affluent households. Nearly one in four rich families expects to spend less on gifts this year, compared to just 6% expecting to spend more.

Yet perhaps the most interesting part of is where the rich shop most often. Although the impression that many people have of rich people is that they spend primarily at luxury-oriented chains, the AARC survey throws cold water on that perception. Rather, Target (NYSE:TGT), Costco (NASDAQ:COST), and Home Depot (NYSE:HD) were the three retail chains drawing the most visits from the affluent. Online, shoe retailer Zappos, which (NASDAQ:AMZN) bought three years ago, was the only shopping website to draw more than 2% of respondents.

Are luxury chains in trouble?
AARC did the survey a couple of months ago, so it only had initial impressions of what would happen during the holiday season. But based on yesterday's post-Black Friday stock moves, it appears that early fears of holiday weakness may well have come to pass.

Among the biggest decliners in the S&P 500 yesterday were Coach (NYSE:TPR) and Nordstrom (NYSE:JWN), two retailers well known for their high-end shoppers. Both stocks had gotten a lift during the second half of last week as investors got excited about the potential for a better-than-expected beginning to the shopping season. Yet with macroeconomic concerns dominating November, especially after Hurricane Sandy hit the northeastern U.S. in late October, some retail analysts pointed to the fact that despite better foot traffic, revenue declined somewhat year over year on Black Friday itself. Moreover, with Cyber Monday diverting traffic onto the Internet, high-end retailers that rely on in-person connections to bolster sales face an ever-increasing challenge of online competition that can cater to the rich as well as a wider audience.

Don't count the rich out
Before you conclude that affluent households will lead us into a new recession, it's important to realize that uncertainty for the rich has never been higher. Staring into the abyss of a potentially huge tax increase, the affluent clearly are more money-conscious than usual, especially with the election's results pointing to a higher probability of a compromise to boost tax rates despite Republican control of the House.

Assuming an actual deal gets done, though, the rich could well find their worst fears won't come to pass. That in turn could lead to some spending in order to fulfill pent-up demand. With a compromise unlikely to happen until late in the holiday season, it may come too late for luxury retailers to salvage what remains of their primary profit generator, thereby pushing stock prices down and creating what could be a buying opportunity based on a temporary blip.

This article represents the opinion of the writer, who may disagree with the “official” recommendation position of a Motley Fool premium advisory service. We’re motley! Questioning an investing thesis -- even one of our own -- helps us all think critically about investing and make decisions that help us become smarter, happier, and richer.