It happens to everyone.

You get an ugly sweater from a well-intentioned relative, or a loved one gives you not-quite the piece of electronics gear you were hoping for as a present during the holiday season. You, of course, offer the proper thank you's, but as soon as you get a chance, it's back to the store to return the offending item.

Returns are a major part of the holiday season, and they wreak havoc with stores. In addition to the problems caused by having to restock items, or being left with merchandise post-December that won't sell, fraud is also a significant problem.

In some cases, retailers, including the big boys Wal-Mart (NYSE:WMT)Amazon  (NASDAQ:AMZN), and Target (NYSE:TGT), can see what looked like a positive season turn out to be not quite as good as initial sales made it look. Yes, all three of those chains were among this year's Black Friday winners, but those numbers only reflect what got sold -- not what comes back as a return.

Returns are a major part of retail
While the post-holiday season is the biggest one for returns, people buying or receiving merchandise as a gift that they then bring back is a problem all year long. "Americans returned $284B worth of merchandise last year," wrote Carly Llewellyn, marketing director for Optoro, a reverse logistics provider in an email to the Fool. "We expect that number to rise for 2015."

About 20% of those returns -- roughly $60 billion -- happen during the holiday season, according to Optoro's figures. Llewellyn wrote:

The number is going up as e-commerce sales rise and retailers are forced to offer more liberal return policies. Returns and excess inventory cost retailers $500B in the U.S., and over $1T worldwide. For many big retailers, they have billions of dollars of returned inventory a year.

Most retailers do not provide specific information on returns, but Best Buy (NYSE:BBY) estimates that returns, replacements, and damaged goods represent about 10% of revenue, costing the electronics retailer $400 million a year, reported a Dec. 26, 2014 Wall Street Journal article.

If return rates are an issue for Best Buy, which has physical stores, they may be an even bigger problem for Amazon and other online retailers because it's tougher to get a sense of sizing online for items like clothes and shoes. The return rate for items purchased online is about three times as high as it is for items purchased in a store, the Journal reported last year.

Amazon doesn't break out returns in its quarterly earnings reports. It's also possible that numbers specifically in clothing will more heavily skew toward exchanges at Target and Wal-Mart this year because both have heavily marketed the idea of in-store returns. 

 Wal Mart Bf

Wal-Mart was busy on Black Friday, but some of these sales will result in returns. Source: Wal-Mart.

Fraud is a problem
While legitimate returns will cause headaches for retailers, fraudulent ones can cut more deeply into their profits.

Return fraud happens in a number of ways, but the biggest problem, according to National Retail Federation, which was reported by 91.9% of retailers surveyed, is the return of stolen merchandise. In addition, 72.6% of stores polled say that wardrobing -- the return of used, non-defective merchandise -- is a problem.

About 3.5% of all holiday returns will be fraudulent this year, according to research from the NRF. That's up slightly from 3% in 2014. NRF estimates that return fraud will cost retailers $2.2 billion in 2015, up from approximately $1.9 billion last year.

According to NRF Vice President of Loss Prevention Bob Moraca:

Return fraud remains a critical issue for retailers with the impact spanning far and wide, in-store and online. While technology has played a significant role in deterring many in-person fraudulent transactions that would have otherwise gone unseen, there is little that can be done to prevent a determined criminal who will find a loophole one way or another. When it comes to retail fraud, retailers can build taller walls, but criminals continue to find taller ladders.

This is not an easy problem to solve
Return policies have actually gotten more liberal in recent years due to increased competition. Wal-Mart and Target specifically push the idea of buying items online, and returning or exchanging them in stores -- a move that eases the cost and volume of return logistics for the retailers. As a means of earning market share and customer loyalty, Amazon has always had an extremely liberal return policy.

For these major retailers, returns are simply the cost of doing business -- particularly during the holidays when shoppers are picking up gifts for others. These big hits are built into the guidance each company provides for the calendar year's final quarter, so investors shouldn't be alarmed, but they also shouldn't make the mistake of thinking every dollar the retailers collect during their holiday sales will stay in the cash register.  

Daniel Kline has no position in any stocks mentioned. He rarely, if ever, returns anything. The Motley Fool owns shares of and recommends Amazon.com. Try any of our Foolish newsletter services free for 30 days. We Fools may not all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors. The Motley Fool has a disclosure policy.