A casual look at the earnings report from Casual Male (NASDAQ:CMRG) might have you thinking that living large was a shrinking proposition. With revenue and same-store sales rising in the quarter, profit was off 19% from last year. Yet that view would be like looking at yourself in a funhouse mirror: all distorted.

Last year, Casual Male sold off its loss prevention division for $5.2 million and realized $3 million of it up front. It recorded a gain on its income statement of $1.5 million, with the balance being paid over time. By my calculations, adjusting for that gain, Casual Male's net income more than doubled this year, which makes sense considering the success it is having catering to the plus-size male. Even so, the results missed analyst projections of $0.04 per share by a penny.

That's in contrast to other men's retailers like Jos. A. Bank (NASDAQ:JOSB), which reported some scary numbers, along with disappointing U.S. comps at Men's Wearhouse (NYSE:MW), although that retailer reported a more dapper picture overall.

What I find more troubling is the extreme dilution in Casual Male's stock, with nearly 20% more shares this quarter than last year. While that's down from the end of the year thanks to the 3.1 million shares that were bought back this quarter, it still represents an overly generous award of stock options, regardless of the turnaround that management has affected. It has repurchased 5.3 million shares over the past 12 months at a cost of more than $61 million, and there don't appear to be any signs that the trend is abating.

Perhaps the members of management feel the need to reward themselves for turning around a retailer on the brink of bankruptcy. For example, last year's first quarter was the first time the company had reported a profit in the first quarter since it was acquired back in 2002. Management has certainly done a good job in making the niche a very profitable one on a regular basis. But for current shareholders, and I count myself among them, the dilution of ownership interests is disconcerting.

Aside from that, the big man around men's retailers has been expanding its presence in numerous directions. The acquisition of Jared M., a men's retailer that caters to professional athletes and performers, looks like it will be a natural fit as it expands beyond the one-store model its founder sold.

Casual Male has also put its girth behind an online venture which it has dubbed LivingXL.com and which seeks to make the plus-sized world a brand extension. For me, that carries the most risk. While there is a certain blend that can be realized between offering a large male a well-tailored suit and showing him other amenities to fit his lifestyle, I get worried when apparel retailers venture off into non-clothing-related arenas. Since acquiring ski maker Rossignol, skate and surf apparel company Quiksilver (NYSE:ZQK) is learning that being a hard-goods manufacturer is more difficult than being able to slap a logo on some T-shirts.

Currently, the men's retailer is trading at levels comparable to both Men's Wearhouse and Jos. A. Bank. It also gets some nominal competition from mass merchants like Wal-Mart (NYSE:WMT) and department stores like J.C. Penney (NYSE:JCP) and Federated (NYSE:FD), but the plus-size male is frequently making Casual Male his final destination.

Wal-Mart is a recommendation of Motley Fool Inside Value. A 30-day risk-free trial gets you full access to all of the market-beating recommendations.

Fool contributor Rich Duprey owns shares of Casual Male and Wal-Mart but does not have a financial position in any of the other stocks mentioned in this article. You can see his holdings here. The Motley Fool has a disclosure policy.