I've plenty of reasons to love Whole Foods Markets (NASDAQ:WFMI). Here's the biggest: My oldest son has life-threatening food allergies. He depends on a bumper crop of organic food to survive.

You'd think that would make me an unabashed Whole Foods fan. And I am; I love the company. But the stock? Not so much. Here's why: Buying what feels good won't make you rich, especially if what you're buying -- (clears throat) Whole Foods -- has notable flaws that the market is only just beginning to realize.

Not quite unique
The investing thesis for Whole Foods depends on believing that no one can compete with it in terms of experience. That's been true to date. But my own shopping tells me that's changing. Take Wegman's, for example. The upscale grocer based in northern New York has been family-owned for years, and its stores are as dynamite an experience as any you'll find at Whole Foods.

Then there's the movement afoot at Safeway (NYSE:SWY) to remodel its stores to create a better ambience. Organic food is a big part of the new look. And no wonder: Organic food is a $15 billion market in the U.S. today and growing. Mainstream grocers naturally want their cut.

Consumers may want that, too. I know I do. Right now, our grocery shopping requires three different trips over each two-week period. First there's the Kroger (NYSE:KR) subsidiary King Soopers, where we shop for most stuff. Then there's Wild Oats (NASDAQ:OATS) for the organic products. (Whole Foods isn't close to us.) And finally, there's Costco (NASDAQ:COST) for all the big-box stuff.

Cans in the aisles
What that means is that, even if the mainstream grocers can't compete with Whole Foods now, a head-on collision is practically inevitable, especially with CEO John Mackey's designs on $12 billion in annual revenue by 2010.

And it's here that the Whole Foods story gets tricky. Its newest stores are expected to average a whopping 55,000 square feet. That's a lot of space to fill with private-label food and organic products from smaller vendors. Accordingly, the increased volume risks sameness. (Seriously. How many brands of bananas are there, anyway?)

Or maybe not. Mackey has certainly proved that he can create a unique store, and with more products going organic, he may be scaling at exactly the right time. But that still doesn't entirely eliminate the problem. Grocers have vast distribution networks and megacontracts with food service firms such as Sysco (NYSE:SYY). Not Whole Foods. According to a BusinessWeek article from last October, its distribution is split into 11 regional divisions, each managed somewhat autonomously. That's terribly inefficient.

Still fresh?
The inefficiency is worrisome for several reasons. First, it could drastically impede growth. Second, it could create issues with meeting demand and, thereby, margins. And, third, it could tarnish the company's currently pristine image.

In fact, it's that image that worries me most. Whole Foods' moat -- that is, its ability to charge premium prices -- has everything to do with a brand that bespeaks of quality and service. It's what feeds the company's unendingly loyal customer base, and it's what converts curious visitors into repeat shoppers. With a P/E ratio roughly three times greater than its long-term earnings growth rate, Whole Foods can't afford even a minor quality problem. Growth must be executed flawlessly, yet its supply chain is still very much lacking.

Losing the expectations game
Then there's the expectations game. For years, Whole Foods routinely blew away analyst estimates. And in the process, it became a darling of the Street. No longer. Suddenly, Whole Foods can't even meet estimates. At least that's the way it has been for the past two quarters.

Don't get me wrong. I'm not about to say you shouldn't buy Whole Foods because it doesn't properly kowtow to Wall Street's elite. But I am saying that investors have come to expect a certain standard from Whole Foods -- a standard that, recently, it hasn't been able to meet. That suggests the stock has run too far, too fast, and needs badly to catch its breath. There's absolutely nothing wrong with that -- especially over the short run -- but it doesn't necessarily bode well for long-term, market-crushing returns, either.

The bottom line is simple: Whole Foods is a wonderful company with a wonderful culture and a necessary set of products. But there's a fair price to every business. For Whole Foods, three times its long-term expected growth rate is anything but fair.

Whole Foods is a Motley Fool Stock Advisor selection. Ask us for a30-day all-access passto find out which other stocks are helping David and Tom Gardner wallop the market by more than 25% each. Orsubscribeorrenewnow, and we'll throw inStocks 2006, which features our analysts' best picks for the year ahead. All you have to lose is the prospect of greater returns.

Think you're done with the Duel? You're not! Go back and read the other three arguments, and then vote for a winner.

Fool contributor Tim Beyers eats loads of organic food. But Wild Oats is closer to his house than Whole Foods is. Tim didn't own shares in any of the companies mentioned in this story at the time of publication. You can find out what is in his portfolio by checking Tim's Fool profile . The Motley Fool has an ironclad disclosure policy .