We all have one of those stocks in our portfolio. The red-headed stepchild. The one that could have soooo much potential if it would just stop hanging out at the mall with that terrible Eddie Shanks and apply itself. (Heavens, Mildred, how that stock frustrates me!)

For me, that stock is New York & Co. (NYSE:NWY).

This women's retailer, a Motley Fool Hidden Gems pick, is clearly failing to succeed in the rough-and-tumble world of women's fashion retailing, where specialty competitors like Ann Taylor (NYSE:ANN), Talbots (NYSE:TLB), Limited Brands (NYSE:LTD), and Gap (NYSE:GPS) are throwing elbows all month long, while big bruisers such as Nordstrom (NYSE:JWN) and Federated Department Stores (NYSE:FD) look to protect their turf.

The May same-store sales numbers show more than a 5% drop. Personally, I figured this was on the way, and I said as much in a report on last month's numbers. Yet I hoped for better, though I know hoping for better from a serial disappointer like New York & Co. is as clever as digging my toast out of the toaster with a fork.

Perhaps my colleague and fellow retail rat, Stephen Simpson, summed up my feelings best when he asked me, "Down 5%? How the [H-E-double toothpicks] do you run a company not named Hot Topic this badly?"

Well, one explanation is that you're just not doing the right thing, selling the right look. You know, moving the merchandise. That, I believe the record shows, is clearly the case. New York & Co. put up anemic comps "growth" for far too many months.

The other, as my colleague Bill Mann recently suggested, is that it takes more than a month or three to clear out a season's worth of mistakes, let alone the two-season flop that still seems to weigh down New York & Co.

That said, the measure of New York & Co.'s potential is evident in the volatile stock price, which is moving up today, despite this worse-than-expected performance. When it's going for $12 a share, New York & Co. is priced for failure. When it's more like $17 a share, Mr. Market expects a modicum of success.

Until there's better evidence that New York & Co. can actually deliver that success in a consistent fashion (pun intended, for once), buy your shares accordingly.

New York & Co. is one of thoseHidden Gemspicks that isn't working out quite as expected. It happens. But if you want to learn about powering a total portfolio with diversified small-cap investing, afree Hidden Gems trialwill get you a look at the full stable of market-beaters.

Gap is both a Stock Advisor and Inside Value recommendation.

Seth Jayson prefers a little more execution, but he can't argue with cheap. At the time of publication, he had shares of New York & Co., but no positions in any other firm mentioned. View his stock holdings and Fool profile here. Fool rules are here.