Xiaomi's latest flagship, the Mi 5. Source: Xiaomi

Things aren't looking great for Chinese handset-maker Xiaomi.

Not too long ago, the fast-growing start-up appeared unstoppable. The phone specialist, which was founded only six years ago, rose rapidly to become one of the world's largest sellers of smartphones. By dominating its home market of China, and expanding into India, Xiaomi's ascendance as one of the globe's premier mobile vendors appeared inevitable. It hadn't made much progress in established economies like the U.S. or Europe, but for most of the last two years, Xiaomi looked like a legitimate threat to industry stalwarts Apple (NASDAQ: AAPL) and Samsung (NASDAQOTH: SSNLF).

No longer. Xiaomi's handset sales have failed to live up to expectations in recent quarters, and the company has been overtaken by several Chinese rivals. Xiaomi remains relevant, but the firm's handsets seem less and less likely to pose much of a challenge to Apple's iPhone.

Falling out of the top five
Xiaomi wasn't among the top five smartphone vendors in the first quarter of the year, according to research firm IDC. Samsung led the way with 81.9 million handset shipments, followed by Apple with 51.2 million. Huawei came in third place, then newcomers OPPO and vivo. To be clear, both firms have been making handsets for some time, but had not previously appeared in IDC's list of the world's top vendors.

Source: IDC.

That's a harsh blow for Xiaomi. The company was the fifth-largest smartphone vendor in 2015, shipping 70.8 million handsets. That was far behind Samsung's 324.8 million and Apple's 231.5 million, but wasn't too far removed from Huawei's 106.6 million.

Source: IDC.

The drop is particularly egregious when viewed within the context of Xiaomi's product portfolio. The company's latest flagship, the Mi 5, made its debut in the first quarter. For a company whose primary selling point is affordability, its most expensive model may not be its most successful. Still, it should've provided some lift to the company's sales. Evidently, it was not enough.

It's possible Xiaomi could bounce back, but the broader smartphone market appears to be moving against the company. Xiaomi's fall from the top five is just another event in a recent string of misfortunates. In 2014, Xiaomi's handset shipments surged an incredible 227%, and management was optimistic that it could maintain that momentum. Early last year, Xiaomi said it could sell between 80 million and 100 million handsets in 2015. But with fewer than 71 million shipments, it fell far short of that goal, and growth slowed to a modest 22.8%.

Competition in the smartphone market remains brutal
Xiaomi's experience may serve as a cautionary reminder of just how difficult it is to compete in the smartphone market. Xiaomi attracted attention by offering powerful phones at affordable prices, but there was more to the company than cheap hardware. Xiaomi emerged as one of the world's top phone vendors despite spending almost nothing on traditional marketing; instead, it relied on viral campaigns and a network of rabid fans to help popularize its products.

And those fans truly seemed to love its handsets. Back in 2014, analytics firm Flurry noted that, in China, owners of Xiaomi's phones spent more time using apps than did owners of Apple's iPhone. Apple has always relied on engagement metrics to illustrate the strength of the iPhone business, and if Xiaomi's users were more engaged than Apple's, it would spell trouble for Chinese iPhone demand.

At the same time, Xiaomi has made no secret about its desires to expand. The company has built a successful business in other emerging markets, most notably India, and has been open about its plans to eventually target the U.S. and Western Europe. More than a Chinese threat, Xiaomi appeared to be a long-term challenger on a global basis.

Xiaomi will still sell tens of millions of smartphones this year, and play a major role in China, but with its popularity tumbling, it's no longer the menace it once appeared to be.

This article represents the opinion of the writer, who may disagree with the “official” recommendation position of a Motley Fool premium advisory service. We’re motley! Questioning an investing thesis -- even one of our own -- helps us all think critically about investing and make decisions that help us become smarter, happier, and richer.