Shares of Qualcomm (NASDAQ:QCOM) have rallied more than 25% this year on robust demand for its Snapdragon 820 chips, promising growth into new markets, and the stabilization of the company's patent licensing business. But despite that impressive recovery, Qualcomm faces four big challenges which could throttle its near-term growth.

Image source: Qualcomm.

1. Competition from MediaTek

Qualcomm's main rival in ARM (NASDAQ:ARMH)-based mobile chips is Taiwanese chipmaker MediaTek. According to Strategy Analytics, Qualcomm's global market share of smartphone application processors fell from 52% in 2014 to 45% in 2015. Meanwhile, MediaTek's share rose from 14% to 19%.

MediaTek's primary strategy is to sell cheaper chips than Qualcomm. MediaTek's chips are generally weaker than Qualcomm's, but their lower price tags are attractive to smartphone makers struggling with low margins. A recent study from Canaccord Genuity revealed that only Apple (NASDAQ:AAPL) and Samsung are selling smartphones at a profit, while everyone else has flat to negative device operating margins.

MediaTek is also expanding into the same markets for wearables, connected cars, drones, and other Internet of Things devices. This competition could force Qualcomm to slash its prices to preserve its market share across multiple markets.

2. Competition from first-party chips

Qualcomm also faces fierce competition from leading smartphone makers like Apple and Huawei, which use their own ARM-based chip designs instead of Qualcomm's. Samsung also installs its own Exynos chips in some of its premium devices, while Xiaomi -- one of Qualcomm's biggest customers in China -- has been developing its own Rifle chips for its low and mid-range devices.

First-party chips enable these bigger OEMs to control costs, maintain tighter control over their supply chains, and optimize their software for their own processors. Out of these first party players, Apple, is the largest chip manufacturer, growing its share of the global application processor market from 18% to 21% between 2014 and 2015.

That makes Apple a bigger chipmaker than MediaTek and the second largest mobile chipmaker after Qualcomm. If other OEMs develop their own processors, Qualcomm's chipmaking revenue, which accounts for the majority of its top line, could face slower growth ahead.

3. Lower patent licensing fees

Qualcomm's patent licensing business, which generates most of its pre-tax profits, also faces plenty of headwinds. Qualcomm's massive IP portfolio in 3G, 4G, and other wireless technologies enable it to take a 3% to 5% cut of the wholesale price of every smartphone sold worldwide.

That fee was acceptable back when smartphone margins were high, but now that margins have crumbled for most OEMs, many companies and government regulators are asking for lower fees. China hit Qualcomm with a $975 million fine over those fees last year, and the chipmaker was forced to calculate royalties based on just 65% of the net selling price of a handset -- which was slightly lower than the wholesale price.

But even after that ruling, many Chinese OEMs underreported shipments in order to pay Qualcomm lower licensing fees. The Chinese government didn't force those companies to pay up, forcing Qualcomm to renegotiate agreements and back payments with each company. Meanwhile, regulators in South Korea and Taiwan -- two of Qualcomm's biggest markets after China -- have also been scrutinizing those fees. A full-blown revolt against Qualcomm's licensing fees could take a big bite out of its future earnings growth.

4. Declining margins

All these challenges have put pressure on Qualcomm's margins. While Qualcomm's margins won't turn negative anytime soon, its gross margin has clearly trended lower over the past decade.

QCOM Gross Profit Margin (Quarterly) Chart

Source: YCharts

To remain competitive in mobile chips and refresh its patent licensing portfolio ahead of upcoming expirations, Qualcomm must pour a lot of money into R&D. Its R&D expenses declined 10% annually to 21% of its revenues last quarter after the launch of the Snapdragon 820, but that percentage will likely rise in the future as it spends more to stay ahead of MediaTek, first-party chipmakers, and defiant OEMs.

The key takeaway

Qualcomm posted some improvement in chip sales and securing new licensing contracts from Chinese OEMs, but analysts still expect its sales and earnings to both fall 8% this year.

This means that its upside potential should be limited until it widens its moat with non-smartphone chips and ensures that its patent licensing fees can keep growing in the low-margin market of mobile devices. But on the bright side, Qualcomm's forward P/E of 13 and its forward yield of 3.4% should limit the stock's downside for the foreseeable future.

This article represents the opinion of the writer, who may disagree with the “official” recommendation position of a Motley Fool premium advisory service. We’re motley! Questioning an investing thesis -- even one of our own -- helps us all think critically about investing and make decisions that help us become smarter, happier, and richer.