The past decade wasn't kind to AMD (AMD 2.24%), which ceded the x86 CPU market to Intel (INTC -9.63%) and lost the GPU market to Nvidia (NVDA 5.61%). AMD's CPUs and GPUs were cheaper, but they always seemed to lag behind Intel and Nvidia in terms of architecture and power efficiency.

Image source: AMD investor presentation.

Between 2005 and 2015, AMD's annual revenue dropped from $5.85 billion to $3.99 billion. During that same period, Intel's annual revenue rose 43% and Nvidia's revenue more than doubled. AMD lost nearly 70% of its market value over the past decade and went through three CEOs before Lisa Su took over in late 2014.

But under Su, AMD recently scored a major victory during its second quarter by achieving its first period of annual sales growth since the second quarter of 2014. Sales rose 9% to $1.03 billion during the quarter, beating estimates by about $75 million. AMD expects that growth to continue with about 18% sales growth for the third quarter. That news lifted AMD stock to four-year highs.

AMD survives by expanding into niche markets

Over the past few years, AMD realized that attacking Intel and Nvidia head-on was futile, since both companies had superior cash flows and higher R&D and marketing budgets. Instead, AMD pivoted toward certain niche markets which its rivals had overlooked.

In 2013, it scored design wins with Sony and Microsoft to respectively power the PS4 and Xbox One with its semi-custom APUs -- which combine the functions of a CPU and GPU on a single SoC (system on chip). It also supplied the GPU for Nintendo's Wii U, which effectively locked Nvidia out of the eighth generation of gaming consoles.

All powered by AMD. Image source: Company websites.

Sales of semi-custom SoCs for consoles became a pillar of growth for AMD's EESC (Enterprise, Embedded, and Semi-Custom) business, which generated 58% of its sales last quarter. Revenue at the unit rose 5% annually and operating income more than tripled to $84 million. That growth offset an $81 million operating loss at its computing and graphics business. Looking ahead, the EESC business will likely get a boost from new updated consoles like Sony's PS4 Neo and Microsoft's Xbox One S and Scorpio.

AMD also sees niche growth potential in data centers, a market which Intel dominates with its Xeon processors. Instead of competing against Intel with x86 CPUs for high-end servers, AMD has started shipping ARM licensed chips for lower-end servers. To complement that move, AMD formed a joint venture with Chinese firm THATIC earlier this year to develop SoCs for the Chinese server market. It also formed another joint venture with Nantong Fujitsu Microelectronics to expand its ATMP (assembly, test, mark, and pack) capabilities in China.

AMD could benefit from the growth of virtual reality

Tech M&A advisory firm Digi-Capital estimates that the fledgling virtual reality market could grow from practically nothing today to $30 billion by 2020. That growth could benefit AMD in two major ways.

When the Oculus Rift and HTC Vive were launched earlier this year, consumers had to buy a $1,000 PC to use the headsets. Since the Rift costs $600 and the Vive costs $800, VR remained a hobby for rich tech enthusiasts instead of mainstream consumers. However, AMD recently lowered the price of VR-ready PCs considerably with its $200 RX 480, the cheapest VR-capable card on the market. Current PC builds using the RX 480 have reduced the price of a VR-ready PC to just $600.

The Rift and Vive currently depend on the PC's GPU power, but companies will likely launch stand-alone headsets which aren't tethered to desktops in the near future. Like gaming consoles, these devices will run better on efficient semi-custom APUs instead of stand-alone CPUs and GPUs. AMD's market-dominating position in semi-custom APUs for consoles gives it a major advantage in this market.

Don't call it a "turnaround" just yet...

Posting 9% sales growth might not seem like a huge victory, but it's certainly AMD's biggest win of the year so far. However, investors shouldn't ignore three upcoming challenges that could derail AMD's turnaround.

First, Nvidia's new GTX cards offer a lot of power for just a little more cash. The new GTX 1060, for example, offers better performance than the RX 480 for just $50 more. Second, AMD must prove that its Zen x86 chips, which will arrive at the end of the year, can effectively compete against Intel's upcoming Kaby Lake and stop the bleeding at its CPU business. Lastly, Nintendo has reportedly selected Nvidia's Tegra to power its upcoming NX console, which will likely arrive next March. If the NX is a surprise hit like the original Wii, it could hurt sales of the PS4 and Xbox One and AMD's EESC sales.

I think AMD's outlook is improving, but I'd be hesitant to call it a "turnaround" until it can overcome those challenges. But if AMD's revenues can continue growing year-over-year for the next few quarters, it could be the start of a major comeback for the beaten-down chipmaker.