Advanced Micro Devices (NASDAQ:AMD) shot into the limelight a couple of years ago when it announced that it would be targeting the high-end computing market with its Ryzen CPUs and Vega GPUs. In fact, 2016 was one of AMD's best years on the stock market after three years of underperformance, as investors thought that it would take the game to the likes of NVIDIA (NASDAQ:NVDA) and Intel (NASDAQ:INTC), which dominate this space.

AMD Chart

AMD data by YCharts.

But reality struck hard in 2017 after the initial hype around its new products died down. AMD conceded market share to NVIDIA after the launch of its new graphics cards, as they turned out to be a year behind NVIDIA's offerings. Meanwhile, AMD was locked in an intense battle with Intel in the desktop CPU and server processor markets, succeeding over there to some extent.

AMD's mixed 2017 dented investor confidence, and the stock has underperformed the market by a wide margin over the past year. Is this the beginning of yet another streak of underperformance for AMD, or should investors remain patient in the hope that it will beat the market over the next five years? Let's find out.

AMD Ryzen logo.

Image Source: AMD. 

A look at GPU prospects

NVIDIA is completely dominant in GPUs right now with a market share of almost 73%, according to the latest data from Jon Peddie Research. This is a great position to be in as Allied Market Research expects global GPU sales to grow at a compounded annual growth rate (CAGR) of 35.6% over the next five years thanks to their applications in augmented reality (AR), virtual reality (VR), artificial intelligence (AI), and increasingly complex computer games.

NVIDIA has deeply embedded itself across all segments of the GPU market, be it games, AI, or AR and VR. More important, it has established a technology lead over AMD. For instance, AMD's Vega GPUs, launched in 2017, barely match NVIDIA's two-year-old offerings in terms of pricing, performance, and power efficiency.

NVIDIA has established itself as the go-to provider in the data center space, where GPU demand is rapidly increasing to power AI applications. It had scored wins at all the top data center providers and cloud service providers before AMD's Vega consumer cards came to the market.

AMD was late to the game, as its cloud-centric Radeon Instinct GPUs went on sale only in the second quarter of 2017. But NVIDIA had swept this market for itself by then as its Titan X GPU was available for training deep-learning applications on the cloud back in 2016. So it is not surprising to see why NVIDIA's GPUs are available across all the top cloud service providers.

More important, NVIDIA customers like Amazon Web Services are already upgrading to new GPUs based on the Volta architecture, making it difficult for AMD to make inroads there.

In fact, NVIDIA's absolute domination in the GPU space has forced rivals AMD and Intel to form an alliance. Intel recently announced Core i5 and Core i7 chips with integrated AMD Radeon RX Vega GPUs targeted at the notebook market. Intel and AMD believe that they can help laptop original equipment manufacturers (OEMs) save space on their notebooks with an integrated chip platform.

This will allow OEMs to increase battery life by packing a bigger battery pack or bump up performance by integrating a bigger memory. But the success of this partnership isn't guaranteed as NVIDIA's Max-Q design already allows laptop makers to offer thin form factors without compromising on performance.

And NVIDIA is reportedly going to counter the Intel-AMD combine with a budget Max-Q chip based on the older GTX 1050 and 1050 Ti GPUs. So AMD's prospects in the GPU space appear to be on thin ice.

What about the CPU market?

AMD has made good progress in CPUs, taking market share from Intel thanks to its Ryzen chips. The company is now aggressively going after Intel in this space, recently claiming that its processors can match Intel's offerings at way lower price points.

At the Consumer Electronics Show (CES) in Las Vegas earlier this month, AMD pointed out that one of its integrated CPU-GPU chips based on the Ryzen platform can match the performance of Intel's Core i5-8400 chip with an NVIDIA GT 1030 graphics card -- and at a price point that's $120 lower.

This is AMD's go-to-market strategy: cutting-edge performance on a budget. CEO Lisa Su made this very clear at CES 2018 when she said that "at every price point, we offer more threads, more multi-threaded performance."

AMD isn't done attacking Intel's position in the CPU space just yet as it will soon come out with a new slate of products. The company indicates that Ryzen CPUs based on Zen 2 and Zen 3 architectures that will be released in 2019 and 2020, respectively, will deliver better performance gains when compared to the 7% to 8% jump usually seen in Intel's recent generational changes.

AMD is also giving Intel sleepless nights in the server processor market. Last year's launch of the EPYC server processor has improved AMD's standing in this market, where Intel currently has a monopoly like position. For instance, the likes of Microsoft, Baidu, and Hewlett-Packard Enterprise have already deployed or are in the process of deploying EPYC-based server systems or cloud computing services.

So AMD can claw back market share from Intel in server processors and CPUs because of its aggressive product development and rapidly growing partnerships. But it remains to be seen if it can gain enough traction in the CPU market to overcome the challenge posed by NVIDIA in GPUs. Investors need to be cautious when investing in AMD, as it could witness a period of volatility over the next five years because of its fight against bigger players like Intel and NVIDIA.

Teresa Kersten is an employee of LinkedIn and is a member of The Motley Fool's board of directors. LinkedIn is owned by Microsoft. Harsh Chauhan has no position in any of the stocks mentioned. The Motley Fool owns shares of and recommends Baidu and Nvidia. The Motley Fool recommends Intel. The Motley Fool has a disclosure policy.