The artificial intelligence (AI) megatrend has been the biggest catalyst moving the stock market since ChatGPT debuted in November 2022. Ever since, hyperscalers, AI specialists, and governments have been spending boatloads of money and using huge amounts of data to train powerful large language models (LLMs).
However, Nvidia CEO Jensen Huang recently pointed out that AI is now at an inflection point.
Whereas previously, a large volume of AI processing power was dedicated to model training, inference -- the actual use of those models -- is poised to become the bigger driver in this space. The time to shift the focus from model training to putting those models to work in the real world seems to have finally arrived after years of huge infrastructure investments.
Inference-based agentic AI tools are gaining popularity because they can automate processes, perform tasks independently, improve productivity, and reduce costs. And the mainstream use of physical AI tools, such as humanoid robots, appears to be moving closer.
According to a forecast from market research company S&S Insider, the AI inference market is poised to quadruple in size from $87 billion in 2024 to $350 billion in 2032.
There are multiple semiconductor stocks one could buy to capitalize on this terrific opportunity. But one seems to me to be the ultimate pick-and-shovel play on the inference economy.
Image source: Getty Images.
Meet Arm Holdings, the AI company powering inference processors
AI inference doesn't need as much processing power as model training. This explains why hyperscalers and AI companies have been shifting toward specialized processors, such as application-specific integrated circuits (ASICs) and central processing units (CPUs), which are sufficient for some inference tasks.
Anthropic, for instance, has extended its partnership with Alphabet's Google and Broadcom to deploy 3.5 gigawatts of the search engine giant's custom tensor processing units (TPUs). Google and Broadcom have been working together to design TPUs for the past decade, and these chips are now playing a central role in AI inference due to their performance and cost advantages.
NASDAQ: ARM
Key Data Points
Google announced the availability of its seventh-generation TPUs, known as Ironwood, in November 2025, promising significant performance gains over its previous-generation chips. At the same time, it announced its Axion general-purpose CPU, based on Arm Holdings' (ARM +2.71%) architecture.
Google claims that the Arm-based Axion server CPUs are up to two times better than comparable x86 processors from Intel and AMD in price-to-performance terms. This may explain why Anthropic has chosen Google's data centers to run its AI workloads. And Google and Broadcom aren't the only ones developing Arm-based server CPUs.
Nvidia's next-generation Vera server CPU, which the company claims is purpose-built for agentic AI, is based on the Armv9.2 architecture. The chipmaker has decided to offer its Vera server CPU on a standalone basis to customers. This is a first for management because it offered its prior-generation Grace server CPUs in rack-scale systems integrated with its Hopper graphics processing units (GPUs).
Nvidia estimates that the Vera CPU could become a multibillion-dollar product. That's great news for Arm, which generates revenue by licensing its chip designs to customers, and earning royalties on each chip made using its intellectual property (IP).
Marvell Technology, the second-largest custom AI processor company, relies on Arm's IP to design its ASICs. As such, it is easy to see why Arm is expected to become the kingpin of the AI inference economy in the next few years by becoming the dominant player in custom ASICs.
Arm is expected to dominate the custom AI processor market
Market research firm Counterpoint Research estimates that the share of Arm-based server CPUs in custom AI processor servers could jump to 90% by 2029. That would be a huge increase over last year's share of just 25%. The server CPU market is currently dominated by Intel and AMD, which manufacture chips based on the x86 architecture.
However, Counterpoint says that the number of companies developing custom in-house chips using Arm's IP will increase rapidly. This should ideally lead to healthy growth in Arm's revenue and earnings, opening new streams of licensing fees and royalties. Arm's AI-focused Armv9 architecture commands double the royalty rate of the previous-generation Armv8 architecture.
So, the continued adoption of Armv9 should produce a bigger spike in the company's earnings in the long run. Analysts are expecting a 21% increase in Arm's earnings per share (EPS) in its fiscal 2027 (which began April 1) to $2.14, followed by a 35% jump in the next fiscal year.
ARM EPS Estimates for Current Fiscal Year, data by YCharts.
This all suggests that Arm's earnings acceleration could continue through the end of the decade, paving the way for impressive gains in this AI stock over the long run.
