The artificial intelligence (AI) infrastructure market is booming, with five of the largest hyperscalers (owners of massive data centers) alone set to spend an eye-popping $700 billion in 2026. To put that in perspective, that exceeds the gross domestic product (GDP) of all but 24 countries.
Right now, this spending is being done for two main purposes. The first is to train large language models (LLMs), such as OpenAI's ChatGPT, Anthropic's Claude, and Alphabet's Gemini. The other is to support AI inference, which then deploys the models to answer queries.
Image source: Getty Images.
Nvidia (NVDA 0.98%) is the clear leader in AI model training, and it has established a wide moat in this area through its CUDA software platform, which is where most foundational AI code has been written and optimized for its graphics processing units (GPUs). However, the company is also the leader in inference. Its Blackwell GB300 Ultra chips were designed specifically for inference in mind, while its upcoming Vera Rubin platform is projected to offer five times the inference performance of previous generations. Meanwhile, Nvidia NIM, (Nvidia Inference Microservices) provides prebuilt, optimized inference microservices, giving it a software edge, as well.
However, Nvidia's moat for inference isn't nearly as wide as it is for training. Because of this, Advanced Micro Devices (AMD 2.04%) has been able to carve out a nice niche in the inference space with its GPUs. Meanwhile, with an investment from OpenAI and commitments from the ChatGPT maker to buy 6 gigawatts worth of its GPUs specifically for inference in the coming years, AMD should be able to gain some market share in the inference market.
However, the biggest winner in the inference market will likely be Broadcom (AVGO 3.53%).
NASDAQ: AVGO
Key Data Points
The rise of AI ASICs in the age of inference
Broadcom is a leader in ASIC (application-specific integrated circuit) technology, by which it provides the building blocks and intellectual property to help customers turn their AI chip designs into reality. It also has access to important components, like high bandwidth memory (HBM), and an established relationship with foundry Taiwan Semiconductor Manufacturing to be able to produce these custom chips at scale.
Since ASICs are hardwired to perform specific tasks, they lack the flexibility and adaptability of GPUs. However, given their specialized nature, they can outperform GPUs at the tasks for which they were designed while also being more energy efficient. This becomes an increasingly important factor with AI inference, since it is an ongoing cost every time an AI model must answer a query or perform a task. AI inference also isn't as complex as AI training, so Nvidia's software doesn't have as big an edge.
ASICs already disrupted GPUs in the cryptocurrency mining space, as they were able to offer better hash rates while being more energy efficient. Now, cryptocurrency mining isn't as technically demanding as AI workloads, as AI models do evolve, but the precedent is there, and ASICs can offer better economics for inference. That's a big opportunity for Broadcom to take share.
Broadcom has already successfully helped Alphabet develop its highly regarded tensor processing units (TPUs), which the company has been using for years to run its internal workloads. With its data center capex spending set to surge this year, Broadcom is well-positioned to profit from this growth. At the same time, Alphabet is also starting to let customers use its TPUs within Google Cloud, and Anthropic has placed a massive $21 billion TPU order with Broadcom to be delivered this year.
Meanwhile, other hyperscalers have also turned to Broadcom to help them develop their own custom AI chips. This includes OpenAI, which has committed to deploying 10 gigawatts of custom chips. Based on Nvidia GPU prices, 10 gigawatts would be worth $350 billion. ASIC prices will be lower, but that's still a huge opportunity.
With Broadcom generating just under $64 billion in total revenue in fiscal 2025, the company is set to see explosive growth in the coming years. Given its order book, the company looks poised to be the new AI inference king by the end of 2026 (even if Nvidia does technically continue to hold a higher market share).
