Shares of Nvidia (NVDA +0.10%) sold off after the company's recent earnings report and are now down about 11% from their all-time high.
Nvidia is still posting excellent growth, with revenue up 122% in the quarter, ahead of analyst expectations. But with the stock exceeding a $3 trillion market cap and expectations extremely high, Nvidia sold off anyway following the release.
One reasons could be concern over the company's gross margin. Here's what happened with this all-important metric last quarter, and whether investors should be concerned about it.
NASDAQ: NVDA
Key Data Points
Why gross margin is front and center for Nvidia and AI stocks
As the AI revolution took off, cloud computing companies and large enterprises clamored for as many of Nvidia's data center GPUs as they could get. The supply-demand mismatch enabled Nvidia to charge a premium for its chips, even as volumes grew. Hence, its gross margin expanded from a range between 55% and 65% "pre-AI" to a whopping 78.4% in the company's Q1 this year ending in April, while operating margin expanded from a range between 20% and 40% all the way up to 64.9% in the first quarter.
NVDA Gross Profit Margin (Quarterly) data by YCharts
But as you can also see, gross and operating margin took a dip in the recently reported second quarter compared with Q1. Gross margin retreated to 75.1%, a decline of 3.3 percentage points, and operating margin declined to 62.1%. Meanwhile, on the conference call with analysts, CFO Colette Kress forecast gross margin of just 74.4% in the current quarter.
That could suggest a cooling of demand or pushback on price from large customers, who may be getting tired of paying through the nose for Nvidia chips. After all, investor concerns over cloud customers' getting a return on all of their AI spending is what led to the pullback in technology stocks this summer.
But Nvidia gave different reasons for the decline
In the release, Nvidia noted that the decline in gross margin was due to "a higher mix of new products within Data Center and inventory provisions for low-yielding Blackwell material."
That explanation should let Nvidia shareholders breathe a sigh of relief, at least for now. It has been known that Nvidia's upcoming Blackwell chip would be pushed back slightly, because of a manufacturing defect discovered late in the manufacturing and sampling process.
On the call, Nvidia noted that it made a change to the GPU mask at its foundry, Taiwan Semiconductor Manufacturing (TSM +0.70%), in order to improve manufacturing yields. Of note, this appears to be a minor tweak, as CEO Jensen Huang noted no functional changes were necessary to the chip. Still, it appears there were enough defects on the initial sampling ramp-up of Blackwell that Nvidia had to scrap a fair amount of chips from early production at TSMC.
It's not clear how much of the decline in gross margin was due to inventory writedowns. But the remaining margin decline was attributed to a "mix shift to new products." This is where things get murky.
Nvidia's gross margin could continue to fall back to earth. Image source: Getty Images.
Can new products garner the same margin?
In chip manufacturing, newer products often take time to ramp up their yields, as the manufacturing process is perfected over time. But is the decline in Nvidia's gross margin due to lower initial yields in manufacturing, or is Nvidia increasingly limited in the premium it can charge for new products?
Make no mistake: The current workhorse, the H200, is no doubt more expensive than the H100. However, newer products are also more complicated, and therefore expensive, for TSMC to produce.
So it's unclear if the remainder of the gross margin decline is due to lower initial yields, which eventually go higher as production ramps up, or if Nvidia, while charging more money for new products, can't maintain the same margin as the sticker price for newer chips go higher and higher.
After all, Nvidia noted that 45% of its revenue comes from cloud service providers, all of whom are now pursuing their own in-house designed AI accelerators to lower costs. In addition, Meta Platforms (META 0.23%), which isn't a cloud service provider but also probably accounts for a big chunk of Nvidia's revenue in addition to that 45%, has begun designing its own accelerators as well. And rival Advanced Micro Devices (AMD +1.23%) forecasts above $4.5 billion in sales for its MI300 line of AI GPUs, which just came out this year and should keep Nvidia on its toes.
All of these companies, however, should continue to buy Nvidia chips. Nvidia still makes the most advanced neutral third-party GPU chip, and with developers already very familiar with its CUDA software, Nvidia should still see strong sales over the next few years.
But with all of its major customers now pursuing their own lower-cost self-designed alternatives and more competition incoming, Nvidia will have to compete that much harder. That can either come by staying a step ahead on the hardware side, or cutting prices -- or both.
Therefore, Nvidia's investors need to keep an eye on gross margin, or perhaps model in some give-back in margin in the years ahead, even as revenue marches higher.
