bottleneckScore 80/100Research
Jensen Huang confirms memory bandwidth is the critical bottleneck in AI model training and inference, not logic or raw HBM capacity
Jensen Huang· NVIDIA· AI· 2026-05-27· about NVIDIA (NVDA)
“the bottleneck often in training and doing inference on these models is the amount of bandwidth. So if you HBM2 I I don't know the numbers off hand but like versus the newest thing you have, you know, you you can be almost an order of magnitude difference in memory bandwidth”
Why it matters
This reveals that HBM capacity alone is insufficient; memory bandwidth is the constraining factor in AI infrastructure. Older HBM2 tech can be nearly an order of magnitude slower than the latest generation, creating demand for next-gen HBM solutions.
Investment implication
Companies supplying advanced HBM (SK Hynix, Samsung, Micron) and packaging solutions that enable higher bandwidth interconnects (advanced chiplets, 3D stacking, silicon photonics) stand to benefit from this bandwidth bottleneck becoming a primary design driver.