Micron Puts a Number on the Memory Wall at Hot Chips: Accelerator TFLOPS 3x Every Two Years, HBM Bandwidth Under 2x

ServeTheHome·medium signalXBlueskyLinkedInCopy link

Micron's August 23 Hot Chips session argued that AI accelerator throughput grows roughly 3x every two years while 2.5D-attached memory bandwidth climbs under 2x in the same period, making memory the binding constraint on system performance. The talk walked HBM1 through HBM4, with HBM3E at 128 banks per die doubling to 256 banks in HBM4, and showed a typical GPU package exceeding 12,000 square millimeters once eight HBM4 stacks are included. Micron also noted roughly 3x as much silicon is consumed to deliver the same HBM3E capacity as DDR5, alongside CoWoS-L, CoWoS-R, and glass substrate packaging.