AI Term of the Moment
Hugging Face
Redirected from: HBM
Definition: high bandwidth memory
High-speed memory used with GPUs for AI and other high-performance applications. High bandwidth memory (HBM) uses a 3D stacked architecture of dynamic RAM (DRAM) modules. In time, because of its space savings, HBM is expected to be employed in laptops. Micron (U.S), Samsung and SK Hynix (South Korea) are major HBM manufacturers.
A Much Wider Interface
The 4096-bit interface connecting HBM to the CPU or GPU is eight times wider than the 512 bits used for the common dynamic RAM (DRAM) memory. This results in memory that is 10 to 30 times faster. See Hybrid Memory Cube and GDDR.
For AI, HBM and DDR Work Together
For AI processing, the common DRAM memory, which is used in every computing device, holds the program code as well as the data before GPU processing. HBM holds the data the GPU works on, which are the nodes and layers of the neural network. In fact, HBM often resides on the same chip package as the GPU. See also high bandwidth flash.
A Micron 24GB HBM3e Cube
HBM in a Superchip
A Much Wider Interface
The 4096-bit interface connecting HBM to the CPU or GPU is eight times wider than the 512 bits used for the common dynamic RAM (DRAM) memory. This results in memory that is 10 to 30 times faster. See Hybrid Memory Cube and GDDR.
For AI, HBM and DDR Work Together
For AI processing, the common DRAM memory, which is used in every computing device, holds the program code as well as the data before GPU processing. HBM holds the data the GPU works on, which are the nodes and layers of the neural network. In fact, HBM often resides on the same chip package as the GPU. See also high bandwidth flash.
HBM modules are built in layers that must be perfectly aligned over each other. For example, HBM4 stacks 16 DDR layers, each holding 4GB for a total of 64GB. SK Hynix was first to ship HBM in 2013, but Micron became the leading supplier of HBM3 by 2026. (Image courtesy of Micron Technology, Inc.)
Data Layers x GB
Date of Rate =
Release (GB/s) Total GB Volts
HBM4 2026 1638 16x4=64 0.4
HBM3e 2023 1229 16x3=48 0.4
HBM3 2022 819 12x2=24 0.4
HBM2e 2019 461 12x2=25 1.2
HBM2 2016 307 8x1=8 1.2
HBM1 2013 128 4x1=4 1.2
NVIDIA's Superchip uses HBM memory for its built-in GPU. See NVIDIA Grace Hopper Superchip.