AI Term of the Moment
AI Pin
Definition: high bandwidth memory
High-speed memory used with GPUs for AI and other high-performance applications. High bandwidth memory (HBM) was introduced by SK Hynix in 2013 and first used in a high-end AMD graphics card. In 2016, NVIDIA used HBM in an AI accelerator. Today, HBM is the primary memory attached to GPUs. In time, because of its space savings, HBM is expected in laptops.
A Much Wider Interface
Instead of a single channel to memory, HBM has multiple channels. For example, an HBM4 2,048-bit interface comprises 32 64-bit channels, which results in a much faster memory bandwidth. See Hybrid Memory Cube and GDDR.
How HBM and DDR Work Together for AI
For AI processing, the common DRAM memory, which is used in every computing device, holds the program code as well as the data before GPU processing. HBM holds the data the GPU works on, which are the nodes and layers of the neural network. In fact, HBM often resides on the same chip package as the GPU. See also high bandwidth flash.
A Micron 24GB HBM3e Cube
Regular Memory and HBM
A Much Wider Interface
Instead of a single channel to memory, HBM has multiple channels. For example, an HBM4 2,048-bit interface comprises 32 64-bit channels, which results in a much faster memory bandwidth. See Hybrid Memory Cube and GDDR.
How HBM and DDR Work Together for AI
For AI processing, the common DRAM memory, which is used in every computing device, holds the program code as well as the data before GPU processing. HBM holds the data the GPU works on, which are the nodes and layers of the neural network. In fact, HBM often resides on the same chip package as the GPU. See also high bandwidth flash.
HBM uses a 3D stacked architecture of dynamic RAM (DRAM) modules, which must be perfectly aligned over each other. For example, HBM4 stacks 16 DDR layers, each holding 4GB for a total of 64GB. SK Hynix was first to ship HBM in 2013, but Micron became a leading supplier by 2026. (Image courtesy of Micron Technology, Inc.)
Data Layers x GB
Date of Rate =
Release (GB/s) Total GB Volts
HBM4 2026 1638 16x4=64 0.4
HBM3e 2023 1229 16x3=48 0.4
HBM3 2022 819 12x2=24 0.4
HBM2e 2019 461 12x2=25 1.2
HBM2 2016 307 8x1=8 1.2
HBM1 2013 128 4x1=4 1.2
This diagram of an NVIDIA Grace Hopper Superchip shows how regular dynamic RAM memory and HBM are used for AI processing. LPDDR stands for low-power double data rate, which is the common memory technology today. See NVIDIA Grace Hopper Superchip.