AI servers use two types of memory, the traditional dynamic RAM (DRAM) and high bandwidth memory (HBM). Dynamic RAM is the common memory chip in every computing device and appliance with a CPU.
HBM is much faster than DRAM but costs roughly twice as much, which is why both types are used in AI servers. Regular DDR (double data rate) dynamic RAM is used by the CPU for orchestration, and HBM is used with the GPUs for high-speed computations when training AI models and generating answers (inference) from them.
DRAM Memory Shortage
By 2025, so many memory chip companies had switched to making HBM that regular DRAM memory was becoming scarce with prices more than tripling in some segments. Gamers and PC enthusiasts wanting to upgrade their machines felt the pain. See
neural network,
dynamic RAM and
high bandwidth memory.