The bottleneck in AI and other memory-intensive applications whereby the transfer of data to and from memory is the slowest operation. For example, CPU register and cache cycle times are less than one nanosecond, whereas the latest dynamic RAM (GDDR) access time can be from 10 to 100 nanoseconds. Flash memory (storage) is much slower, taking thousands of nanoseconds. See
compute-in-memory.
GDDR vs. HBM
GDDR memory is used for gaming GPUs, whereas high bandwidth memory (HBM) with cycle times less than one nanosecond are used with AI GPUs, especially in datacenters (see
high bandwidth memory and
GDDR).