Definition: long-horizon context

The capability of an AI model to analyze a huge amount of information beyond the current "context window," which is essentially the amount of memory available.

Training and Inference
Long-horizon context is constantly being improved for both training and inference. For example, larger context windows see dependencies over a greater amount of training data. If a book were the training input, the context window might be a chapter. In 2026, the Kimi K3 model from China's Moonshot AI had a one-million token context window, which could be the whole book. See AI token.

At the inference stage, an improved long-horizon context means generating answers faster (see below). See AI training vs. inference.

Context GPUs for Inference Processing

In this Vera Rubin example from NVIDIA, the context GPUs (CPX GPUs) handle large amounts of memory to load the user's prompt in the "prefill" stage. The regular Rubin GPUs use high-speed HBM memory to generate the response. See GPU. (Image courtesy of NVIDIA.