The latest approach to training GPT-style AI models and generating answers (inference). Attention is a mathematical scoring system that enables the model to learn language patterns more effectively. It computes a "relationship score" between every word in the text to learn which words depend on each other, and it can determine relationships between words that are farther apart in a block of text.
A model trained with Attention can generate up to four additional matrices that a conventional neural network does not have. For example, modern GPT-style neural networks have a Query, Key, Value and Output matrix. See
backpropagation.