The most advanced neural network architecture to date. The transformer is a major improvement over the recurrent neural network (RNN). Initially presented in a 2017 paper by eight scientists at Google entitled "Attention Is All You Need," a transformer is able to understand the relationships between words in a sentence much better. Each word (token) is assigned a relevance score and is weighed with every other word in the sentence.
Tokens, Embeddings and Vectors
First cleaned by removing punctuation and symbols, the text is broken up into "tokens." The numerical representation of tokens is called an "embedding" and the mathematical form of the embedding is called a "vector. In a neural network, the embeddings are spread across many neurons. See
neural network and
AI token.
Will Some New Method Be Better?
Why not? Technology never stands still for too long. However, from the standpoint of global popularity, the AI industry is very new even though it was discussed and debated in the 1950s. A hardware-based neuron was even developed back then (see
SNARC). In the meantime, the transformer method is transforming the world.