Definition: neural network

The architecture of today's small and large language models, which provide the source of knowledge and intelligence that chatbots such as ChatGPT, Gemini and Claude use to generate answers.

Essentially a "pattern recognition" system, a neural network is technically an "artificial" neural network (ANN) because it is only loosely based on the human nervous system and is by no means a true replica. Hypothesized in the 1940s, there were elementary neural networks back in the 1950s (see Mark I Perceptron).

The neural network is the architecture of chatbots and image/video creation (generative AI), as well as image, language and speech recognition, text-to-speech conversion, diagnosing and forecasting. The neural network has no resemblance to the regular business applications programmed for the past 75 years (if this-do that). Large language models are "trained" on millions and billions of words and images, and the answers generated for users are provided by software known as "inference." See inference, AI secret sauce, AI programming, AI training, AI model and generative AI.

Servers and GPUs
Data centers can have thousands of servers. The more servers, the larger the neural network, the larger the language model, the greater capacity to learn complex patterns and relationships. Desktop and mobile devices run the chatbots that communicate with the models in a data center; however, local devices are increasingly used to run small models and even train them; for example, see DGX Spark.

Every server has a number of GPUs, each having hundreds or thousands of tiny math processors that calculate in parallel. The more GPUs, the shorter both the training time and inference (execution) because AI processing is constant computation. However, in a large data center, even with tens of thousands of servers, it can take months to train a language model with a trillion parameters (numbers) by constantly multiplying and adding them. To reduce the training time of ever-larger models and to support the millions of people who want to use them at the same time, an AI "factory" can have a million GPUs. In fact, Meta and SpaceX have both announced a million GPUs for their AI factories. See GPU and AI training vs. inference.

Layers and Nodes

A neural network comprises multiple layers of computational units called "nodes" or "neurons" that are mathematically connected to each other (see diagrams below). There is an input layer, an output layer and any number of "hidden" layers in between. To learn how the network is trained, see backpropagation.

A Single Neuron/Node

A neuron, or "node," is a mathematical unit within the network, and a single GPU processes the calculations for thousands of neurons simultaneously. The activation function determines if the value leaving the neuron should be altered. The weights and bias numbers are modified in the training passes as answers are compared against predictions. If this all sounds very strange... it is! See weights and biases.

Tracing a Neuron
The neural network is a pattern detection system. The text used for training the network is turned into "tokens" (words, parts of words and punctuation). A token is converted into an "embedding," which is a one-dimensional array of numbers known as a "vector." Each layer transforms the embeddings mathematically and makes them increasingly useful for predicting the next token. See AI token and backpropagation.

Training models means constantly predicting what comes next, evaluating that prediction and changing the numbers to make the model more intelligent (see AI training passes). Essentially, AI companies have taken all the information ever published and converted it into a gigantic maze of numbers! See AI secret sauce and copyright infringement.

A Tiny Network

This neural network has only 17 neurons/nodes in three hidden layers and might be used for limited pattern recognition in a small sensor.

A Larger Network

This neural network has 8,192 neurons/nodes in eight hidden layers for a total of 65 thousand nodes and 4.3 billion connections that are adjusted in the training stages. Now imagine a neural network with 100 million nodes and you can begin to visualize the complexity of large AI models. For the neural network that is easier to grasp, see convolutional neural network.

It's All Just Numbers

Part of the neural network, the embedding table is created with one row for each token in the model's vocabulary. For example, when the model sees token #2, it looks up row 2 in the embedding table. During training, the embedding table is constantly updated, but for inference (execution), it remains constant just like the model. This example shows three hidden layers, but imagine a model with hundreds of hidden layers and thousands of neurons in each layer, and you understand why neural networks are enormous "math machines!"

There Are Many Neural Network Designs

This is a very small sample from the Asimov Institute in the Netherlands. Only a few layers are shown for each type, but there can be hundreds of layers between the input and output. (Images courtesy of Fjodor van Veen and Stefan Leijnen (2019). The Neural Network Zoo.)

misc

AI Term of the Moment

BlueField

Look Up Another Term

Definition: neural network