Term of the Moment
early computers
Redirected from: neural network model
Definition: neural network
A major AI architecture. A neural network is employed for many pattern recognition applications; however, its most popular use is the creation of language models used by ChatGPT, Gemini and other chatbots. Loosely based on the human nervous system, a neural network is technically an "artificial" neural network (ANN).
Layers and Nodes
Neural networks comprise one input and one output layer and any number of hidden layers in between (see below). Each layer contains a number of nodes, which are mathematical units computed with all the nodes in the next layer within the same machine or another machine. Datacenters can have thousands of servers, each containing a small number of nodes. The more servers, the larger the network and the quicker computations can be made between the layers. For smaller AI applications, a single desktop machine can also contain a neural network; for example, see DGX Spark.
Nodes, Servers and GPUs
A server has been called a "node" forever. Now we have neural network nodes; therefore, each server node processes some number of neural network nodes. There are generally four to eight GPUs in a server and up to 32 in a cluster. Today, huge language models are trained using 10 to 30,000 GPUs. It is estimated that 100,000 and more GPUs will be required to train a model in days instead of weeks and months. See GPU.
A Neuron in a Neural Network
There Are Many Network Designs
The following diagrams from the Asimov Institute in the Netherlands reveal the variety of neural network architectures that have been created. For a neural network example that recognizes the letters of the alphabet, which is easier to grasp than language recognition, see convolutional neural network.
Neural Network Architectures
Layers and Nodes
Neural networks comprise one input and one output layer and any number of hidden layers in between (see below). Each layer contains a number of nodes, which are mathematical units computed with all the nodes in the next layer within the same machine or another machine. Datacenters can have thousands of servers, each containing a small number of nodes. The more servers, the larger the network and the quicker computations can be made between the layers. For smaller AI applications, a single desktop machine can also contain a neural network; for example, see DGX Spark.
Nodes, Servers and GPUs
A server has been called a "node" forever. Now we have neural network nodes; therefore, each server node processes some number of neural network nodes. There are generally four to eight GPUs in a server and up to 32 in a cluster. Today, huge language models are trained using 10 to 30,000 GPUs. It is estimated that 100,000 and more GPUs will be required to train a model in days instead of weeks and months. See GPU.
Nothing like "if-then-else" business logic, neural networks cannot be debugged or reverse engineered like regular computer programs. This is a single neuron. See AI weights and biases.
There Are Many Network Designs
The following diagrams from the Asimov Institute in the Netherlands reveal the variety of neural network architectures that have been created. For a neural network example that recognizes the letters of the alphabet, which is easier to grasp than language recognition, see convolutional neural network.
AI networks are one of the most researched areas of computing in the 21st century. The examples above from the Asimov Institute in the Netherlands reveal the variety of network architectures that have been created. (Images courtesy of Fjodor van Veen and Stefan Leijnen (2019). The Neural Network Zoo.)