AI Term of the Moment

accelerated computing


Look Up Another Term


Definition: weights and biases


A neural network is a "math machine" that learns from examples. It comprises multiple layers of computational units called "nodes" or "neurons" that are mathematically connected to each other with numerical values called weights and biases, collectively known as "parameters."

When a neural network is programmed, a table of weights known as a "weight matrix" is created and generally populated with random values. However, the weights are updated in the training passes.

Neural networks are developed to create models for language and vision recognition as well as chatbots, all of which are major applications of AI. Chatbot "language models" have the largest number of neurons, sometimes hundreds of billions and more, because the larger their vocabulary, the more questions they can answer and the greater their accuracy and reasoning capacity. See large language model, neural network and hyperparameters.




Weights and Biases
Neurons are computational units, not fixed hardware. Weights control the strength of the connections, and the bias adds a fixed value to the neuron. The activation function is a formula that uses weights and biases to compute the strength of the neuron's output, which goes to every neuron in the next layer. Randomly set at the start, weights are constantly adjusted in the training phase. Bias values initially set to zero are adjusted to reduce errors. See backpropagation.






More Parameters than Neurons
There are orders of magnitude more parameters than nodes/neurons. For example, two adjacent layers each with a thousand neurons have a million (1,000 x 1,000) weights plus 1,000 biases. This example has 65,000 neurons between input and output layers for more than four billion connections/parameters. In 2026, xAI's Grok 5 model is said to have at least six trillion parameters (see Grok AI). (Example generated by ChatGPT.)