Redirected from: inference vs. training

Definition: AI training vs. inference

A machine learning (ML) model, which is the foundation of chatbots such as ChatGPT and Gemini, requires a training phase. After the AI neural network architecture is designed and programmed, the training phase analyzes the patterns in the data samples. The training stages can take days, weeks and months in datacenters with tens to hundreds of thousands of servers operating together. For example, GPT-4 took several months and was trained on trillions of words. See AI training and AI model.

Inference Processing
The AI application that people use (ChatGPT, Gemini, etc.), is called the "inference engine," a term that dates back to the first AI programs (see expert system). Once programmed, the inference engine is used to run many models for countless purposes. However, like everything in the computer field, the inference engine must also be programmed and fine-tuned (see below).

Running an inference engine to get a result does not take anywhere near as much computer time as the learning phases of model development. However, consider that millions of people may be using an AI chatbot simultaneously; thus, inference processing may add up to much more computer time in the long run than the training phases of the model it is using. See neural network and AI datacenter.


 REGULAR DATA PROCESSING DEVELOPMENT:

 1. design the logic
 2. code the logic
 3. test application
 4. run application

 --------------------------------------------

 AI NEURAL NETWORK DEVELOPMENT:


 Model Development

 1.  INITIAL DESIGN
 1a.  select network type (CNN, RNN, GAN, etc.)
 1b.  code the model
 1c.  set layers, neurons, passes (hyperparameters)
 1d.  software sets weights and biases (parameters)

 2.  PRE-TRAIN (with example datasets)
 2a.  hyperparameters mostly adjusted by people
 2b.  parameters adjusted by software

 3.  FINE-TUNE (with example datasets)
 3a.  hyperparameters mostly adjusted by people
 3b.  parameters adjusted by software
      See AI hyperparameter.


 Inference Engine

 1.  design
 2.  code
 3.  optimize (see AI quantization)


 Execute AI Application

 1.  run inference engine with built-in model
      or
 2.  run inference engine and select model

A Clear-cut Comparison

This clever comparison of machine learning programming vs. traditional programming comes from Techopedia's "The Ultimate Guide to Applying AI in Business."

misc

Term of the Moment

cellular generations

Look Up Another Term

Redirected from: inference vs. training

Definition: AI training vs. inference