(
GPT-
GUType
File) An AI model file format that is compressed by quantization. Although GPT is in the name, the GGUF file is not tied to a GPT model. It is a container format that along with the compressed weights describes the number of vocabulary tokens, context window length and runtime parameters. GGUF also describes the file structure and compression method, which is actual model quantization rather than common ZIP file compression. For example, the conversion might be changing 32-bit numbers into 4-bit and 8-bit numbers (see
AI quantization).