Quantization — AI Glossary

Quantization is a technique that reduces an AI model's numerical precision — for example, converting 16-bit floating-point weights to 4-bit integers — to shrink the model's memory footprint and speed up inference. While this trades a small amount of accuracy for major efficiency gains, well-implemented quantization can reduce model size by 75% or more with minimal impact on output quality.