# What is model quantization?

Quantization compresses a model by storing its weights at lower numeric precision, such as 8-bit or 4-bit instead of 16-bit. It cuts memory use and inference cost by 2 to 4 times with small, measurable quality loss, letting the same hardware serve larger models or more traffic.

Production platforms quantize carefully and test against task-specific evaluations, because the acceptable quality loss for a support agent differs from a fraud reviewer.

Quantization is one of several serving optimizations, alongside batching, caching, and speculative decoding, that separate purpose-built inference infrastructure from raw GPU rental.

## See also

- [GPU VRAM calculator](https://allocate.network/tools/gpu-vram-calculator)

## Related terms

- [Inference](https://allocate.network/glossary/inference.md)
- [Mixture of experts](https://allocate.network/glossary/mixture-of-experts.md)
- [Inference latency](https://allocate.network/glossary/latency.md)
- [Quantization formats (GGUF, AWQ, FP8)](https://allocate.network/glossary/quantization-formats.md)
- [GPU memory bandwidth](https://allocate.network/glossary/gpu-memory-bandwidth.md)

---

[HTML page](https://allocate.network/glossary/quantization) · [Machine-readable catalog](https://allocate.network/catalog.json)