# What are GGUF, AWQ, and FP8?

GGUF, AWQ, and FP8 are common formats for storing quantized model weights. GGUF is a file format built for CPU and consumer-GPU inference; AWQ is an activation-aware 4-bit method that protects the weights that matter most; FP8 is an 8-bit floating-point format served natively by modern accelerators.

The format decides where a model can run. GGUF targets local runtimes on laptops and workstations; AWQ and FP8 target production GPU serving stacks. The same open model is often published in all three.

Memory math is the constant underneath: FP16 spends 2 bytes per parameter, 8-bit formats about 1, and 4-bit formats roughly half of that. A 70B model is about 140 GB at FP16 and around 39 GB at 4-bit before the KV cache.

Catalog entries with FP8 or FP4 in the name are served in that precision. Quality loss is small but real; validate on your own tasks before switching a production route.

## See also

- [GPU VRAM calculator](https://allocate.network/tools/gpu-vram-calculator)

## Related terms

- [Quantization](https://allocate.network/glossary/quantization.md)
- [GPU utilization](https://allocate.network/glossary/gpu-utilization.md)
- [KV cache](https://allocate.network/glossary/kv-cache.md)

---

[HTML page](https://allocate.network/glossary/quantization-formats) · [Machine-readable catalog](https://allocate.network/catalog.json)