allocateGPU VRAM calculator
How much memory does that model actually need? Pick a model, quantization, and context length; see what fits, instantly and free.
Smallest single device that fits: L40S (48 GB)
How it works
Common questions
How is the VRAM requirement calculated?
Weights (parameters times bytes per parameter at your chosen quantization) plus the KV cache (which grows linearly with context length), plus roughly 10% runtime overhead. It matches what serving stacks like vLLM report in practice within a few percent.
Why do MoE models need so much memory when few parameters are active?
Active parameters set compute speed, not memory. All experts must be resident, so a 1.6T-parameter MoE needs the full 1.6T parameters' worth of memory even though only about 49B are active per token.
Does 4-bit quantization hurt quality?
Modern 4-bit methods cost a small, usually acceptable amount of quality for a 4x memory saving. 8-bit is near-lossless. For production judgment tasks, test on your own evaluations before committing.
What if the model doesn't fit any GPU I have?
You can shard across multiple GPUs (with overhead), pick a smaller model, or run it on managed inference. Allocate serves every model on this list token-metered, so idle hardware costs you nothing.
Is this calculator free?
Yes, free and unlimited, no account needed. It runs entirely in your browser.
More free tools
If it doesn't fit your hardware, it fits ours. Allocate runs every open-weight model here, token-metered, inside your own boundary; no GPUs to buy, no idle burn.