AllocateGPU VRAM calculator
Work out how much memory a model needs to run. Pick the model, quantization, and context length; see which GPUs fit it.
Smallest single device that fits: L40S (48 GB)
How it works
Common questions
How is the VRAM requirement calculated?
Weights (parameters times bytes per parameter at your chosen quantization) plus the KV cache (which grows linearly with context length), plus roughly 10% runtime overhead. It matches what production serving stacks report in practice within a few percent.
Why do MoE models need so much memory when few parameters are active?
Active parameters set compute speed, not memory. All experts must be resident, so a 400B-parameter MoE needs the full 400B parameters' worth of memory even though only about 17B are active per token.
Does 4-bit quantization hurt quality?
Modern 4-bit methods cost a small, usually acceptable amount of quality for a 4x memory saving. 8-bit is near-lossless. For production judgment tasks, test on your own evaluations before committing.
What if the model doesn’t fit any GPU I have?
You can shard across multiple GPUs (with overhead), pick a smaller model, or run it on managed inference. Allocate serves every model on this list token-metered, so idle hardware costs you nothing.
Is this calculator free?
Yes, free and unlimited, no account needed. It runs entirely in your browser.
More free tools
Allocate serves every open-weight model here, token-metered, inside your own boundary. No GPUs to buy, and nothing bills while it sits idle.