Allocate1T MoE class GPU requirements
What it takes to run 1T MoE class (Kimi, DeepSeek) locally: memory by quantization, the smallest GPU that fits, and the managed alternative.
1T MoE class serves up to 262,144 tokens of context; the KV cache grows linearly toward that ceiling, so the slider below shows exactly what longer context costs in memory.
No single GPU fits this configuration. Shard across devices, quantize harder, or run it managed.
Or skip the hardware: run 1T MoE class on Allocate, token-metered.
How it works
Common questions
How much VRAM does 1T MoE class need?
At 8K context: roughly 2205 GB at FP16, 1102 GB at 8-bit, and 606 GB at 4-bit, including KV cache and runtime overhead. Longer context adds memory linearly.
Can a single GPU run 1T MoE class?
Not at practical quantizations: even 4-bit needs about 606 GB, beyond any single device here. Sharding across a fleet or managed serving are the options.
Why does 1T MoE class need so much memory as a MoE model?
Only some experts activate per token, which sets speed, but all 1.0 trillion parameters must sit in memory. MoE trades memory for throughput.
What license is 1T MoE class under?
The catalog does not list a license for this model. Check the lab’s model card for the exact terms before commercial deployment.
Is there an alternative to buying GPUs for 1T MoE class?
Yes: managed inference. Allocate serves comparable models token-metered inside your own boundary, and if you fine-tune an open base on your data, the weights belong to you.
More free tools
Allocate serves open-weight models like 1T MoE class token-metered inside your own boundary. No hardware to buy, and if you fine-tune on your data, the weights are yours.