# What is speculative decoding?

Speculative decoding speeds up generation by letting a small draft model propose several tokens ahead, which the large model then verifies in a single forward pass. Accepted tokens arrive in batches instead of one by one; rejected ones are recomputed, so the output is identical to the large model alone.

It works because verifying tokens in parallel is much cheaper than generating them serially, and because on predictable text (code, structured output, boilerplate) the draft model guesses right most of the time.

The gain depends on the workload: repetitive, formatted output accelerates the most; creative free-form prose the least. Providers apply it server-side, which is one reason identical models differ in speed across platforms.

## Related terms

- [Inference latency](https://allocate.network/glossary/latency.md)
- [Model distillation](https://allocate.network/glossary/model-distillation.md)
- [Tokens per second](https://allocate.network/glossary/tokens-per-second.md)

---

[HTML page](https://allocate.network/glossary/speculative-decoding) · [Machine-readable catalog](https://allocate.network/catalog.json)