Inference Optimisation

Speculative Decoding

Use a small draft model to propose tokens that a large verifier accepts or rejects in parallel, giving lossless 2-3x latency wins on autoregressive generation.

advanced · 9 min read · Premium