Speculative Decoding
Using a small, fast model to guess ahead and a large model to verify in parallel — getting large-model quality at closer to small-model speed.
10 min readUpdated 2026-09-03ai-inferencespeculative-decodingmodel-optimization
Using a small, fast model to guess ahead and a large model to verify in parallel — getting large-model quality at closer to small-model speed.