Model Optimization

Quantization

Trading numerical precision for memory and bandwidth — the highest-leverage single technique in inference engineering.

Pruning and Distillation

Making a model smaller by removing what it doesn't need, or by teaching a smaller model to imitate a larger one.

Speculative Decoding

Using a small, fast model to guess ahead and a large model to verify in parallel — getting large-model quality at closer to small-model speed.