A benchmark harness for a single-GPU vLLM server. One server, three configs, the same 24 prompts across code editing, creative writing and RAG-style answering, at temperature 0 with fixed seeds.
Determinism is checked rather than assumed: two identical baseline runs have to produce byte-identical output before a number counts. That is how FP8 KV cache ended up staying off on my server despite the throughput win.
Current headline numbers live on the benchmarks page: 350 tok/s on code edits with n-gram speculation (4.65× baseline) and 656 aggregate tok/s at 16-way concurrency.
