← The bench · updated Oct 13, 2026

Local LLM speed on one RTX 5090.

4 open-weight models served with vLLM on a single RTX 5090 (32 GB): tokens per second, context limits, quant that fit and what broke. Every number links to a logged run.

rig: NVIDIA GeForce RTX 5090 32 GB · AMD Ryzen 9 9950X3D · 64 GB DDR5

Every model, one line each

ModelQuant, serverHeadlineRun
Krea 3 closed access build · 2736×1536 PNG · conversational edits, first result every step, no rerolls 40% Still changed after 'return to start'
Gemma 4 31B AWQ 4-bit · vLLM · RTX 5090, 32 GB · seed 7, temp 0.7 · Three.js r169 126 Trees placed, thinking on
Qwen3.8-27B QUASAR QAT NVFP4, 20.6 GB · vLLM 0.20.0 · RTX 5090, 32 GB 103,459 Tokens in one prompt
Qwen3-32B-AWQ vLLM 0.20.0 · RTX 5090, 32 GB · AMD Ryzen 9 9950X3D · 64 GB DDR5 350 Fastest decode, tok/s

Krea 3

Full run →

Every one of twelve chained edits landed first try and the sign stayed legible through all of them, but told to return to the start the model moved 0.3% of the pixels and stayed 40% off.

Krea 3 · closed access build · 2736×1536 PNG · conversational edits, first result every step, no rerolls

40%
Still changed after 'return to start'
pixels vs the original · the return edit itself moved 0.3%
12 / 12
Edits accepted first try
no rerolls, every output kept
12 / 12
Sign still legible
Teahouse of the Thousand Steps, even as a woodblock print
68%
Peak drift, the ukiyo-e step
the only edit that touched every pixel
0.3%
Pixels moved by the final undo
the biggest ask produced the smallest change
Step 4
Sea of clouds became mountains
and stayed mountains through eight more edits
15 min
Whole session, wall clock
13 prompts, including the typing
2736×1536
Every output
16:9 PNG, 6 to 10 MB each

Gemma 4 31B

Full run →

Thinking mode cost 3.8 times the tokens and 93 seconds, and was the difference between shards and an island.

Gemma 4 31B · AWQ 4-bit · vLLM · RTX 5090, 32 GB · seed 7, temp 0.7 · Three.js r169

2 / 2
Ran first try
0 console errors, both runs · 7 of 8 features each
8,136
Tokens, thinking on
vs 2,115 with thinking off (3.8×)
124 s
Wall time, thinking on
vs 31 s off · 67.3 and 65.4 tok/s, same server
126
Trees placed, thinking on
vs 21 · 50,410 vs 11,134 triangles

Qwen3.8-27B

Full run →

A 27B model read a whole novel in one prompt on 32 GB, after four fixes nobody documents, and still scored 96.8% on gsm8k.

Qwen3.8-27B · QUASAR QAT NVFP4, 20.6 GB · vLLM 0.20.0 · RTX 5090, 32 GB

103,459
Tokens in one prompt
all of Frankenstein
67.5 s
Prompt to answer
wall time, novel test
1,606
Prefill tok/s
measured, vLLM log
31.8
Decode tok/s
measured, vLLM log
96.8%
gsm8k, strict match
n=250, ±1.1%
8.74
wikitext word ppl
62 rolling 8K windows
75,264
Token KV pool
at 131,072 max context
0
Tokens from official FP8
does not load on 32 GB

Qwen3-32B-AWQ

Full run →

n-gram speculation made code edits 4.65 times faster with no draft model, and the determinism check kept FP8 KV cache off.

Qwen3-32B-AWQ · vLLM 0.20.0 · RTX 5090, 32 GB · AMD Ryzen 9 9950X3D · 64 GB DDR5

350
Fastest decode, tok/s
n-gram speculation on code edits · 75 baseline
4.65×
Code edit speedup
2.41× on RAG · 0.95× on creative writing
51.3%
Draft acceptance
2,635 of 5,140 drafted tokens accepted
0
Extra models loaded
drafts come from the prompt itself
24 / 24
Control, byte-identical
two baseline runs, same 24 prompts
18 / 24
n-gram, byte-identical
all six differences in creative writing
12 / 24
FP8 KV, byte-identical
why FP8 KV stays off on my server
656
Aggregate tok/s, peak
16 concurrent · FP8 KV cache

How to read this

Each run is one model on the same machine, with the server flags, the prompts and the log files in the write-up. The numbers are what the log said, not what the model card promised. Decode tok/s is single stream unless the label says aggregate. If a model is missing it has not been run yet; new runs land here automatically.