I have an RTX 3090 sitting in a Windows workstation that is about to be torn down and rebuilt as a Proxmox AI host. Before that happens, I wanted one honest number for every model I keep on it — not a vibes check, not a leaderboard screenshot, but a measurement I could repeat later on different hardware and actually compare.
So I let 26 models run overnight. Roughly 600 requests, zero errors, followed by a 5.3-hour soak test. Here is what came out, including the parts where my own method was wrong.
The rules I set myself
Every task had to have an answer a script could check. No “this response feels better”. Eleven tasks per model:
- Five reasoning traps — the classic ones. Three machines making three widgets in three minutes; bat and ball costing 1.10 with the bat a dollar more. Scored on the last number in the reply.
- Three coding tasks — palindrome, Fibonacci, merge overlapping intervals. The model’s code is extracted, written to disk and actually executed by Python against an assert block. It passes or it does not.
- One JSON extraction — parsed, every field value-checked.
- Two vision tasks for the image models — on images I generated myself, so the ground truth is unambiguous rather than “whatever I happened to screenshot”.
Temperature 0, fixed seed, so the whole thing repeats.
Thinking is not cosmetic
Every reasoning task ran twice — once with the model’s chain of thought disabled, once with it on. Across all 19 text models the hit rate went from 74/95 to 88/95. Individual models moved dramatically: one went from 1/5 to 5/5.
The cost varies enormously. qwen3-coder reaches full marks using about 65 thinking tokens. qwen3.5:27b burns 2,435 and still only manages 3/5 — the worst ratio in the field, and the reason it got deleted.
One task is the exception that makes the rule useful: the three-machines problem gained nothing from thinking. 18/22 either way. A model that misreads that trap misreads it just as confidently with a chain of thought — it simply explains the wrong answer at greater length.
24 GB holds 64k context — for almost everything
I swept eight models across 4k, 16k, 32k and 64k context and watched what Ollama reports for VRAM versus total size. The moment part of a model lands on the CPU, throughput collapses.
Seven of eight held 64k entirely in VRAM — including the 35B models sitting at around 20 GB — and lost only 11 % to 39 % throughput doing it. One did not: qwq:32b offloaded 11 % to CPU and fell to 2.9 tok/s, a 91 % collapse.
That looked like a clean VRAM story until I checked the model metadata days later. qwq:32b has an architectural ceiling of 40,960 tokens. I had been testing it at 32k and 64k — at and beyond its own limit. The collapse may be as much about exceeding spec as about memory pressure, and I no longer trust my original reading of it.
Size tells you almost nothing about speed
ornith:35b runs at 116 tok/s. qwen3.6:27b manages 42 — and it is the smaller file. The difference is architecture: mixture-of-experts models activate only a few billion parameters per token, dense models activate all of them, and decode speed on this card is bound by memory bandwidth.
There is a hard ceiling worth remembering. A dense 35B model at Q4 cannot exceed roughly 47 tok/s on a 3090 — 936 GB/s divided by 19.7 GB. So any three-digit throughput figure on a “35B” model means few active parameters, never simply “good model”. If you compare two models on speed without checking that, you are comparing architectures and calling it quality.
5.3 hours of sustained load
53 rounds cycling the five largest models. Throughput drift across all five: ±1.3 %. Thermal throttling occurred in 2 of 7,432 samples — 0.03 %. The card holds up.
The more useful finding is a warning about benchmarking hardware for AI at all. Same card, same 350 W, two different loads:
| Sensor | LLM inference | 3DMark Steel Nomad |
|---|---|---|
| Core | 78.6 °C | 77.7 °C |
| Hot spot | 93.7 °C | 91.8 °C |
| Memory junction | 104.0 °C | 92.0 °C |
Core and hot spot are nearly identical. The VRAM runs 12 °C hotter under LLM load, because inference is memory-bandwidth bound and hammers the memory chips continuously in a way a graphics benchmark does not. The graphics test showed 18 °C of headroom to the 110 °C throttle. The real workload showed six.
3DMark cannot qualify an inference host. It will tell you everything is fine right up until it is not.
Where I got it wrong
Two models scored badly and I initially wrote them off. Then I looked at the raw per-request data instead of the summary.
qwq:32b scored 1/5 on direct reasoning and 0/3 on code. That reads as “cannot program”. It is not. All four failed reasoning attempts ended at exactly the token limit, and the coding failures all looked like this in the Python interpreter:
File "_pruef_C1.py", line 1
Okay, I need to write a Python function called is_palindrome...
^^^^^ SyntaxErrorThe model ignores the request to disable thinking and emits its reasoning as ordinary text until the budget runs out. It never produces a code block. That is a completely different defect from “writes bad algorithms”, and it changes what you do about it.
The lesson generalises: failure has three outcomes, not two. Solved, answered wrong, and never finished. Merge the last two and you will publish confident nonsense. Nine of my cells turned out to be budget exhaustion rather than inability.
A third case was purely my fault. One model scored 0/3 on JSON. It had produced perfectly correct JSON — and then repeated it endlessly. My regular expression was greedy and spanned all the repetitions, so the parse failed. The model was fine; my parser was not.
What I actually kept
| Role | Model | Why |
|---|---|---|
| All-rounder | gemma4:26b | Full marks everywhere at 96 tok/s, and the flattest falloff to 64k context in the field |
| Code and tools | qwen3-coder | 154 tok/s, everything solved, ~65 thinking tokens of overhead |
| Fast everyday | qwen3:8b | 120 tok/s at 9 GB — two fit on one card |
| Hard reasoning | ornith:35b | Full marks in both modes at 116 tok/s, no context spill |
| Vision | qwen2.5vl:7b | Both image tasks solved at 9.2 GB |
| Retired | qwq:32b, llama3.2:1b, qwen3.5:27b | See above — and qwen3.5 is beaten by qwen3.6 at identical size |
The part worth stealing
If you take one thing from this, take the method rather than the numbers — my numbers are one card and one set of models.
- Execute the code. Reading a model’s output and judging it plausible is not a measurement.
- Log the raw per-request data, not just a summary. Every correction in this post came from going back to it. A summary is a projection under one metric definition; change the metric and the summary is worthless while the raw data survives.
- Separate “wrong” from “never finished”. This single distinction was worth more than any other design decision.
- Check whether disabling thinking actually works for each model. It costs two requests: send the same prompt both ways and compare the token counts. On some models the flag does nothing at all, and you will be measuring an axis that does not exist.
Next: the same twelve models on a newer Ollama, and then the same twelve on a second RTX 3090 attached over Thunderbolt to a Linux box — same card, entirely different stack.