Reserving 128k Context Costs Nothing. Filling It Costs Two Thirds.
Two RTX 3090s, eleven models with identical digests, 528 requests and no errors. Reserving a 128k KV cache is nearly free; filling it costs 10 to…
Upgrading Ollama Made Nothing Faster — But It Broke One Model
Ollama 0.32.5 was out, my workstation was still on 0.31.2, and I had just finished benchmarking 26 models on it. So I had a rare opportunity:…
I Benchmarked 26 Local LLMs on One RTX 3090 — Here Is What Actually Held Up
I have an RTX 3090 sitting in a Windows workstation that is about to be torn down and rebuilt as a Proxmox AI host. Before that…