For seven installments this series has measured what local models answer when you ask them. This one is about the question that comes before that, and which I had never asked: is the model named in the configuration the one that answers at all? It sounds too trivial to measure. Then I spent a single day checking what the services in my setup actually point at.
The result in one sentence: Four configurations pointing at the same model server named models that either do not exist there or are not what their name claims — and all four failed silently, without a single error message ever reaching a human.
The loudest of the four had been firing an average of more than 40,000 requests per day for eleven days, every one of them rejected — and it was only noticed because someone happened to be counting accesses to the model server for an entirely unrelated reason.
The counting rule, before the cases
As in every part of this series, the rule was fixed before anything was counted. A case makes the list only if three conditions hold at once:
- An actively used configuration line names a model tag that does not exist on the server — or that demonstrably is something other than what its name says.
- The service kept delivering answers regardless.
- No error ever reached a human.
Commented-out lines and documentation do not count. Two candidates were eliminated by exactly this rule: there, the dead model name only appeared in comments or in a Markdown how-to, while the active model existed. And one caveat up front, because numbers are about to appear: only one host was available for this installment. The byte-identical second measurement across two machines — the strongest form of evidence this series has — does not exist here. Every number comes from one machine and its journal.
Case 1: 43,000 rejections a day — a data collector
A data collector that gathers market data and has a model assess it was configured with the tag qwen3:32b, in five places across four files. What was installed was qwen3:8b. The server’s reply to every one of those requests, quoted verbatim and reproduced live:
{"error":"model 'qwen3:32b' not found"}
HTTP 404, done in two to six milliseconds. The model server’s journal still went back eight days and counts over 214,000 rejected requests for that window:
| Day | rejected requests |
|---|---|
| Jul 30 | 576 |
| Jul 31 | 30,211 |
| Aug 1 | 50,893 |
| Aug 2 | 49,324 |
| Aug 3 | 32,790 |
| Aug 4 | 43,406 |
| Aug 5 | 6,977 (service stopped during the day) |
| Aug 6 | 52 |
On Aug 4, 43,344 of the 43,406 rejections came from the machine the collector runs on, almost all of them against /api/generate. The attribution is unambiguous.
Why nobody noticed for eleven days is written in the code, in two places. First, the service’s availability check only verifies that the server answers the model list with HTTP 200 — it never compares its own configured tag against that list. The server was up, so the model counted as present. Second, the service catches the 404, writes a warning into its own log, and falls back to a keyword matcher that returns a hard-coded confidence=60. So the service kept producing assessments carrying a number that looks like a measurement — just, for eleven days, without a model.
The failure did not even load the machine: a rejection in two to six milliseconds is cheaper than any real inference. Anyone watching load or latency saw those eleven days as an improvement.
Case 2: four models declared, one exists — a coding agent
A coding agent’s configuration declared four models. Checked against the installed tags:
| declared | finding |
|---|---|
qwen3-coder:latest | 404 — does not exist |
huihui_ai/Qwen3.6-abliterated:27b | 404 — installed is :27b-q4_K |
qwen3:14b | 404 — installed is qwen3:8b |
devstral:24b | exists |
Three of four dead — including the default, the model that is selected on startup. The second row is the instructive one, because it confirms a rule from this series’ ground rules in a new guise: never trust a model name without checking the variant. :27b and :27b-q4_K look like the same model spelled two ways. To the server they are two different tags, and one of them does not exist.
Case 3: two names, one blob
An application with separately configured text and vision models pointed its text-model variable at huihui_ai/qwen3-coder-abliterated:30b-a3b-instruct-q4_K_M. That tag actually exists — under the counting rule, this case almost slipped through. But the tag and the vision model huihui_ai/qwen3-vl-abliterated:30b-a3b-instruct-q4_K_M carry the same checksum: 1641bad23b0cb9cb. Two names, one model.
The counter-check, to rule out a measurement artifact: the same publisher’s thinking variant carries a different checksum (e89241bfa4109e24) — identical checksums do not appear here by accident. And the server’s model inspection says this about the “coder” tag: capabilities completion, vision, tools; a mixture-of-experts vision model with 31 billion parameters. This application’s coder model is a vision model wearing the wrong label. There was no coder model anywhere in the inventory.
The telling detail: a watchdog process whose job is to keep the “right” model pinned in GPU memory had re-pinned 119 times since Jul 17, and on Jul 27, within 33 minutes, switched the pinned name from the coder tag to the vision tag. What changed about the loaded model: nothing. The watchdog was guarding a label.
Case 4: the model that never saw an image
The coding agent from case 2 has a second, quieter defect. Its configuration never declares which input types a model understands (modalities). The consequence, verified on the live system: attach an image to a prompt, and the agent silently replaces the image with an error text and sends that text to the model. The model never sees a pixel. It answers anyway — fluent, grammatical, with no indication whatsoever that it is responding to a placeholder string instead of a picture.
Of the four, this is the most dangerous variant, because here not even a 404 lands in the journal. Request valid, response valid, content — as far as the image that never arrived is concerned — invented.
Why all four are silent
Four configurations, four different services, four different mechanisms — and one shared structure: the client gets an answer every time. A fallback with a fixed score (case 1), an instant error nobody reads (case 2), a wrong but functioning model (case 3), an error text standing in for an image (case 4). An HTTP API has no concept of silence. The difference between “the configured model answered” and “something answered” is not carried by any status code — the client has to check it itself, and none of the four did.
That is why I consider this a failure class rather than four mishaps. A dead model name produces no symptom: no load, no latency, no error page. It only produces answers that are something other than what they claim to be. The label gets read once, when the configuration is written, and is never again compared against the inventory — and every inventory changes.
The price of the countermeasure
The check that would have caught all four cases is one line: hold the output of ollama list against every configured tag. It costs under a second and zero GPU time. On the other side of the ledger, case 1 alone put over 214,000 failed requests into eight days, and a service spent eleven days delivering invented scores. That is the cheapest cost-benefit ratio this series has seen so far.
Two additions for the cases a name comparison cannot catch: for case 3 you additionally have to compare checksums — two tags with the same checksum are one model, whatever the names say. For case 4 the only remedy is this series’ ground rule: send one image through and read the answer, not the score.
My own mistake
It arrived promptly. The same week, my own nightly test run wrote itself onto this list. The run took 21 seconds instead of hours, reported 17 out of 17 models as “skipped (not installed)”, and exited three times with return code 0. The chain behind it: the command-line tool crashes when the HOME environment variable is missing; the container execution passed through an empty environment; a 2>/dev/null in my script swallowed exactly that crash message; and an empty model list was read as “nothing installed” instead of as an error.
The fix is threefold: HOME is now set, stderr is no longer discarded, and an empty model list aborts the run hard instead of letting it complete empty. But the point is a different one: I have written repeatedly in this series that a suspiciously round zero is suspect and that you read outputs, not scores. 17 out of 17 “not installed” in 21 seconds is a very round zero, and my script signed it off with return code 0. The failure class in this post does not distinguish between other people’s services and one’s own test rigs.
What this does not show
- Not that there are only four cases. A full-text search for model names across the rest of the configuration landscape returned over 3,000 matching lines; two large chunks of that have not been worked through. Four is the proven lower bound, not a tally.
- Not that the model server does anything wrong. Ollama answers an unknown tag correctly and quickly with a 404. The silence originates entirely on the client side.
- Not that fallbacks are wrong. A fallback that announces itself as a fallback would have been the correct behavior in case 1. What is wrong is a fallback that presents itself as a measurement.
- Not that it looks like this everywhere. Four cases against one model server with 24 tags and a handful of consumers — that says nothing about other environments. It says the check costs one line and would have paid off four times here.
The setup
- One model server: Ollama 0.32.5, two RTX 3090s, 24 installed tags at measurement time.
- The failed-request numbers come from the model server’s journal, counted per day; the quoted 404 response was reproduced with a direct API call, not inferred from logs.
- Checksums and per-tag capabilities come from the server’s model inspection, not from file names.
- Only one host available. This series prefers to back claims with byte-identical outputs across two machines; no second machine with the same model inventory was available for this installment. The numbers are therefore singly evidenced, not doubly.
What gets measured next: what abliteration actually takes away from a model — a test rig with scenarios fixed in advance, on the same cards.