Blog - Page 2 of 4 - Tobias Liauw

Bar chart on a very dark background comparing two models on a structured-extraction task. The incumbent, a dense twenty-six-billion model, reaches recall 1.000 and precision 0.967. The challenger, which activates nine billion parameters per token, reaches recall 0.431 — less than half — and precision 0.517. A side panel lists what the scores hide: flattening failures 0 for the incumbent against 4 for the challenger, three identifiers the challenger invented that do not exist in the vocabulary, and the incumbent as the only model to finish all six documents with zero flattening failures.

Aug 13, 2026 Uncategorized

The challenger with the better argument lost by half

Twelve models, one structured-extraction task, and a challenger with the better argument on paper The companion piece to this one reports a zero: local models, driven…

Two panels. Left, the task that has no solution: five parcels weighing 3, 5, 8, 12 and 14 kilograms across two vans, each carrying at least two parcels, van A carrying exactly 6 kilograms more than B, and the 8 kilogram parcel travelling with B — the correct answer is that no solution exists. Right, what the largest model did: it invented a plausible-looking split in five runs out of six, and to make the arithmetic work it dropped the 14 kilogram parcel from the problem entirely.

Aug 8, 2026 AI & Tools

The Largest Model Invents a Solution Where There Is None

Yesterday this blog reported that the test set had hit its ceiling: one model solved all 48 cells, and about a model that solves everything nothing…

Two-field panel on a very dark ground. Left, the common advice: take the abliterated build and it will stop refusing you, with the untested part named — nobody checks whether the base would have refused at all. Right, the measurement: the non-abliterated base answered 34 of 34 image cells with not one refusal, and across all models 208 of 208 verdicts on the harmless tasks read allow. The strip below lists the price for the same work: base 14.8 and 15.4 minutes over the two nights, abliterated 27B build 34.8 and 74.1 minutes, six failures all on abliterated variants, and two calls that ran into the 1200-second timeout. Footer marks: 272 image cells, 8 vision models, 2 nights, one host, n = 2.

Aug 7, 2026 AI & Tools

The Abliteration Test Rig — What Unlocking Actually Removes, and What It Costs

Part 8 closed with a promise: next I would measure what abliteration actually takes away from a model. The advice itself is everywhere — take the…

Four task cards — a multi-stage price calculation, four talks into four slots, six fields into JSON, and a five-seat constraint puzzle — above a full-width bar filled completely and labelled 48 of 48. Result badges note that all 48 cells were correct, that the winning model was nine times faster and half the size, and a panel explains that a model solving everything tells you only that it did not fail.

Aug 7, 2026 AI & Tools

48 out of 48 — And Why That Is a Problem

Until today, every measurement in this series ran on a single graphics card. 24 gigabytes, and the boundary was clear: anything larger simply went unmeasured. As…

Four cards in a 2x2 grid, one per case on the same model server. Case 1, data collector: configured tag qwen3:32b, installed qwen3:8b, over 214,000 rejected requests in eight days, eleven days of a fallback returning a hard-coded confidence=60. Case 2, coding agent: four models declared, only one exists, three are dead including the default; :27b and :27b-q4_K are two different tags to the server. Case 3: coder tag and vision model share the checksum 1641bad23b0cb9cb, the coder tag is in fact a vision model. Case 4: modalities never declared, an attached image is replaced by an error text, no 404 in the journal, the content is invented. Below, the core statement that the client gets an answer every time, and a line noting that holding ollama list against every configured tag costs under a second.

Aug 7, 2026 AI & Tools

What the Label Says — Four Configurations Pointing at Models That Don’t Exist

For seven installments this series has measured what local models answer when you ask them. This one is about the question that comes before that, and…

Bar chart of answer-field length for three models, two bars each. deepseek-r1:14b 2069 characters against 31, qwen3:14b 1870 against 31, qwen3:8b 1527 against 31; the second bar is a sliver in every pair. Side panel: the reasoning sits in the separate thinking field the API returns, on 3 of 15 models that field stays empty, qwq:32b drops from 5060 to 3948 characters, and all 15 models solve the task correctly.

Aug 6, 2026 AI & Tools

The Think Switch Decides Where the Reasoning Goes, Not Whether

Models that “can think” have a switch for it in Ollama. This series has had it off from the start and said so in every post:…

Split dark panel. Left, under 'What I had written down': 65 of 176 groups identical, 111 groups deviate, the run log only reported STREUT, three seeds were meant to catch it. Right, under 'What the data actually held': 0 groups in which all three outputs differ, in 111 of 111 cases the odd one out is seed 42, seeds 43 and 44 never part, the outlier is always the first pass. A strip below covers the two-machine checks: five different seeds on six models yield the same text every time, 108 cell pairs across two machines show not one difference, token counts 503, 508, 508, 508, 508. Tags: temperature 0, Ollama 0.32.5, RTX 3090 on two machines, 528 cells, 11 models, byte-wise comparison.

Aug 5, 2026 AI & Tools

The Seed Does Nothing. The First Run Does

Every measurement in this series runs at temperature 0 with three seeds. The reasoning was simple: if the model does drift, three passes will catch it.…

Two panels side by side. Left, the migration tool's own output: "migration finished successfully", rc=0, a transfer of 42 minutes at a steady 117 MB/s, and not a single log entry. Right, the state at the destination: 1,263 files silently corrupted, 77,899 files missing entirely, roughly 20 percent of the files gone, and 2 of 19 Docker containers failing to start. A strip underneath explains why none of it showed up: the damaged files carry the same size and the same mtime as the originals, so rsync's default heuristic skips them, and a scan for zeroed first blocks found only 579 of the 1,263 damaged files.

Aug 4, 2026 Infrastructure

From Workstation to Hypervisor — and What the Migration Actually Cost

A 2020 desktop build turns into a Proxmox node with 48 GB of VRAM. The interesting part is the move itself: a guest migration that reported…

Grouped bar chart for three of eleven models, two bars each: pass rate in the unklar case and overall rate. gemma4:26b 100 and 100 per cent, ornith:35b 50 and 91 per cent, glm-4.7-flash 25 and 87 per cent. The overall bars stay close together while the unklar bars collapse. Side card: direkt passed 31 of 33, unklar passed 13 of 33, five models invent the hostname, identical on both machines. Footer line: three of eleven pass, the same three on both machines.

Aug 4, 2026 AI & Tools

All Eleven Models Call Tools. Three Know When Not To

An agent is not a chat. The question is not whether the model sounds clever, but whether it reaches for the right tool — and whether…

Grouped bar chart of fully solved cells out of 33 per task, bare prompt against the step-by-step instruction. rechnung rises from 19 to 24, extrakt from 15 to 25, while plan falls from 32 to 25. A side panel lists the cost of the step-by-step variant: median 10.2 seconds against 6.9 (+49 per cent), 769 against 707 tokens (+9 per cent), 11 truncated cells against 3 for the bare prompt, format compliance 115 of 132.

Aug 3, 2026 AI & Tools

Think Step by Step? On One Task in Four It Makes Things Worse

Three pieces of advice appear in every prompting guide: tell the model to think step by step. Give it a role. Show it an example. All…