Ladder free, budget not: three quants, four context stages, twelve runs
On October 10th I put one model and one question on a GB10 with 124.5 GB: how much does the context window cost, and is a…
Blog
Infrastructure, AI, tools, and the decisions behind them.
On October 10th I put one model and one question on a GB10 with 124.5 GB: how much does the context window cost, and is a…
A Nvidia GB10 has 124.5 GB of shared memory. I built one single model on it, measured where the context window ends - and counted Tetris.
Jev, TypeSafe’s hosted “decision model”, is having a moment. You hand it a piece of state and a few typed questions (yes/no, pick one, score on…
I wanted to split a dual-RTX-3090 box into “one model per card”: two independent 27B models instead of one model stretched across both GPUs at a…
The candidate passed. The pass rule was written down before the run, and it met that rule cleanly: 65 of 65 layers on the cards, gpu_pct=100.00.…
While I was writing this text, the search over my knowledge store reported a coverage of 100 percent. And it was right. It counted notes, and…
A few hours ago I published a throughput table: nine local models, tokens per second, cleanly measured. Its closing paragraph said speed is one dimension of…
Nine local models, three prompt lengths, reasoning on and off, all at a 262144-token context on two RTX 3090s — and the model that has been…
I gave a 120-billion-parameter mixture-of-experts model a second GPU. It got five times slower. Here is the measurement, and the more useful thing I found by…
I had two RTX 3090s and the compute of roughly one. This is the rebuild that put both cards on native PCIe in a single box…