Card: „One PC. One AI team. Zero sub.“
Put differently: one work machine becomes an office. A locally running language model is the brain, an agent harness is the workforce, skills and notes are the company knowledge. This article explains the „one AI coworker“ model on our example, shows the five parts, and hands over the 60-minute path for your own hardware.
1. What one AI coworker is
A template that has been circulating recently casts three roles: a big model as the advisor (reads huge contexts, steers the plan), a mid-size one as session lead (executes edits, runs tests), small ones as the swarm (scan repos, pull docs). We build the same thing with local models: flash-next (SC117, 262k context) as advisor, 27b as lead, nimble plus a 7b on an old RTX 2080 Ti as the swarm. Three brains, three machines, zero cloud.
An AI coworker is not an app, it is a setup of five parts:
- Brain — a language model (template: cloud API; at our shop: a model on your own card).
- Harness — an agent harness that loops thinking, acting and observing (template: Claude Code; at our shop: pi).
- Job description — one file saying what the coworker may do, how it reports, what it is measured against (ours: AGENTS.md).
- Expertise — skills and procedures it loads per task (template: playbooks; our shop: SKILL.md folders).
- Continuity — a notebook (vault) and a task board so work carries from session to session.
Assemble the five and the coworker „wakes up“: reads its job description, sees what is due, works, and leaves notes. That exact coworker runs at ours — several of them, sequential and parallel, for 0 € in API costs.
2. Our brain: the GB10
The hardware is an NVIDIA DGX Spark (GB10): a workstation system with 124.5 GiB of shared memory and 175 GB/s of memory bandwidth. The special thing is not the size, it is the shared address: model, context and KV cache live in one pool neither CPU nor GPU has to shuttle across.
On top runs our production model SC117 (27B class) in a llama.cpp server with 262,144 tokens of context. Our benchmarks (13 documented runs, published at bench.tobiasliauw.de) show:
| Model (GGUF) | Decode @ 32k | Role |
|---|---|---|
| HuiHui (27B) | ≈ 33.6 tok/s | fastest candidate |
| SC117 (27B) | ≈ 26.3 tok/s | production model |
| Q4Mix | ≈ 24.3 tok/s | compact candidate |
26–34 tok/s in everyday terms: a coworker that thinks two A4 pages per minute — fast enough to think, slow enough to enjoy. A cloud model is stronger; it also knows what you are thinking about.
3. The five parts, concretely
Brain: llama-server (llama.cpp, 27B GGUF, CUDA) on port 8080. Operating rule: the production model is never taken out of service; tests run inside a window with automatic restore.
Harness (pi): the agent reads files, runs bash, edits code, writes new files. Every session gets an AGENTS.md with hard rules (e.g. „never compress flowing text“, „board: read → edit → write → re-read“).
Job description (AGENTS.md): examples from ours —
- „Bench runs live on the GB10 in
~/gb10-bench-tetris/runs/— not on Atlas.“ - „Stop only inside the operating window; restore via
restore.sh; hard limit 22:00 CEST.“ - „Vault index always bge-m3 (1024 dims).“
Expertise (skills): e.g. a second-brain skill that auto-summarizes 27 folders, or a design-skill kit for every frontend redesign.
Continuity (board + vault): our task board is one markdown file on the NAS. One session shrank 499 kilobytes of confusion to 243 kilobytes of truth (63 duplicates removed, 31 stale entries archived), checksum verified. That was not a backup — that was decluttering.
4. 60 minutes: your coworker
Prerequisites: a machine with ≥ 64 GB of shared memory (ideal 128 GB, e.g. GB10 or Strix Halo) or a GPU with ≥ 24 GB VRAM. Linux. Sixty minutes.
- Start the brain: Ollama or llama.cpp plus a 27B Q4/Q6 GGUF model. Goal:
curl http://localhost:8080/v1/modelsanswers. - Start the harness: point
pi(or any comparable agent) at the local OpenAI-compatible API. - Job description: write an
AGENTS.mdwith hard rules (what you may do, what never, where files live, how you report). - One expertise: a single skill with a
SKILL.md(e.g. „daily dashboard“). - Continuity: a
board.mdand a~/vault/folder. In the AGENTS.md: „At session end: update the board.“ - The order: „Read your board, find the open task, do it, write the report, update the board.“
That is the coworker. Everything beyond (more coworkers, more expertise, own models, benchmark suites) is what we built over the last weeks.
5. Honest limits
- Speed/quality: 27B-local << frontier cloud. For ops work, code, research drafts: fully enough. For top creation: not yet.
- Three brains, three machines — one spearhead: the tri-pattern (advisor/lead/swarm) is covered locally. The quality peak stays frontier and runs at ours as a frontliner for top creation, hard planning and review; routing rules decide what stays local and what may go out. What it replaces is their baseline load and data exfiltration.
- Ops: board, operating rules and restore are half the work. An AI coworker without an operating charter is an intern without an ID badge.
Card: „Not an AI that answers — a coworker that works.“
Part 2: Private
(Teaser, next week): why sensitive files are the real product promise of a local team; the data-flow map with it (what stays in, what never leaves), and the honest duel against Ollama + Open-WebUI.