Two RTX 3090s, eleven models with identical digests, 528 requests and no errors. Reserving a 128k KV cache is nearly free; filling it costs 10 to 66 percent of throughput — with nothing offloaded to the CPU on ten of eleven models. Plus: an attached monitor costs you 1.8 GB of usable VRAM.
Ollama 0.32.5 was out, my workstation was still on 0.31.2, and I had just finished benchmarking 26 models on it. So I had a rare opportunity: the same…
I have an RTX 3090 sitting in a Windows workstation that is about to be torn down and rebuilt as a Proxmox AI host. Before that happens, I…
Venice.ai runs open-source models with zero-knowledge privacy. No prompt logging, no training on your data. The VVV token staking model gives you permanent AI access without monthly fees. Here is how it works and where it fits next to self-hosted Ollama.