What $2,000 and $40,000 Actually Buy in Local LLM Hardware

James O’Beirne, a developer, published a GitHub guide called local-llm that lists the exact hardware he buys to run state of the art language models on his own machines instead of through a hosted API. The guide carries no publish date, but internal references place it around July 2026. It turns an abstract debate over local versus hosted inference into concrete dollar figures.

At the low end, roughly $2,000 buys two Nvidia RTX 3090 cards for 48GB of combined VRAM. That is enough to run Qwen3.6-27B, which O’Beirne calls an excellent model, plus a full speech to text stack built on OpenAI’s whisper-large-v3, running on a single GPU with about 11GB of VRAM to spare. Local speech to text removes the need to send audio to a hosted transcription API at all.

At the high end, about $40,000 buys four Nvidia RTX 6000 Pro Blackwell workstation cards for 384GB of combined VRAM, which O’Beirne describes as producing something close to Claude Opus. His current pick for that rig, as of July 2026, is GLM-5.2-Int8Mix-NVFP4-REAP-594B, served through a vLLM docker-compose setup that he clocks at roughly 80 tokens per second at a 460,000 token context window.

The bill of materials splits that $40,000 figure into two very different line items. The base system, an older EPYC server built mostly from eBay parts plus a PCIe switch from c-payne.com for direct GPU to GPU communication, comes to $5,587. The four GPUs alone cost roughly $46,000, a sum O’Beirne says is the right place to spend money once memory prices make new motherboards prohibitively expensive.

O’Beirne does not just serve the models and stop. He points a coding agent called opencode at the served endpoint from a separate virtual machine, then lets it work through a sandboxed session with its own web search, alerting, and source control tooling. That detail is the real payoff for builders: the same $40,000 rig that approximates Opus can also host an autonomous coding agent with no per-token bill and no data leaving the local network.

The split matters for any team sizing up local inference against a metered API. A GPU rig purchased once keeps costing the same regardless of token volume, and it keeps working even if a frontier lab changes its pricing tiers or retires a model entirely. It also keeps sensitive audio and text off a third party’s servers, which is the reason O’Beirne gives for preferring local speech to text over a hosted equivalent.

The comparison to Opus is O’Beirne’s own characterization, not an independent benchmark result, and the guide publishes no head to head scores against any hosted frontier model. Anyone pricing out the $40,000 tier should treat close to Opus as one builder’s informal read of a self-hosted stack, not a verified claim.

Teams weighing a move off a hosted API should first total their monthly token spend and set it against these two tiers. The math shifts fast once volume clears a break-even point, and O’Beirne’s parts list, laid out in his local-llm guide on GitHub, gives a real starting budget rather than a vendor’s marketing estimate.

Sourced from James O’Beirne’s local-llm guide on GitHub, published in 2026.