{"key":"local-builds","locale":"nl","title":"Pc-configuraties voor lokale AI-modellen, 20B tot 120B — 2026","url":"https://fedi.software/nl/ai/infographics/local-builds","fields":[{"title":"Configuratie","key":"title","format":"text","visible":true,"unit":null},{"title":"Budget","key":"tier","format":"badge","visible":true,"unit":null},{"title":"Videogeheugen","key":"vram","format":"gb","visible":true,"unit":null},{"title":"RAM","key":"ram","format":"gb","visible":true,"unit":null},{"title":"Prijs","key":"price","format":"usd","visible":true,"unit":null},{"title":"Onderdelen","key":"components","format":"list","visible":true,"unit":null},{"title":"Draait","key":"models","format":"tok_s","visible":true,"unit":null},{"title":"Opmerkingen","key":"notes","format":"text","visible":true,"unit":null}],"rows":[{"_id":"1x-16gb","title":"One 16 GB GPU","tier":"Budget","vram":16,"ram":32,"price":null,"components":["GPU: RTX 5060 Ti 16 GB / RTX 4060 Ti 16 GB","CPU: any 6–8-core desktop CPU","Moederbord: any ATX/mATX with a PCIe x16 slot","RAM: 32 GB DDR4/DDR5","Voeding: 550–650 W","PCIe: 1× x16 (x8 electrical is enough)"],"models":[{"id":"gpt-oss-20b","name":"gpt-oss-20b MXFP4 (RTX 5060 Ti 16 GB)","href":"https://fedi.software/nl/ai/compare?m=gpt-oss-20b","quant":"MXFP4","tok_s":111.5,"kind":"measured","source_url":"https://github.com/ggml-org/llama.cpp/discussions/15396","source_date":"2025-08"},{"id":"qwen3-30b-a3b","name":"Qwen3-30B-A3B Q4_K_M, experts partly in RAM","href":"https://fedi.software/nl/ai/compare?m=qwen3-30b-a3b","quant":"Q4_K_M","kind":"estimate"}],"notes":"gpt-oss-20b (≈12 GB) fits entirely in VRAM with room for a long context. Qwen3-30B-A3B Q4_K_M (≈18.6 GB) does not fit: a few expert layers go to system RAM with --n-cpu-moe, still usable because only ~3B parameters are active. Measured on an RTX 5060 Ti 16 GB (llama-bench tg128)."},{"_id":"1x-3090","title":"One RTX 3090 24 GB (used)","tier":"Budget","vram":24,"ram":64,"price":null,"components":["GPU: RTX 3090 24 GB","CPU: any 6–8-core desktop CPU","Moederbord: any ATX with a PCIe x16 slot","RAM: 32–64 GB DDR4/DDR5","Voeding: 750–850 W","PCIe: 1× x16"],"models":[{"id":"gpt-oss-20b","name":"gpt-oss-20b MXFP4","href":"https://fedi.software/nl/ai/compare?m=gpt-oss-20b","quant":"MXFP4","tok_s":161.8,"kind":"measured","source_url":"https://github.com/ggml-org/llama.cpp/discussions/15396","source_date":"2025-08"},{"id":"qwen3-30b-a3b","name":"Qwen3-30B-A3B Q4_K","href":"https://fedi.software/nl/ai/compare?m=qwen3-30b-a3b","quant":"Q4_K_M","tok_s":153.6,"ctx":"4k","kind":"measured","source_url":"https://www.hardware-corner.net/gpu-llm-benchmarks/rtx-3090/","source_date":"2026-03"},{"id":"gemma-3-27b-it","name":"Gemma 3 27B Q4_K_M","href":"https://fedi.software/nl/ai/compare?m=gemma-3-27b-it","quant":"Q4_K_M","tok_s":[31,42],"kind":"estimate"}],"notes":"The cheapest way to 24 GB of fast VRAM. Small MoE models fly; dense ~27–32B models fit at Q4 with a moderate context. The Gemma 3 27B figure is our estimate: 936 GB/s × 55–75 % efficiency ÷ 16.5 GB of weights."},{"_id":"hybrid-gpu-ram","title":"12–24 GB GPU + 64–128 GB RAM (MoE offload)","tier":"Budget","vram":24,"ram":64,"price":null,"components":["GPU: RTX 3090 24 GB / RTX 4070 12 GB / RTX 3080 Ti 12 GB","CPU: Core i5-12600K / Core Ultra 7 265K class","Moederbord: desktop board, both memory channels populated","RAM: 64–128 GB DDR5 (XMP/EXPO on) or DDR4-3600","Voeding: 750–850 W","PCIe: 1× x16"],"models":[{"id":"openai-gpt-oss-120b","name":"gpt-oss-120b MXFP4 — RTX 3090 + 64 GB DDR5-5200, --n-cpu-moe 24–26","href":"https://fedi.software/nl/ai/compare?m=openai-gpt-oss-120b","quant":"MXFP4","tok_s":[26.2,27.5],"kind":"measured","source_url":"https://github.com/ggml-org/llama.cpp/discussions/15396","source_date":"2026-01-20"},{"id":"openai-gpt-oss-120b","name":"gpt-oss-120b MXFP4 — RTX 4070 12 GB + 64 GB DDR5","href":"https://fedi.software/nl/ai/compare?m=openai-gpt-oss-120b","quant":"MXFP4","tok_s":[25,28],"kind":"measured","source_url":"https://carteakey.dev/blog/local-inference/optimizing-gpt-oss-120b-local-inference/","source_date":"2025-09-21"},{"id":"openai-gpt-oss-120b","name":"gpt-oss-120b MXFP4 — RTX 3080 Ti 12 GB + 128 GB DDR4-3600, --n-cpu-moe 36","href":"https://fedi.software/nl/ai/compare?m=openai-gpt-oss-120b","quant":"MXFP4","tok_s":[18,22],"kind":"measured","source_url":"https://github.crookster.org/running-gpt-oss-120b-on-rtx-3080-ti-12-gb-at-home/","source_date":"2025-08-25"}],"notes":"llama.cpp --n-cpu-moe N keeps attention and the shared layers on the GPU and moves the experts of N layers to system RAM. Works well only for MoE models; RAM speed matters — the 4070 build ran at 10–11 tok/s until XMP was turned on."},{"_id":"2x-3090","title":"Two RTX 3090 (48 GB)","tier":"Middenklasse","vram":48,"ram":64,"price":null,"components":["GPU: 2× RTX 3090 24 GB","CPU: desktop CPU with x8/x8 bifurcation, or HEDT","Moederbord: two x16-size slots spaced for 3-slot cards (x8/x8)","RAM: 64 GB","Voeding: 1000–1200 W","PCIe: 2× x8 PCIe 4.0; NVLink optional"],"models":[{"id":"llama-3-70b-instruct","name":"Llama 3 70B Q4_K_M (same size as Llama 3.3 70B)","href":"https://fedi.software/nl/ai/compare?m=llama-3-70b-instruct","quant":"Q4_K_M","tok_s":16.3,"kind":"measured","source_url":"https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference","source_date":"2024-05"}],"notes":"48 GB holds a dense 70B model at Q4_K_M (≈42.5 GB) with a short-to-medium context. llama.cpp splits the layers between the cards and runs them one after another, so speed is that of one 3090 reading the whole model; NVLink and x8 lanes barely matter here (they do for vLLM tensor parallel)."},{"_id":"epyc-ddr4","title":"Used EPYC server, 8-channel DDR4 (CPU only)","tier":"Middenklasse","vram":0,"ram":512,"price":2000,"components":["CPU: AMD EPYC 7002/7003 (Rome/Milan), e.g. EPYC 7702","Moederbord: ASRock Rack ROMED8-2T / Supermicro H12SSL-i / Gigabyte MZ32-AR0","RAM: 256–512 GB DDR4-3200 RDIMM, all 8 channels populated","Voeding: 750–1000 W","PCIe: 5–7× x16 PCIe 4.0 — room to add GPUs later"],"models":[{"id":"deepseek-r1","name":"DeepSeek R1 671B Q4 — EPYC 7702 + 512 GB DDR4-2400, MZ32-AR0","href":"https://fedi.software/nl/ai/compare?m=deepseek-r1","quant":"Q4","tok_s":[3.5,4.25],"kind":"measured","source_url":"https://digitalspaceport.com/how-to-run-deepseek-r1-671b-fully-locally-on-2000-epyc-rig/","source_date":"2025-01-29"},{"id":"openai-gpt-oss-120b","name":"gpt-oss-120b MXFP4","href":"https://fedi.software/nl/ai/compare?m=openai-gpt-oss-120b","quant":"MXFP4","tok_s":[12,16],"kind":"estimate"}],"notes":"Huge memory for little money: 8 channels of DDR4-3200 give 204.8 GB/s per socket. Only MoE models are practical — speed follows the active parameters. Fill every channel; a second socket adds NUMA trouble and little speed in llama.cpp (--numa distribute). The gpt-oss-120b figure is an estimate scaled by memory bandwidth from a DDR5 EPYC run. The source build cost about $2000 (prices of January 2025)."},{"_id":"strix-halo","title":"AMD Strix Halo mini-PC, 128 GB","tier":"Middenklasse","vram":96,"ram":128,"price":null,"components":["GPU: Radeon 8060S (integrated)","CPU: AMD Ryzen AI Max+ 395","Moederbord: Framework Desktop / other Strix Halo mini-PCs","RAM: 128 GB LPDDR5X-8000 unified (≈96 GB+ usable as VRAM)","Voeding: built-in"],"models":[{"id":"openai-gpt-oss-120b","name":"gpt-oss-120b MXFP4 (ROCm)","href":"https://fedi.software/nl/ai/compare?m=openai-gpt-oss-120b","quant":"MXFP4","tok_s":50.2,"kind":"measured","source_url":"https://quozul.dev/posts/2025-10-27-gpt-oss-120b-on-ryzen-ai-max-395/","source_date":"2025-10-27"}],"notes":"Quiet, ~120 W, the whole model in unified memory. Bandwidth (256 GB/s) is about a quarter of a 3090, so dense 70B models are slow; large MoE models with few active parameters are its sweet spot."},{"_id":"3x-3090","title":"Three RTX 3090 (72 GB)","tier":"High-end","vram":72,"ram":64,"price":null,"components":["GPU: 3× RTX 3090 24 GB","CPU: AMD EPYC 7002/7003 or Threadripper (enough PCIe lanes)","Moederbord: ASRock Rack ROMED8-2T / Supermicro H12SSL-i, risers for 3-slot cards","RAM: 64–128 GB","Voeding: 1200–1600 W (or two PSUs)","PCIe: 3× x8–x16 PCIe 4.0"],"models":[{"id":"openai-gpt-oss-120b","name":"gpt-oss-120b MXFP4, all in VRAM","href":"https://fedi.software/nl/ai/compare?m=openai-gpt-oss-120b","quant":"MXFP4","tok_s":[41.1,73.3],"ctx":"12.8k→93.7k","kind":"measured","source_url":"https://hardware-corner.net/3x-rtx-3090-gpt-oss-120b-test/","source_date":"2025-10-19"}],"notes":"gpt-oss-120b (≈65 GB) fits fully in 72 GB of VRAM with a long context: 73 tok/s at 12.8k tokens of context, 41 tok/s at 93.7k. The measured machine was rented, so the platform here is our suggestion: a server board gives every card enough lanes. Three 350 W cards need a big PSU; many owners power-limit them."},{"_id":"dgx-spark","title":"NVIDIA DGX Spark, 128 GB","tier":"High-end","vram":128,"ram":128,"price":3999,"components":["GPU: NVIDIA GB10 (integrated Blackwell)","CPU: Arm CPU of the GB10","Moederbord: NVIDIA DGX Spark","RAM: 128 GB LPDDR5X unified","Voeding: built-in"],"models":[{"id":"openai-gpt-oss-120b","name":"gpt-oss-120b MXFP4","href":"https://fedi.software/nl/ai/compare?m=openai-gpt-oss-120b","quant":"MXFP4","tok_s":60.6,"kind":"measured","source_url":"https://github.com/ggml-org/llama.cpp/discussions/16578","source_date":"2025-11-02"}],"notes":"A CUDA box with 128 GB of unified memory at 273 GB/s: the same speed class as Strix Halo, with the NVIDIA software stack. Early llama.cpp builds gave ~35 tok/s on gpt-oss-120b; the figure is after the November 2025 update (llama-bench tg128)."},{"_id":"mac-ultra","title":"Mac Studio, M2/M3 Ultra","tier":"High-end","vram":192,"ram":192,"price":null,"components":["GPU: Apple M2 Ultra 76-core / M3 Ultra 80-core GPU","CPU: Apple M2 Ultra / M3 Ultra","Moederbord: Mac Studio","RAM: 192–512 GB unified (800–819 GB/s)","Voeding: built-in"],"models":[{"id":"openai-gpt-oss-120b","name":"gpt-oss-120b MXFP4 — M2 Ultra 192 GB","href":"https://fedi.software/nl/ai/compare?m=openai-gpt-oss-120b","quant":"MXFP4","tok_s":79.7,"kind":"measured","source_url":"https://github.com/ggml-org/llama.cpp/discussions/15396","source_date":"2025-08"},{"id":"gpt-oss-20b","name":"gpt-oss-20b MXFP4 — M3 Ultra 512 GB","href":"https://fedi.software/nl/ai/compare?m=gpt-oss-20b","quant":"MXFP4","tok_s":115.5,"kind":"measured","source_url":"https://github.com/ggml-org/llama.cpp/discussions/15396","source_date":"2025-08"}],"notes":"The simplest way to run the biggest models at home: the M3 Ultra goes up to 512 GB of unified memory, enough for DeepSeek-class MoE models at 4 bits. Prompt processing is much slower than on NVIDIA GPUs. For very large models raise the GPU memory limit with sysctl iogpu.wired_limit_mb."}],"meta":{"sources":[{"name":"github.com","url":"https://github.com/ggml-org/llama.cpp/discussions/15396","license":null,"license_url":null,"modified":false},{"name":"www.hardware-corner.net","url":"https://www.hardware-corner.net/gpu-llm-benchmarks/rtx-3090/","license":null,"license_url":null,"modified":false},{"name":"carteakey.dev","url":"https://carteakey.dev/blog/local-inference/optimizing-gpt-oss-120b-local-inference/","license":null,"license_url":null,"modified":false},{"name":"github.crookster.org","url":"https://github.crookster.org/running-gpt-oss-120b-on-rtx-3080-ti-12-gb-at-home/","license":null,"license_url":null,"modified":false},{"name":"hardware-corner.net","url":"https://hardware-corner.net/3x-rtx-3090-gpt-oss-120b-test/","license":null,"license_url":null,"modified":false},{"name":"digitalspaceport.com","url":"https://digitalspaceport.com/how-to-run-deepseek-r1-671b-fully-locally-on-2000-epyc-rig/","license":null,"license_url":null,"modified":false},{"name":"quozul.dev","url":"https://quozul.dev/posts/2025-10-27-gpt-oss-120b-on-ryzen-ai-max-395/","license":null,"license_url":null,"modified":false}],"updated":"2026-10-02","notes":["Gemeten snelheden linken naar hun bron; ≈ markeert een schatting uit de geheugenbandbreedte. Prijzen zijn bij benadering."],"license":{"name":"fedi.software","license":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/","scope":"fedi.software data: hardware, builds, glossary, Fit calculations, curation and the compilation; third-party data keeps the licence of its source (sources[])"},"attribution":["Data by fedi.software, CC BY 4.0 — a link to the source page is required."]}}