適用本機 AI 模型的 PC 配置，20B 至 120B — 2026
從單張 16 GB 顯示卡到三張 RTX 3090、EPYC 伺服器與統一記憶體電腦：每套配置能跑什麼、速度多快，並附上每項實測的來源。
-
One 16 GB GPU
- 預算
- 入門級
- 顯示記憶體
- ≈16 GB
- RAM
- ≈32 GB
- 零組件
- GPU: RTX 5060 Ti 16 GB / RTX 4060 Ti 16 GB
- CPU: any 6–8-core desktop CPU
- 主機板: any ATX/mATX with a PCIe x16 slot
- RAM: 32 GB DDR4/DDR5
- 電源供應器: 550–650 W
- PCIe: 1× x16 (x8 electrical is enough)
- 可執行
- gpt-oss-20b MXFP4 (RTX 5060 Ti 16 GB) MXFP4 112 tok/s [實測]
- Qwen3-30B-A3B Q4_K_M, experts partly in RAM Q4_K_M [≈ 依記憶體頻寬估算]
- 備註
- gpt-oss-20b (≈12 GB) fits entirely in VRAM with room for a long context. Qwen3-30B-A3B Q4_K_M (≈18.6 GB) does not fit: a few expert layers go to system RAM with --n-cpu-moe, still usable because only ~3B parameters are active. Measured on an RTX 5060 Ti 16 GB (llama-bench tg128).
-
One RTX 3090 24 GB (used)
- 預算
- 入門級
- 顯示記憶體
- ≈24 GB
- RAM
- ≈64 GB
- 零組件
- GPU: RTX 3090 24 GB
- CPU: any 6–8-core desktop CPU
- 主機板: any ATX with a PCIe x16 slot
- RAM: 32–64 GB DDR4/DDR5
- 電源供應器: 750–850 W
- PCIe: 1× x16
- 可執行
- gpt-oss-20b MXFP4 MXFP4 162 tok/s [實測]
- Qwen3-30B-A3B Q4_K Q4_K_M 154 tok/s @4k [實測]
- Gemma 3 27B Q4_K_M Q4_K_M ≈31–42 tok/s [≈ 依記憶體頻寬估算]
- 備註
- The cheapest way to 24 GB of fast VRAM. Small MoE models fly; dense ~27–32B models fit at Q4 with a moderate context. The Gemma 3 27B figure is our estimate: 936 GB/s × 55–75 % efficiency ÷ 16.5 GB of weights.
-
12–24 GB GPU + 64–128 GB RAM (MoE offload)
- 預算
- 入門級
- 顯示記憶體
- ≈24 GB
- RAM
- ≈64 GB
- 零組件
- GPU: RTX 3090 24 GB / RTX 4070 12 GB / RTX 3080 Ti 12 GB
- CPU: Core i5-12600K / Core Ultra 7 265K class
- 主機板: desktop board, both memory channels populated
- RAM: 64–128 GB DDR5 (XMP/EXPO on) or DDR4-3600
- 電源供應器: 750–850 W
- PCIe: 1× x16
- 可執行
- gpt-oss-120b MXFP4 — RTX 3090 + 64 GB DDR5-5200, --n-cpu-moe 24–26 MXFP4 26–28 tok/s [實測]
- gpt-oss-120b MXFP4 — RTX 4070 12 GB + 64 GB DDR5 MXFP4 25–28 tok/s [實測]
- gpt-oss-120b MXFP4 — RTX 3080 Ti 12 GB + 128 GB DDR4-3600, --n-cpu-moe 36 MXFP4 18.0–22 tok/s [實測]
- 備註
- llama.cpp --n-cpu-moe N keeps attention and the shared layers on the GPU and moves the experts of N layers to system RAM. Works well only for MoE models; RAM speed matters — the 4070 build ran at 10–11 tok/s until XMP was turned on.
-
Two RTX 3090 (48 GB)
- 預算
- 中階
- 顯示記憶體
- ≈48 GB
- RAM
- ≈64 GB
- 零組件
- GPU: 2× RTX 3090 24 GB
- CPU: desktop CPU with x8/x8 bifurcation, or HEDT
- 主機板: two x16-size slots spaced for 3-slot cards (x8/x8)
- RAM: 64 GB
- 電源供應器: 1000–1200 W
- PCIe: 2× x8 PCIe 4.0; NVLink optional
- 可執行
- Llama 3 70B Q4_K_M (same size as Llama 3.3 70B) Q4_K_M 16.3 tok/s [實測]
- 備註
- 48 GB holds a dense 70B model at Q4_K_M (≈42.5 GB) with a short-to-medium context. llama.cpp splits the layers between the cards and runs them one after another, so speed is that of one 3090 reading the whole model; NVLink and x8 lanes barely matter here (they do for vLLM tensor parallel).
-
Used EPYC server, 8-channel DDR4 (CPU only)
- 預算
- 中階
- 顯示記憶體
- ≈0 GB
- RAM
- ≈512 GB
- 價格
- US$2,000
- 零組件
- CPU: AMD EPYC 7002/7003 (Rome/Milan), e.g. EPYC 7702
- 主機板: ASRock Rack ROMED8-2T / Supermicro H12SSL-i / Gigabyte MZ32-AR0
- RAM: 256–512 GB DDR4-3200 RDIMM, all 8 channels populated
- 電源供應器: 750–1000 W
- PCIe: 5–7× x16 PCIe 4.0 — room to add GPUs later
- 可執行
- DeepSeek R1 671B Q4 — EPYC 7702 + 512 GB DDR4-2400, MZ32-AR0 Q4 3.5–4.2 tok/s [實測]
- gpt-oss-120b MXFP4 MXFP4 ≈12.0–16.0 tok/s [≈ 依記憶體頻寬估算]
- 備註
- Huge memory for little money: 8 channels of DDR4-3200 give 204.8 GB/s per socket. Only MoE models are practical — speed follows the active parameters. Fill every channel; a second socket adds NUMA trouble and little speed in llama.cpp (--numa distribute). The gpt-oss-120b figure is an estimate scaled by memory bandwidth from a DDR5 EPYC run. The source build cost about $2000 (prices of January 2025).
-
AMD Strix Halo mini-PC, 128 GB
- 預算
- 中階
- 顯示記憶體
- ≈96 GB
- RAM
- ≈128 GB
- 零組件
- GPU: Radeon 8060S (integrated)
- CPU: AMD Ryzen AI Max+ 395
- 主機板: Framework Desktop / other Strix Halo mini-PCs
- RAM: 128 GB LPDDR5X-8000 unified (≈96 GB+ usable as VRAM)
- 電源供應器: built-in
- 可執行
- gpt-oss-120b MXFP4 (ROCm) MXFP4 50 tok/s [實測]
- 備註
- Quiet, ~120 W, the whole model in unified memory. Bandwidth (256 GB/s) is about a quarter of a 3090, so dense 70B models are slow; large MoE models with few active parameters are its sweet spot.
-
Three RTX 3090 (72 GB)
- 預算
- 高階
- 顯示記憶體
- ≈72 GB
- RAM
- ≈64 GB
- 零組件
- GPU: 3× RTX 3090 24 GB
- CPU: AMD EPYC 7002/7003 or Threadripper (enough PCIe lanes)
- 主機板: ASRock Rack ROMED8-2T / Supermicro H12SSL-i, risers for 3-slot cards
- RAM: 64–128 GB
- 電源供應器: 1200–1600 W (or two PSUs)
- PCIe: 3× x8–x16 PCIe 4.0
- 可執行
- gpt-oss-120b MXFP4, all in VRAM MXFP4 41–73 tok/s @12.8k→93.7k [實測]
- 備註
- gpt-oss-120b (≈65 GB) fits fully in 72 GB of VRAM with a long context: 73 tok/s at 12.8k tokens of context, 41 tok/s at 93.7k. The measured machine was rented, so the platform here is our suggestion: a server board gives every card enough lanes. Three 350 W cards need a big PSU; many owners power-limit them.
-
NVIDIA DGX Spark, 128 GB
- 預算
- 高階
- 顯示記憶體
- ≈128 GB
- RAM
- ≈128 GB
- 價格
- US$3,999
- 零組件
- GPU: NVIDIA GB10 (integrated Blackwell)
- CPU: Arm CPU of the GB10
- 主機板: NVIDIA DGX Spark
- RAM: 128 GB LPDDR5X unified
- 電源供應器: built-in
- 可執行
- gpt-oss-120b MXFP4 MXFP4 61 tok/s [實測]
- 備註
- A CUDA box with 128 GB of unified memory at 273 GB/s: the same speed class as Strix Halo, with the NVIDIA software stack. Early llama.cpp builds gave ~35 tok/s on gpt-oss-120b; the figure is after the November 2025 update (llama-bench tg128).
-
Mac Studio, M2/M3 Ultra
- 預算
- 高階
- 顯示記憶體
- ≈192 GB
- RAM
- ≈192 GB
- 零組件
- GPU: Apple M2 Ultra 76-core / M3 Ultra 80-core GPU
- CPU: Apple M2 Ultra / M3 Ultra
- 主機板: Mac Studio
- RAM: 192–512 GB unified (800–819 GB/s)
- 電源供應器: built-in
- 可執行
- gpt-oss-120b MXFP4 — M2 Ultra 192 GB MXFP4 80 tok/s [實測]
- gpt-oss-20b MXFP4 — M3 Ultra 512 GB MXFP4 116 tok/s [實測]
- 備註
- The simplest way to run the biggest models at home: the M3 Ultra goes up to 512 GB of unified memory, enough for DeepSeek-class MoE models at 4 bits. Prompt processing is much slower than on NVIDIA GPUs. For very large models raise the GPU memory limit with sysctl iogpu.wired_limit_mb.
實測速度附有來源連結；≈ 表示依記憶體頻寬估算。價格為概略值。
更新日期 · 來源: github.com , www.hardware-corner.net , carteakey.dev , github.crookster.org , hardware-corner.net , digitalspaceport.com , quozul.dev
三個預算級距
卡片依預算整理出適合本機模型的整機配置。「入門級」有單張 16 GB 顯示卡、單張二手 RTX 3090，以及搭配大量 RAM 做 MoE 卸載的 GPU 配置；「中階」有兩張 RTX 3090、8 通道 DDR4 的二手 EPYC 伺服器、AMD Strix Halo 迷你電腦；「高階」有三張 RTX 3090、NVIDIA DGX Spark，以及搭載 M2/M3 Ultra 的 Mac Studio。每張卡片列出顯示記憶體、RAM、大約價格、從 GPU 到電源供應器與 PCIe 通道的零組件、可執行的模型與速度，以及備註。
實測與估算
沒有標記的速度是實測值，附有來源連結（多半是 llama.cpp 的效能測試討論串或公開報告）與日期；標 ≈ 的則是依記憶體頻寬推算的估算值。實測結果取決於當時的執行環境版本與設定，您的機器跑出來可能略高或略低。
怎麼選配置
先決定想跑多大的模型，這決定了記憶體；再看能接受的速度，也就是記憶體頻寬；最後才比較噪音、耗電與價格。只要模型放得進顯示卡的記憶體，顯示卡就很快；Mac、Strix Halo、DGX Spark 這類統一記憶體機器能放更大的模型，但生成較慢；CPU 伺服器能便宜地裝下巨大的 MoE 模型，不過實用的只有 MoE。
兩個常見誤解
- 多插幾張卡不會變快。llama.cpp 會把模型分給多張卡，但各卡依序運算，能載入更大的模型，速度卻仍接近單張卡。
- MoE 卸載（--n-cpu-moe）把共用層留在 GPU、把專家放到系統 RAM，只適用於 MoE 模型，而且 RAM 速度影響很大。
價格僅供參考，二手卡差異尤其大。執行模型還需要 App（見本機 AI 工具）與合適量化的檔案，量化的讀法請看AI 模型類型。