New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
NVIDIA RTX PRO 6000 Blackwell
Runs downloadable LLMs up to ~149.5B at Q4, ~292-348 tok/s on an 8B model.
What you can run
At 8k context, FP16 KV. Full-speed tok/s holds up to "max ctx"; beyond it the context spills to system RAM (CPU/PCIe offload) and drops to the degraded figure.
| Model size | FP16 | Q8 | Q4 |
|---|---|---|---|
| 3B |
218-259.8 t/s ≤1233k ctx · then ~4.9-5.8 (slow) |
436.1-519.7 t/s ≤1279k ctx · then ~9.7-11.6 (slow) |
778.7-928 t/s ≤1299k ctx · then ~17.4-20.7 (slow) |
| 7-8B |
81.8-97.4 t/s ≤540k ctx · then ~1.8-2.2 (slow) |
163.5-194.9 t/s ≤601k ctx · then ~3.7-4.4 (slow) |
292-348 t/s ≤628k ctx · then ~6.5-7.8 (slow) |
| 13B |
50.3-60 t/s ≤371k ctx · then ~1.1-1.3 (slow) |
100.6-119.9 t/s ≤451k ctx · then ~2.2-2.7 (slow) |
179.7-214.2 t/s ≤485k ctx · then ~4-4.8 (slow) |
| 32B |
20.4-24.4 t/s ≤87k ctx · then ~0.5-0.5 (slow) |
40.9-48.7 t/s ≤209k ctx · then ~0.9-1.1 (slow) |
73-87 t/s ≤263k ctx · then ~1.6-1.9 (slow) |
| 70B | won't run |
18.7-22.3 t/s ≤51k ctx · then ~0.4-0.5 (slow) |
33.4-39.8 t/s ≤145k ctx · then ~0.7-0.9 (slow) |
| 120B | won't run | won't run |
19.5-23.2 t/s ≤60k ctx · then ~0.4-0.5 (slow) |
| 400B | won't run | won't run | won't run |
Estimate: tok/s ≈ bandwidth ÷ (params × bytes/param) × efficiency (NVIDIA). Cyan = fits ~88.3 GB usable. Beyond "ctx", KV spills to system RAM (CPU/PCIe offload) (rough degraded speed shown).
What else you need for a full build▶
Minimum supporting components for a desktop built around the NVIDIA RTX PRO 6000 Blackwell. Guidance — exact needs depend on your case, other parts and how much you offload to system RAM.
Run popular models on the NVIDIA RTX PRO 6000 Blackwell
Videos about the NVIDIA RTX PRO 6000 Blackwell
RTX Pro 6000 LLM test
768GB of VRAM. 8x RTX PRO 6000 Blackwell. Liquid-cooled. 4U. Full review at StorageReview.com.
More hardware like the NVIDIA RTX PRO 6000 Blackwell
About the NVIDIA RTX PRO 6000 Blackwell
NVIDIA RTX PRO 6000 Blackwell is a discrete GPU from NVIDIA, 2025. For local genAI the numbers that matter are its 96 GB (how big a model + context fits) and 1792 GB/s bandwidth (generation speed). Plan for a ~1000W power supply.
Frequently asked questions
What size LLM can the NVIDIA RTX PRO 6000 Blackwell run?▶
With ~88.3 GB usable it runs up to ~149.5B at Q4 (~42B at FP16) at 8k context. Bigger spills into system RAM (CPU/PCIe offload) and slows sharply.
How many tokens/sec does the NVIDIA RTX PRO 6000 Blackwell generate?▶
Generation is bandwidth-bound. At 1792 GB/s an 8B model at Q4 runs ~292-348 tok/s (estimate).
Why does long context slow it down?▶
The KV cache for the context must also fit fast memory; once weights+KV exceed 88.3 GB it spills to system RAM (CPU/PCIe offload) and throughput collapses. Each row shows the max usable context.
What else do I need to run the NVIDIA RTX PRO 6000 Blackwell?▶
Besides the card: a ≥1000W 80+ Gold power supply, at least 96GB of system RAM (2× the 96GB VRAM is ideal for CPU offload), a free PCIe x16 slot, any modern multi-core CPU, and a ≥1TB NVMe SSD for model weights.