New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
NVIDIA B200
Runs downloadable LLMs up to ~305B at Q4, ~1303.6-1553.6 tok/s on an 8B model.
What you can run
At 8k context, FP16 KV. Full-speed tok/s holds up to "max ctx"; beyond it the context spills to system RAM (CPU/PCIe offload) and drops to the degraded figure.
| Model size | FP16 | Q8 | Q4 |
|---|---|---|---|
| 3B |
973.3-1160 t/s ≤2581k ctx · then ~4.9-5.8 (slow) |
1946.7-2320 t/s ≤2627k ctx · then ~9.7-11.6 (slow) |
3476.2-4142.9 t/s ≤2647k ctx · then ~17.4-20.7 (slow) |
| 7-8B |
365-435 t/s ≤1214k ctx · then ~1.8-2.2 (slow) |
730-870 t/s ≤1275k ctx · then ~3.7-4.4 (slow) |
1303.6-1553.6 t/s ≤1302k ctx · then ~6.5-7.8 (slow) |
| 13B |
224.6-267.7 t/s ≤910k ctx · then ~1.1-1.3 (slow) |
449.2-535.4 t/s ≤990k ctx · then ~2.2-2.7 (slow) |
802.2-956 t/s ≤1025k ctx · then ~4-4.8 (slow) |
| 32B |
91.3-108.8 t/s ≤424k ctx · then ~0.5-0.5 (slow) |
182.5-217.5 t/s ≤546k ctx · then ~0.9-1.1 (slow) |
325.9-388.4 t/s ≤600k ctx · then ~1.6-1.9 (slow) |
| 70B |
41.7-49.7 t/s ≤107k ctx · then ~0.2-0.2 (slow) |
83.4-99.4 t/s ≤321k ctx · then ~0.4-0.5 (slow) |
149-177.6 t/s ≤415k ctx · then ~0.7-0.9 (slow) |
| 120B | won't run |
48.7-58 t/s ≤168k ctx · then ~0.2-0.3 (slow) |
86.9-103.6 t/s ≤329k ctx · then ~0.4-0.5 (slow) |
| 400B | won't run | won't run | won't run |
Estimate: tok/s ≈ bandwidth ÷ (params × bytes/param) × efficiency (NVIDIA). Cyan = fits ~176.6 GB usable. Beyond "ctx", KV spills to system RAM (CPU/PCIe offload) (rough degraded speed shown).
What else you need for a full build▶
Minimum supporting components for a desktop built around the NVIDIA B200. Guidance — exact needs depend on your case, other parts and how much you offload to system RAM.
Run popular models on the NVIDIA B200
Videos about the NVIDIA B200
Inside a NEW AI Cluster - Tour with NVIDIA B200
INSIDE a 1.44TB HBM3e NVIDIA HGX B200 AI Server from ASRock Rack
More hardware like the NVIDIA B200
About the NVIDIA B200
NVIDIA B200 is a discrete GPU from NVIDIA, 2025. For local genAI the numbers that matter are its 192 GB (how big a model + context fits) and 8000 GB/s bandwidth (generation speed). Plan for a ~1400W power supply.
Frequently asked questions
What size LLM can the NVIDIA B200 run?▶
With ~176.6 GB usable it runs up to ~305B at Q4 (~86B at FP16) at 8k context. Bigger spills into system RAM (CPU/PCIe offload) and slows sharply.
How many tokens/sec does the NVIDIA B200 generate?▶
Generation is bandwidth-bound. At 8000 GB/s an 8B model at Q4 runs ~1303.6-1553.6 tok/s (estimate).
Why does long context slow it down?▶
The KV cache for the context must also fit fast memory; once weights+KV exceed 176.6 GB it spills to system RAM (CPU/PCIe offload) and throughput collapses. Each row shows the max usable context.
What else do I need to run the NVIDIA B200?▶
Besides the card: a ≥1400W 80+ Gold power supply, at least 192GB of system RAM (2× the 192GB VRAM is ideal for CPU offload), a free PCIe x16 slot, any modern multi-core CPU, and a ≥1TB NVMe SSD for model weights.