New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
NVIDIA DGX Spark
Runs downloadable LLMs up to ~195B at Q4, ~44.5-53 tok/s on an 8B model.
What you can run
At 8k context, FP16 KV. Full-speed tok/s holds up to "max ctx"; beyond it the context spills to SSD swap and drops to the degraded figure.
| Model size | FP16 | Q8 | Q4 |
|---|---|---|---|
| 3B |
33.2-39.6 t/s ≤1643k ctx · then ~1.2-1.5 (slow) |
66.4-79.2 t/s ≤1689k ctx · then ~2.4-2.9 (slow) |
118.6-141.4 t/s ≤1709k ctx · then ~4.3-5.2 (slow) |
| 7-8B |
12.5-14.8 t/s ≤745k ctx · then ~0.5-0.5 (slow) |
24.9-29.7 t/s ≤806k ctx · then ~0.9-1.1 (slow) |
44.5-53 t/s ≤833k ctx · then ~1.6-1.9 (slow) |
| 13B |
7.7-9.1 t/s ≤535k ctx · then ~0.3-0.3 (slow) |
15.3-18.3 t/s ≤615k ctx · then ~0.6-0.7 (slow) |
27.4-32.6 t/s ≤650k ctx · then ~1-1.2 (slow) |
| 32B |
3.1-3.7 t/s ≤190k ctx · then ~0.1-0.1 (slow) |
6.2-7.4 t/s ≤312k ctx · then ~0.2-0.3 (slow) |
11.1-13.3 t/s ≤365k ctx · then ~0.4-0.5 (slow) |
| 70B | won't run |
2.8-3.4 t/s ≤133k ctx · then ~0.1-0.1 (slow) |
5.1-6.1 t/s ≤227k ctx · then ~0.2-0.2 (slow) |
| 120B | won't run | won't run |
3-3.5 t/s ≤142k ctx · then ~0.1-0.1 (slow) |
| 400B | won't run | won't run | won't run |
Estimate: tok/s ≈ bandwidth ÷ (params × bytes/param) × efficiency (NVIDIA). Cyan = fits ~115.2 GB usable. Beyond "ctx", KV spills to SSD swap (rough degraded speed shown).
Run popular models on the NVIDIA DGX Spark
Videos about the NVIDIA DGX Spark
NVIDIA DGX Spark vs RTX 4090 | LLM inference, training speed and more
Get Started With Your DGX Spark! | Setting Up LLMs & NIM
More hardware like the NVIDIA DGX Spark
About the NVIDIA DGX Spark
NVIDIA DGX Spark is a unified-memory system from NVIDIA, 2025. For local genAI the numbers that matter are its 128 GB (how big a model + context fits) and 273 GB/s bandwidth (generation speed).
Frequently asked questions
What size LLM can the NVIDIA DGX Spark run?▶
With ~115.2 GB usable it runs up to ~195B at Q4 (~55.5B at FP16) at 8k context. Bigger spills into SSD swap and slows sharply.
How many tokens/sec does the NVIDIA DGX Spark generate?▶
Generation is bandwidth-bound. At 273 GB/s an 8B model at Q4 runs ~44.5-53 tok/s (estimate).
Why does long context slow it down?▶
The KV cache for the context must also fit fast memory; once weights+KV exceed 115.2 GB it spills to SSD swap and throughput collapses. Each row shows the max usable context.