New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
Jetson Thor T2000
Runs downloadable LLMs up to ~19B at Q4, ~22.3-26.6 tok/s on an 8B model.
What you can run
At 8k context, FP16 KV. Full-speed tok/s holds up to "max ctx"; beyond it the context spills to SSD swap and drops to the degraded figure.
| Model size | FP16 | Q8 | Q4 |
|---|---|---|---|
| 3B |
16.7-19.9 t/s ≤105k ctx · then ~1.2-1.5 (slow) |
33.3-39.7 t/s ≤151k ctx · then ~2.4-2.9 (slow) |
59.5-70.9 t/s ≤171k ctx · then ~4.3-5.2 (slow) |
| 7-8B | won't run |
12.5-14.9 t/s ≤37k ctx · then ~0.9-1.1 (slow) |
22.3-26.6 t/s ≤64k ctx · then ~1.6-1.9 (slow) |
| 13B | won't run | won't run |
13.7-16.4 t/s ≤34k ctx · then ~1-1.2 (slow) |
| 32B | won't run | won't run | won't run |
| 70B | won't run | won't run | won't run |
| 120B | won't run | won't run | won't run |
| 400B | won't run | won't run | won't run |
Estimate: tok/s ≈ bandwidth ÷ (params × bytes/param) × efficiency (NVIDIA). Cyan = fits ~14.4 GB usable. Beyond "ctx", KV spills to SSD swap (rough degraded speed shown).
Run popular models on the Jetson Thor T2000
More hardware like the Jetson Thor T2000
About the Jetson Thor T2000
Jetson Thor T2000 is a unified-memory system from NVIDIA, 2027. For local genAI the numbers that matter are its 16 GB (how big a model + context fits) and 137 GB/s bandwidth (generation speed).
Frequently asked questions
What size LLM can the Jetson Thor T2000 run?â–¶
With ~14.4 GB usable it runs up to ~19B at Q4 (~5.5B at FP16) at 8k context. Bigger spills into SSD swap and slows sharply.
How many tokens/sec does the Jetson Thor T2000 generate?â–¶
Generation is bandwidth-bound. At 137 GB/s an 8B model at Q4 runs ~22.3-26.6 tok/s (estimate).
Why does long context slow it down?â–¶
The KV cache for the context must also fit fast memory; once weights+KV exceed 14.4 GB it spills to SSD swap and throughput collapses. Each row shows the max usable context.