New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
Apple M2 Ultra 192GB
Runs downloadable LLMs up to ~298B at Q4, ~121.4-146.4 tok/s on an 8B model.
What you can run
At 8k context, FP16 KV. Full-speed tok/s holds up to "max ctx"; beyond it the context spills to SSD swap and drops to the degraded figure.
| Model size | FP16 | Q8 | Q4 |
|---|---|---|---|
| 3B |
90.7-109.3 t/s ≤2522k ctx · then ~1.1-1.4 (slow) |
181.3-218.7 t/s ≤2568k ctx · then ~2.3-2.7 (slow) |
323.8-390.5 t/s ≤2588k ctx · then ~4-4.9 (slow) |
| 7-8B |
34-41 t/s ≤1185k ctx · then ~0.4-0.5 (slow) |
68-82 t/s ≤1246k ctx · then ~0.9-1 (slow) |
121.4-146.4 t/s ≤1273k ctx · then ~1.5-1.8 (slow) |
| 13B |
20.9-25.2 t/s ≤887k ctx · then ~0.3-0.3 (slow) |
41.8-50.5 t/s ≤966k ctx · then ~0.5-0.6 (slow) |
74.7-90.1 t/s ≤1001k ctx · then ~0.9-1.1 (slow) |
| 32B |
8.5-10.3 t/s ≤409k ctx · then ~0.1-0.1 (slow) |
17-20.5 t/s ≤531k ctx · then ~0.2-0.3 (slow) |
30.4-36.6 t/s ≤585k ctx · then ~0.4-0.5 (slow) |
| 70B |
3.9-4.7 t/s ≤96k ctx · then ~0-0.1 (slow) |
7.8-9.4 t/s ≤309k ctx · then ~0.1-0.1 (slow) |
13.9-16.7 t/s ≤403k ctx · then ~0.2-0.2 (slow) |
| 120B | won't run |
4.5-5.5 t/s ≤157k ctx · then ~0.1-0.1 (slow) |
8.1-9.8 t/s ≤318k ctx · then ~0.1-0.1 (slow) |
| 400B | won't run | won't run | won't run |
Estimate: tok/s ≈ bandwidth ÷ (params × bytes/param) × efficiency (Apple). Cyan = fits ~172.8 GB usable. Beyond "ctx", KV spills to SSD swap (rough degraded speed shown).
Run popular models on the Apple M2 Ultra 192GB
Videos about the Apple M2 Ultra 192GB
Apple fans, start typing your angry comments now… M2 Ultra Mac Studio Review
Mac Studio M2 Ultra Review - It DESTROYS my MacBook Pro! 🔥
More hardware like the Apple M2 Ultra 192GB
About the Apple M2 Ultra 192GB
Apple M2 Ultra 192GB is a unified-memory system from Apple on Apple M2, 2023. For local genAI the numbers that matter are its 192 GB (how big a model + context fits) and 800 GB/s bandwidth (generation speed).
Frequently asked questions
What size LLM can the Apple M2 Ultra 192GB run?▶
With ~172.8 GB usable it runs up to ~298B at Q4 (~84B at FP16) at 8k context. Bigger spills into SSD swap and slows sharply.
How many tokens/sec does the Apple M2 Ultra 192GB generate?▶
Generation is bandwidth-bound. At 800 GB/s an 8B model at Q4 runs ~121.4-146.4 tok/s (estimate).
Why does long context slow it down?▶
The KV cache for the context must also fit fast memory; once weights+KV exceed 172.8 GB it spills to SSD swap and throughput collapses. Each row shows the max usable context.