New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
Apple M2 Max 96GB
Runs downloadable LLMs up to ~146.5B at Q4, ~60.7-73.2 tok/s on an 8B model.
What you can run
At 8k context, FP16 KV. Full-speed tok/s holds up to "max ctx"; beyond it the context spills to SSD swap and drops to the degraded figure.
| Model size | FP16 | Q8 | Q4 |
|---|---|---|---|
| 3B |
45.3-54.7 t/s ≤1204k ctx · then ~1.1-1.4 (slow) |
90.7-109.3 t/s ≤1250k ctx · then ~2.3-2.7 (slow) |
161.9-195.2 t/s ≤1270k ctx · then ~4-4.9 (slow) |
| 7-8B |
17-20.5 t/s ≤526k ctx · then ~0.4-0.5 (slow) |
34-41 t/s ≤587k ctx · then ~0.9-1 (slow) |
60.7-73.2 t/s ≤614k ctx · then ~1.5-1.8 (slow) |
| 13B |
10.5-12.6 t/s ≤359k ctx · then ~0.3-0.3 (slow) |
20.9-25.2 t/s ≤439k ctx · then ~0.5-0.6 (slow) |
37.4-45.1 t/s ≤474k ctx · then ~0.9-1.1 (slow) |
| 32B |
4.3-5.1 t/s ≤80k ctx · then ~0.1-0.1 (slow) |
8.5-10.3 t/s ≤202k ctx · then ~0.2-0.3 (slow) |
15.2-18.3 t/s ≤256k ctx · then ~0.4-0.5 (slow) |
| 70B | won't run |
3.9-4.7 t/s ≤45k ctx · then ~0.1-0.1 (slow) |
6.9-8.4 t/s ≤139k ctx · then ~0.2-0.2 (slow) |
| 120B | won't run | won't run |
4-4.9 t/s ≤54k ctx · then ~0.1-0.1 (slow) |
| 400B | won't run | won't run | won't run |
Estimate: tok/s ≈ bandwidth ÷ (params × bytes/param) × efficiency (Apple). Cyan = fits ~86.4 GB usable. Beyond "ctx", KV spills to SSD swap (rough degraded speed shown).
Run popular models on the Apple M2 Max 96GB
Videos about the Apple M2 Max 96GB
M2 Max MacBook Pro Review: Back to Bumps!
Apple Mac Studio M2 Max / M2 Ultra - A Long Term User Review
More hardware like the Apple M2 Max 96GB
About the Apple M2 Max 96GB
Apple M2 Max 96GB is a unified-memory system from Apple on Apple M2, 2023. For local genAI the numbers that matter are its 96 GB (how big a model + context fits) and 400 GB/s bandwidth (generation speed).
Frequently asked questions
What size LLM can the Apple M2 Max 96GB run?▶
With ~86.4 GB usable it runs up to ~146.5B at Q4 (~41B at FP16) at 8k context. Bigger spills into SSD swap and slows sharply.
How many tokens/sec does the Apple M2 Max 96GB generate?▶
Generation is bandwidth-bound. At 400 GB/s an 8B model at Q4 runs ~60.7-73.2 tok/s (estimate).
Why does long context slow it down?▶
The KV cache for the context must also fit fast memory; once weights+KV exceed 86.4 GB it spills to SSD swap and throughput collapses. Each row shows the max usable context.