New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
AMD Radeon RX 7900 XTX
Runs downloadable LLMs up to ~32.5B at Q4, ~135-165 tok/s on an 8B model.
What you can run
At 8k context, FP16 KV. Full-speed tok/s holds up to "max ctx"; beyond it the context spills to system RAM (CPU/PCIe offload) and drops to the degraded figure.
| Model size | FP16 | Q8 | Q4 |
|---|---|---|---|
| 3B |
100.8-123.2 t/s ≤222k ctx · then ~4.2-5.1 (slow) |
201.6-246.4 t/s ≤268k ctx · then ~8.4-10.3 (slow) |
360-440 t/s ≤288k ctx · then ~15-18.3 (slow) |
| 7-8B |
37.8-46.2 t/s ≤35k ctx · then ~1.6-1.9 (slow) |
75.6-92.4 t/s ≤96k ctx · then ~3.2-3.9 (slow) |
135-165 t/s ≤123k ctx · then ~5.6-6.9 (slow) |
| 13B | won't run |
46.5-56.9 t/s ≤46k ctx · then ~1.9-2.4 (slow) |
83.1-101.5 t/s ≤81k ctx · then ~3.5-4.2 (slow) |
| 32B | won't run | won't run |
33.8-41.3 t/s ≤10k ctx · then ~1.4-1.7 (slow) |
| 70B | won't run | won't run | won't run |
| 120B | won't run | won't run | won't run |
| 400B | won't run | won't run | won't run |
Estimate: tok/s ≈ bandwidth ÷ (params × bytes/param) × efficiency (AMD). Cyan = fits ~22.1 GB usable. Beyond "ctx", KV spills to system RAM (CPU/PCIe offload) (rough degraded speed shown).
What else you need for a full build▶
Minimum supporting components for a desktop built around the AMD Radeon RX 7900 XTX. Guidance — exact needs depend on your case, other parts and how much you offload to system RAM.
Run popular models on the AMD Radeon RX 7900 XTX
Videos about the AMD Radeon RX 7900 XTX
7900 XTX + Instinct MI50-32gb: AMD's Unholy LLM Alliance. ROCm ROCon.
AMD Radeon RX 7900 XTX Review & GPU Benchmarks: Gaming, Thermals, Power, & Noise
More hardware like the AMD Radeon RX 7900 XTX
About the AMD Radeon RX 7900 XTX
AMD Radeon RX 7900 XTX is a discrete GPU from AMD on RDNA 3, 2022. For local genAI the numbers that matter are its 24 GB (how big a model + context fits) and 960 GB/s bandwidth (generation speed). Plan for a ~850W power supply.
Frequently asked questions
What size LLM can the AMD Radeon RX 7900 XTX run?▶
With ~22.1 GB usable it runs up to ~32.5B at Q4 (~9.5B at FP16) at 8k context. Bigger spills into system RAM (CPU/PCIe offload) and slows sharply.
How many tokens/sec does the AMD Radeon RX 7900 XTX generate?▶
Generation is bandwidth-bound. At 960 GB/s an 8B model at Q4 runs ~135-165 tok/s (estimate).
Why does long context slow it down?▶
The KV cache for the context must also fit fast memory; once weights+KV exceed 22.1 GB it spills to system RAM (CPU/PCIe offload) and throughput collapses. Each row shows the max usable context.
What else do I need to run the AMD Radeon RX 7900 XTX?▶
Besides the card: a ≥850W 80+ Gold power supply, at least 24GB of system RAM (2× the 24GB VRAM is ideal for CPU offload), a free PCIe x16 slot, any modern multi-core CPU, and a ≥1TB NVMe SSD for model weights.