New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
NVIDIA GeForce RTX 5070 Ti
Runs downloadable LLMs up to ~19.5B at Q4, ~146-174 tok/s on an 8B model.
What you can run
At 8k context, FP16 KV. Full-speed tok/s holds up to "max ctx"; beyond it the context spills to system RAM (CPU/PCIe offload) and drops to the degraded figure.
| Model size | FP16 | Q8 | Q4 |
|---|---|---|---|
| 3B |
109-129.9 t/s ≤110k ctx · then ~4.9-5.8 (slow) |
218-259.8 t/s ≤156k ctx · then ~9.7-11.6 (slow) |
389.3-464 t/s ≤176k ctx · then ~17.4-20.7 (slow) |
| 7-8B | won't run |
81.8-97.4 t/s ≤40k ctx · then ~3.7-4.4 (slow) |
146-174 t/s ≤67k ctx · then ~6.5-7.8 (slow) |
| 13B | won't run |
50.3-60 t/s ≤1k ctx · then ~2.2-2.7 (slow) |
89.8-107.1 t/s ≤36k ctx · then ~4-4.8 (slow) |
| 32B | won't run | won't run | won't run |
| 70B | won't run | won't run | won't run |
| 120B | won't run | won't run | won't run |
| 400B | won't run | won't run | won't run |
Estimate: tok/s ≈ bandwidth ÷ (params × bytes/param) × efficiency (NVIDIA). Cyan = fits ~14.7 GB usable. Beyond "ctx", KV spills to system RAM (CPU/PCIe offload) (rough degraded speed shown).
What else you need for a full build▶
Minimum supporting components for a desktop built around the NVIDIA GeForce RTX 5070 Ti. Guidance — exact needs depend on your case, other parts and how much you offload to system RAM.
Run popular models on the NVIDIA GeForce RTX 5070 Ti
Videos about the NVIDIA GeForce RTX 5070 Ti
Should You Buy nVidia RTX 5070ti 16gb GPU for Local AI? Qwen 3.6 Agents?
RTX 5070 Ti Productivity Performance: AI, 3D Modeling, and Video Editing
More hardware like the NVIDIA GeForce RTX 5070 Ti
About the NVIDIA GeForce RTX 5070 Ti
NVIDIA GeForce RTX 5070 Ti is a discrete GPU from NVIDIA on Blackwell, 2025. For local genAI the numbers that matter are its 16 GB (how big a model + context fits) and 896 GB/s bandwidth (generation speed). Plan for a ~750W power supply.
Frequently asked questions
What size LLM can the NVIDIA GeForce RTX 5070 Ti run?▶
With ~14.7 GB usable it runs up to ~19.5B at Q4 (~6B at FP16) at 8k context. Bigger spills into system RAM (CPU/PCIe offload) and slows sharply.
How many tokens/sec does the NVIDIA GeForce RTX 5070 Ti generate?▶
Generation is bandwidth-bound. At 896 GB/s an 8B model at Q4 runs ~146-174 tok/s (estimate).
Why does long context slow it down?▶
The KV cache for the context must also fit fast memory; once weights+KV exceed 14.7 GB it spills to system RAM (CPU/PCIe offload) and throughput collapses. Each row shows the max usable context.
What else do I need to run the NVIDIA GeForce RTX 5070 Ti?▶
Besides the card: a ≥750W 80+ Gold power supply, at least 16GB of system RAM (2× the 16GB VRAM is ideal for CPU offload), a free PCIe x16 slot, any modern multi-core CPU, and a ≥1TB NVMe SSD for model weights.