🔌
New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
// LOCAL-GENAI HARDWARE
Hardware for local LLMs
GPUs and unified-memory systems ranked for running downloadable models — memory, bandwidth, max model, estimated tokens/sec.
Missing something? Add it.
31 devices
Consumer NVIDIA GeForce RTX 5090
NVIDIA
32GB
Memory
1792 GB/s
Bandwidth
45B
Max model Q4
292-348
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 4090
NVIDIA
24GB
Memory
1008 GB/s
Bandwidth
32.5B
Max model Q4
164.3-195.8
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 3090 Ti
NVIDIA
24GB
Memory
1008 GB/s
Bandwidth
32.5B
Max model Q4
164.3-195.8
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 5080
NVIDIA
16GB
Memory
960 GB/s
Bandwidth
19.5B
Max model Q4
156.4-186.4
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 3090
NVIDIA
24GB
Memory
936 GB/s
Bandwidth
32.5B
Max model Q4
152.5-181.8
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 3080 Ti
NVIDIA
12GB
Memory
912 GB/s
Bandwidth
14.5B
Max model Q4
148.6-177.1
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 5070 Ti
NVIDIA
16GB
Memory
896 GB/s
Bandwidth
19.5B
Max model Q4
146-174
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 3080 10GB
NVIDIA
10GB
Memory
760 GB/s
Bandwidth
11B
Max model Q4
123.8-147.6
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 4080
NVIDIA
16GB
Memory
736 GB/s
Bandwidth
19.5B
Max model Q4
119.9-142.9
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 4070 Ti Super
NVIDIA
16GB
Memory
672 GB/s
Bandwidth
19.5B
Max model Q4
109.5-130.5
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 5070
NVIDIA
12GB
Memory
672 GB/s
Bandwidth
14.5B
Max model Q4
109.5-130.5
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 3070 Ti
NVIDIA
8GB
Memory
608 GB/s
Bandwidth
8.5B
Max model Q4
99.1-118.1
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 4070 Super
NVIDIA
12GB
Memory
504 GB/s
Bandwidth
14.5B
Max model Q4
82.1-97.9
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 4070
NVIDIA
12GB
Memory
504 GB/s
Bandwidth
14.5B
Max model Q4
82.1-97.9
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 5060 Ti 16GB
NVIDIA
16GB
Memory
448 GB/s
Bandwidth
19.5B
Max model Q4
73-87
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 5060 Ti 8GB
NVIDIA
8GB
Memory
448 GB/s
Bandwidth
8.5B
Max model Q4
73-87
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 3070
NVIDIA
8GB
Memory
448 GB/s
Bandwidth
8.5B
Max model Q4
73-87
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 4060 Ti 16GB
NVIDIA
16GB
Memory
288 GB/s
Bandwidth
19.5B
Max model Q4
46.9-55.9
8B Q4 tok/s
Consumer NVIDIA GeForce RTX 4060 Ti 8GB
NVIDIA
8GB
Memory
288 GB/s
Bandwidth
8.5B
Max model Q4
46.9-55.9
8B Q4 tok/s
NV Consumer
NVIDIA GeForce RTX 2050
NVIDIA
8GB
Memory
112 GB/s
Bandwidth
7B
Max model Q4
18.3-21.8
8B Q4 tok/s
NV Consumer
NVIDIA GeForce RTX 2050 Mobile
NVIDIA
4GB
Memory
112 GB/s
Bandwidth
2.5B
Max model Q4
18.3-21.8
8B Q4 tok/s
Prosumer NVIDIA RTX PRO 6000 Blackwell
NVIDIA
96GB
Memory
1792 GB/s
Bandwidth
149.5B
Max model Q4
292-348
8B Q4 tok/s
Prosumer NVIDIA RTX 6000 Ada 48GB
NVIDIA
48GB
Memory
960 GB/s
Bandwidth
71B
Max model Q4
156.4-186.4
8B Q4 tok/s
Prosumer NVIDIA DGX Spark
NVIDIA
128GB
Memory
273 GB/s
Bandwidth
195B
Max model Q4
44.5-53
8B Q4 tok/s
SME NVIDIA H100 PCIe 80GB
NVIDIA
80GB
Memory
2000 GB/s
Bandwidth
123.5B
Max model Q4
325.9-388.4
8B Q4 tok/s
SME NVIDIA A100 PCIe 80GB
NVIDIA
80GB
Memory
1935 GB/s
Bandwidth
123.5B
Max model Q4
315.3-375.8
8B Q4 tok/s
SME NVIDIA L40S 48GB
NVIDIA
48GB
Memory
864 GB/s
Bandwidth
71B
Max model Q4
140.8-167.8
8B Q4 tok/s
Enterprise NVIDIA B200
NVIDIA
192GB
Memory
8000 GB/s
Bandwidth
305B
Max model Q4
1303.6-1553.6
8B Q4 tok/s
Enterprise NVIDIA H200
NVIDIA
141GB
Memory
4800 GB/s
Bandwidth
221B
Max model Q4
782.1-932.1
8B Q4 tok/s
Enterprise NVIDIA H100 SXM5 80GB
NVIDIA
80GB
Memory
3350 GB/s
Bandwidth
123.5B
Max model Q4
545.9-650.6
8B Q4 tok/s
Enterprise NVIDIA A100 SXM4 80GB
NVIDIA
80GB
Memory
2039 GB/s
Bandwidth
123.5B
Max model Q4
332.2-396
8B Q4 tok/s
How to read this
Two numbers dominate local inference: memory (how big a model + context fits) and bandwidth (token speed). Click any hardware for the full size×quant matrix, max usable context and measured reports, or browse runtimes.