🔌
New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
// INFERENCE RUNTIME
Ollama
User-friendly local LLM runner with a REST API and model library. Ships with llama.cpp under the hood and exposes an OpenAI-compatible API. Designed for ease of use and quick model deployment.
- Platforms
- Windows, macOS, Linux
- Backends
- CUDA, Metal, ROCm, CPU
- Quant formats
- GGUF (Q4_K_M, Q5_K_M, Q8_0, F16 and custom manifests)
Measured with Ollama
| Hardware | Model | Quant | tok/s | Source |
|---|---|---|---|---|
| NVIDIA GeForce RTX 4090 | Gemma 4 26B | Q4_K_M | 149.6 | blog ↗ |
| NVIDIA GeForce RTX 4090 | Gemma 4 31B | Q4_K_M | 7.8 | blog ↗ |
Videos about Ollama
Learn Ollama in 15 Minutes - Run LLM Models Locally for FREE
Ollama Course – Build AI Apps Locally
What is Ollama? Running Local LLMs Made Simple
FAQ
What platforms does Ollama support?▶
Ollama runs on Windows, macOS, Linux with backends CUDA, Metal, ROCm, CPU.
What quant formats does Ollama support?▶
Ollama supports GGUF (Q4_K_M, Q5_K_M, Q8_0, F16 and custom manifests).