Language Proprietary
- Params
- —
- Context
- —
- Released
- Feb 2024
🔌
New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
// COMPARISON
Gemini 1.5 Pro
A head-to-head benchmark comparison of Gemini 1.5 Pro across 16 evaluations.
The models
Benchmark comparison
| Benchmark | Gemini 1.5 Pro |
|---|---|
| code | |
| HumanEval | 86.6% |
| MBPP+ | 75.4% |
| general | |
| ArenaHard | 85.3% |
| IFEval | 89.4% |
| MMLU | 86.8% |
| MMLU-Pro | 75.8% |
| math | |
| GSM8K | 95.2% |
| MATH | 84.6% |
| multimodal | |
| ChartQA | 87.2% |
| DocVQA | 93.100 |
| MMMU | 66.3% |
| MathVista | 67.8% |
| reasoning | |
| DROP | 89.2% |
| GPQA Diamond | 59.1% |
| safety | |
| C-SimpleQA | 59.4% |
| SimpleQA | 23.4% |
Best result per row highlighted in cyan. Each benchmark links to its definition and sources; each model links to its full scorecard.
Frequently asked questions
Which of these models is best for coding? ▶
Gemini 1.5 Pro has the strongest result on HumanEval among the models compared here. See the code-category rows in the table for the full picture.
How many benchmarks are compared? ▶
This comparison covers 16 benchmarks on which at least one of the selected models has a published score.
Where do the benchmark scores come from? ▶
Scores are aggregated from official model cards, technical reports and standard public evaluations, and link back to each benchmark's source. They are updated as new results are published.
Explore further
Build a different comparison
Models (0 selected)
Benchmarks (auto)
Select at least two models.