New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
HumanEval+ benchmark
AI model leaderboard for the HumanEval+ benchmark. Compare how large language models score on HumanEval+, see the full ranking, and understand what this AI benchmark measures. Mistral Small 3.2 currently leads with 92.9%. EvalPlus extension of HumanEval with ~80x more tests per problem to expose buggy completions.
Leaderboard
| # | Model | Organization | Score | Variant | Source |
|---|---|---|---|---|---|
| #1 | Mistral Small 3.2 | Mistral AI | 92.9% | instruct-pass@5 | official ↗ |
| #2 | Tulu 3 405B | Allen Institute for AI | 92.9% | instruct-pass@10 | official ↗ |
| #3 | GPT-4o (Nov 2024) | OpenAI | 92.7% | pass@10-cited-tulu3 | official ↗ |
| #4 | DeepSeek-V3 | DeepSeek | 91.6% | pass@10-cited-tulu3 | official ↗ |
| #5 | Qwen2.5 Instruct (72B) | Qwen | 90.8% | instruct-pass@10-cited-tulu3 | official ↗ |
| #6 | Llama 3.1-405B | Meta AI | 90.3% | instruct-pass@10-cited-tulu3 | official ↗ |
| #7 | Qwen2.5-7B | Qwen | 89.91% | instruct-cited-granite-3-2 | official ↗ |
| #8 | Llama 3.1-70B | Meta AI | 89.5% | instruct-pass@10-cited-tulu3 | official ↗ |
| #9 | Mistral Small 3.1 | Mistral AI | 88.99% | instruct-pass@5 | official ↗ |
| #10 | Ministral 8B | Mistral AI | 88.5% | instruct-pass@10-cited-tulu3 | official ↗ |
| #11 | Hermes 3 405B | Nous Research | 86.9% | pass@10-cited-tulu3 | official ↗ |
| #12 | Hermes 3 70B | Nous Research | 85.9% | pass@10-cited-tulu3 | official ↗ |
| #13 | Granite 3.1 8B | IBM | 85.79% | instruct-cited-granite-3-2 | official ↗ |
| #14 | Granite 3.2 8B | IBM | 85.72% | instruct-from-hf-readme | official ↗ |
| #15 | Granite-4.0-H-Small | IBM | 83% | pass@1 | official ↗ |
| #16 | Llama 3.1-8B | Meta AI | 82.9% | instruct-pass@10-cited-tulu3 | official ↗ |
| #17 | Llama 3.1-8B | Meta AI | 80.15% | instruct-cited-granite-3-2 | official ↗ |
| #18 | Tulu 3 8B | Allen Institute for AI | 79.2% | instruct-pass@10 | official ↗ |
| #19 | DeepSeek-R1-Distill-Qwen-7B | Qwen | 78.43% | cited-granite-3-2 | official ↗ |
| #20 | DeepSeek-R1-Distill-Qwen-7B | DeepSeek | 78.43% | cited-granite-3-2 | official ↗ |
| #21 | Granite-4.0-H-Tiny | IBM | 76% | pass@1 | official ↗ |
| #22 | Granite 3.1 2B | IBM | 75.26% | instruct-cited-granite-3-2 | official ↗ |
| #23 | Granite-4.0-H-Micro | IBM | 75% | pass@1 | official ↗ |
| #24 | Granite 3.2 2B | IBM | 73.39% | instruct-from-hf-readme | official ↗ |
| #25 | LongCat-Flash | Meituan Inc | 67.8% | pass@1 | official ↗ |
| #26 | Gemma 2 9B | Google DeepMind | 67% | instruct-pass@10-cited-tulu3 | official ↗ |
| #27 | DeepSeek-R1-Distill-Llama-8B | DeepSeek | 62.91% | cited-granite-3-2 | official ↗ |
| #28 | Nemotron-H 47B | NVIDIA | 56.1% | base-0-shot | official ↗ |
| #29 | Nemotron-H 8B | NVIDIA | 55.49% | base-0-shot | official ↗ |
| #30 | Nemotron-H 56B | NVIDIA | 54.27% | base-0-shot | official ↗ |
Frequently asked questions about HumanEval+
What is the HumanEval+ benchmark?
EvalPlus extension of HumanEval with ~80x more tests per problem to expose buggy completions.
How is the HumanEval+ benchmark scored?
HumanEval+ is scored using the pass@1 (%) metric, where a higher score is better. The maximum achievable score is 100.000. GenAIList aggregates reported scores from model providers and papers into a single ranked leaderboard.
Which AI model scores highest on HumanEval+?
As of the latest reported scores on GenAIList, Mistral Small 3.2 achieves the highest result on HumanEval+ with a score of 92.9%.
Is a higher HumanEval+ score better?
Yes. On HumanEval+ a higher score indicates better performance, so models near the top of the leaderboard are the strongest.