New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
MBPP benchmark
AI model leaderboard for the MBPP benchmark. Compare how large language models score on MBPP, see the full ranking, and understand what this AI benchmark measures. Ling-1T currently leads with 96.87%. Mostly Basic Python Programming - 1,000 Python problems for entry-level program synthesis.
Leaderboard
| # | Model | Organization | Score | Variant | Source |
|---|---|---|---|---|---|
| #1 | Ling-1T | Ant Group | 96.87% | sanitized-pass@1 | official ↗ |
| #2 | GPT-5 | OpenAI | 91.72% | main-sanitized-cited-ling-1t | official ↗ |
| #3 | gemini-2.5-pro | Google DeepMind | 91.01% | lowthink-sanitized-cited-ling-1t | official ↗ |
| #4 | DeepSeek-V3.1-Terminus | DeepSeek | 90.69% | sanitized-non-thinking-cited-ling-1t | official ↗ |
| #5 | Kimi K2 | Moonshot | 89.96% | instruct-0905-sanitized-cited-ling-1t | official ↗ |
| #6 | IBM Granite 4.1 | IBM | 87.3% | official ↗ | |
| #7 | Llama 3.1-70B | Meta AI | 86% | cited-telechat2 | official ↗ |
| #8 | Llama Nemotron Nano 8B | NVIDIA | 84.6% | reasoning-on-0-shot-pass@1 | official ↗ |
| #9 | GPT-4o mini | OpenAI | 84.1% | cited-phi35moe | official ↗ |
| #10 | Granite-4.0-H-Small | IBM | 84% | pass@1 | official ↗ |
| #11 | Phi-3.5-MoE | Microsoft | 80.8% | instruct-3-shot | official ↗ |
| #12 | Qwen2-72B | Qwen | 80.2% | instruct-cited-telechat2 | official ↗ |
| #13 | Granite-4.0-H-Tiny | IBM | 80% | pass@1 | official ↗ |
| #14 | Telechat2-115B | China Telecom | 78% | from-hf-readme | official ↗ |
| #15 | Nemotron-H 56B | NVIDIA | 77.82% | base-sanitized-3-shot | official ↗ |
| #16 | Gemini 1.5 Flash (Sep 2024) | Google DeepMind | 77.5% | cited-phi35moe | official ↗ |
| #17 | Nemotron-H 47B | NVIDIA | 75.9% | base-sanitized-3-shot | official ↗ |
| #18 | Nemotron-4 340B | NVIDIA | 75.4% | 0-shot-instruct-from-hf-readme | official ↗ |
| #19 | TeleChat2-35B | China Telecom | 75% | from-hf-readme | official ↗ |
| #20 | Granite-4.0-H-Micro | IBM | 73% | pass@1 | official ↗ |
| #21 | DeepSeek-V2 (MoE-236B) | DeepSeek | 72% | cited-telechat2 | official ↗ |
| #22 | Qwen2-57B-A14B | Qwen | 71.9% | base-from-hf-readme | official ↗ |
| #23 | Llama 3.1-8B | Meta AI | 69.4% | instruct-cited-phi35moe | official ↗ |
| #24 | Gemma 2 9B | Google DeepMind | 69.3% | instruct-cited-phi35moe | official ↗ |
| #25 | Mistral NeMo | Mistral AI | 68.1% | instruct-cited-phi35moe | official ↗ |
| #26 | Llama Nemotron Nano 8B | NVIDIA | 66.1% | reasoning-off-0-shot-pass@1 | official ↗ |
| #27 | Qwen2-7B | Qwen | 65.9% | base-from-hf-readme | official ↗ |
| #28 | Yi-1.5-34B | 01.AI | 65.5% | cited-qwen2-57b | official ↗ |
| #29 | Nemotron-H 8B | NVIDIA | 65.37% | base-sanitized-3-shot | official ↗ |
| #30 | Qwen1.5-32B | Qwen | 64.2% | cited-qwen2-57b | official ↗ |
| #31 | Mixtral 8x7B | Mistral AI | 63.9% | cited-qwen2-57b | official ↗ |
| #32 | TeleChat2-7B | China Telecom | 62.6% | from-hf-readme | official ↗ |
| #33 | Qwen1.5-110B | Qwen | 58.1% | cited-telechat2 | official ↗ |
| #34 | Phi-2 | Microsoft | 55% | cited-qwen2 | official ↗ |
| #35 | Llama 3-8B | Meta AI | 53.9% | base-cited-qwen2-7b | official ↗ |
| #36 | Qwen1.5-7B | Qwen | 51.6% | base-cited-qwen2-7b | official ↗ |
| #37 | Mistral 7B | Mistral AI | 51.1% | base-cited-qwen2-7b | official ↗ |
| #38 | Gemma 7B | Google DeepMind | 50.6% | base-cited-qwen2-7b | official ↗ |
| #39 | TeleChat2-3B | China Telecom | 47% | from-hf-readme | official ↗ |
| #40 | Qwen2-1.5B | Qwen | 37.4% | base-from-hf-readme | official ↗ |
| #41 | Gemma 2B | Google DeepMind | 29.2% | cited-qwen2 | official ↗ |
| #42 | Qwen2-0.5B | Qwen | 22% | base-from-hf-readme | official ↗ |
Frequently asked questions about MBPP
What is the MBPP benchmark?
Mostly Basic Python Programming - 1,000 Python problems for entry-level program synthesis.
How is the MBPP benchmark scored?
MBPP is scored using the pass@1 (%) metric, where a higher score is better. The maximum achievable score is 100.000. GenAIList aggregates reported scores from model providers and papers into a single ranked leaderboard.
Which AI model scores highest on MBPP?
As of the latest reported scores on GenAIList, Ling-1T achieves the highest result on MBPP with a score of 96.87%.
Is a higher MBPP score better?
Yes. On MBPP a higher score indicates better performance, so models near the top of the leaderboard are the strongest.