AI benchmarks leaderboard

Aider editor benchmark covering Python, JavaScript, Rust, Go, C++, and more — measures real-world code-editing skill.

Codeforces competitive programming rating estimated from solving rates.

Full-stack code generation benchmark covering 11 real-world programming domains.

Python function-completion benchmark introduced by OpenAI to measure functional correctness of generated code.

EvalPlus extension of HumanEval with ~80x more tests per problem to expose buggy completions.

Contamination-resistant coding benchmark continuously updated from new competitive-programming problems.

LiveCodeBench v5 — 599 programming problems aggregated 2024-07 to 2025-02.

LiveCodeBench v6 — newer cutoff continuing the LCB approach.

Mostly Basic Python Programming - 1,000 Python problems for entry-level program synthesis.

EvalPlus extension of MBPP with ~35x more tests per problem.

Real GitHub issues from popular Python repos that the model must resolve with a multi-file patch.

Harder professional-grade SWE-bench variant with multi-file refactors and stricter test suites.

Human-verified subset of SWE-bench filtered for unambiguous task descriptions and reliable test suites.

Scientific code generation benchmark across physics, biology, chemistry, mathematics.

Web-browsing benchmark of hard real-world questions answerable via online research.

Chinese-language counterpart to BrowseComp — agentic web-browsing tasks requiring multi-page information synthesis.

Financial-domain agentic search benchmark — Tier-3 difficulty subset on multi-source financial QA.

General AI Assistants benchmark — multi-step real-world tasks requiring web search, code, and reasoning.

Lifelong robot-manipulation benchmark for VLA models.

Benchmark for MCP-tool calling correctness on isolated, well-specified single-server tasks.

Agentic benchmark spanning Model Context Protocol tool use across heterogeneous MCP servers.

Computer-use benchmark — real desktop environment tasks across Linux/macOS, agents control GUI.

Verified split of OSWorld — real computer-use GUI agent tasks.

Bimanual robot-manipulation benchmark with domain randomization.

Agentic web-browsing benchmark — sealed (private) hard subset for clean leaderboard tracking.

Airline customer-service track of τ-bench — multi-turn agentic tool-use.

Retail customer-service track of τ-bench — multi-turn agentic tool-use.

Long-horizon shell/agent benchmark testing multi-step terminal automation.

Revised Terminal-Bench 2 suite of real terminal/agentic engineering tasks.

Ten-task agentic suite probing varied tool-use patterns: search, code, calculation, scheduling, etc.

Agentic web-navigation QA — agent must traverse web pages to find answers to multi-hop questions.

Agentic deep-search benchmark from xbench — evaluates multi-source web research and synthesis.

October-2025 release of xbench-DeepSearch — refreshed question set with new contamination-resistant items.

Multi-turn agent benchmark simulating realistic customer-service tool environments.

Two-agent dual-control variant of τ-Bench — dialogue benchmark where both user and assistant are role-played, increasing realism.

Berkeley Function Calling Leaderboard v3 — agentic tool-use benchmark across single, parallel, and multi-step calls.

Berkeley Function-Calling Leaderboard, version 4.

Agentic tool-use benchmark over Model Context Protocol servers.

Original Abstraction and Reasoning Corpus — François Chollet's test of fluid, sample-efficient abstraction.

Abstraction and Reasoning Corpus v2 — visual abstract-pattern puzzles designed to resist memorisation.

Private/holdout split of ARC-AGI-2 — used for system-card evaluations.

Open-ended logic-puzzle benchmark — automatically constructed deduction problems with verifiable solutions.

BIG-Bench Extra Hard — successor to BBH that probes deeper reasoning on harder, more diverse tasks.

23 challenging tasks from BIG-Bench where prior LLMs fell below the human baseline.

Discrete reasoning over paragraphs — reading comprehension with arithmetic.

Quantitative finance reasoning benchmark — multi-step problems involving valuation, risk, and accounting.

198 hardest expert-validated graduate-level science questions resistant to web search.

Commonsense sentence-completion benchmark — pick the most plausible ending out of four for a short narrative.

~3,000 expert-curated questions across 100+ subjects designed to remain hard for frontier AI.

Multilingual logical-reasoning QA benchmark — Chinese LogiQA + multilingual translations.

Physical Interaction QA — commonsense benchmark on physical-world reasoning.

Large-scale Winograd schema challenge — pronoun coreference resolution requiring commonsense.

1,000 logic-grid puzzles ("zebra puzzles") covering 6 difficulty tiers.

Chinese alignment benchmark — multi-dimensional rubric judging open-ended LLM responses on Chinese tasks.

GPT-4-judge LLM preference benchmark — length-controlled.

Crowdsourced hard-prompt Arena set graded by GPT-4 judge.

Arena-Hard v2 — automatic preference benchmark of 500 hard prompts judged by an LLM, used to estimate model rankings against the LMArena leaderboard.

Multi-discipline Chinese exam benchmark across 52 subjects.

Chinese multiple-choice reading comprehension benchmark.

Chinese Classical Poetry Matching benchmark.

Chinese Winograd Schema Challenge — coreference / commonsense in Chinese.

Chinese Machine Reading Comprehension — span-extraction from Chinese passages.

Open-ended creative-writing benchmark — LLM judge scores prose quality, originality, and instruction adherence.

Long-context evaluation suite covering retrieval, summarization, and reasoning.

Verifiable instruction-following benchmark with 25 instruction types.

Contamination-resistant general benchmark with monthly question rotation across reasoning, coding, math, language.

Long-context benchmark of 21 tasks at contexts up to 100K tokens.

Long-context understanding benchmark across 8 task types and contexts up to 2M tokens.

Long-form generation benchmark — extended composition tasks judged by LLM rubric.

57-subject multiple-choice exam covering elementary to professional knowledge.

Harder MMLU successor with 10-option questions and reasoning-heavy items.

Re-annotated, error-corrected version of MMLU.

Multilingual MMLU professionally translated into 14 languages.

Multilingual instruction-following benchmark — IFEval-style prompts in 8 languages.

Multi-turn instruction-following benchmark, English subset.

Multi-turn conversational benchmark testing instruction retention across 273 dialogues.

Multi-round coreference / retrieval benchmark at 128K context length.

Multi-round coreference / retrieval benchmark at 1M context length.

RACE benchmark — high-school subset with longer, more complex passages.

Reading Comprehension from Examinations — middle-school subset.

Synthetic long-context benchmark probing retrieval, multi-hop, and aggregation at varying lengths.

Benchmark for LLM-based voice assistants on spoken instructions.

Long-form writing benchmark — open-ended creative tasks judged by LLM rubric.

American Invitational Mathematics Examination 2024 — 30 olympiad problems.

American Invitational Mathematics Examination 2025 — 30 hard high-school olympiad problems.

2026 American Invitational Mathematics Examination problems.

Chinese elementary-school math test — multi-step word problems from Chinese curriculum.

Chinese National Mathematical Olympiad 2024 — high-difficulty Chinese olympiad problems.

8,500 grade-school word problems requiring multi-step arithmetic reasoning.

HMMT 2025 — combined Feb + Nov contest score reported as a single number in some papers.

Harvard-MIT Mathematics Tournament February 2025 — 30 algebra/number-theory/combinatorics/geometry questions.

Harvard-MIT Mathematics Tournament November 2025 contest used as a frontier-math reasoning benchmark for LLMs.

International Mathematical Olympiad answer-only benchmark — graded on numeric/short answers from IMO problems.

12,500 high-school competition math problems with step-by-step solutions.

500-problem subset of MATH curated by OpenAI for benchmark stability.

Multilingual GSM8K — translations of grade-school math problems into 10 languages.

Multilingual AIME 2024 — translations of AIME 2024 problems into multiple languages, tests cross-lingual math.

Comprehensive olympiad-level mathematics benchmark covering 33 sub-domains and 10 difficulty tiers.

173-problem hardest subset of Omni-MATH at olympiad-grade difficulty.

Multilingual mathematical reasoning benchmark — English split.

Undergraduate-level mathematics benchmark — calculus, linear algebra, probability, etc.

Visual question-answering over real-world charts and infographics.

Visual question answering over scanned business documents.

11,500 college-level multi-modal questions across 30 disciplines that require visual reasoning.

Standard split of MMMU-Pro — 10-option multiple-choice text+image questions across college-level disciplines.

Vision split of MMMU-Pro — questions are rendered as screenshots so the model must read the image text.

6,141 mathematical reasoning problems with visual context (charts, graphs, geometry).

High-resolution professional GUI grounding benchmark.

Comprehensive multimodal video-understanding benchmark.

Chinese factuality benchmark — Chinese-language counterpart of SimpleQA.

Realistic patient/clinician dialogues for health-domain LLM evaluation.

Hardest subset of HealthBench targeting challenging health conversations.

Short-answer factuality benchmark with 4,326 fact-seeking questions.

817 questions designed to elicit common human misconceptions and test factual honesty.

Real-world Java vulnerability benchmark used in Anthropic's Project Glasswing cybersecurity evaluation.

Benchmark for AI agents on real-world cybersecurity tasks.

Aider's code-editing benchmark — derived from 133 Exercism Python exercises, edits run in a sandbox.

Benchmark for general code generation — 1140 programming tasks across 139 libraries, evaluated by full unit tests.

Code reasoning, understanding and execution benchmark — predict input or output of short Python functions.

EvalPlus harness — augments HumanEval/MBPP with adversarial test cases that expose surface-level passes.

Multilingual software-engineering benchmark — extension of SWE-bench across multiple languages.

Multi-language code-generation benchmark — translates HumanEval/MBPP into 18+ programming languages.

Online-judge competitive-programming benchmark — C++ contest problems with strict test cases.

Multilingual extension of SWE-Bench — real GitHub issues across Python, Java, JavaScript, Go, etc.

Human-centric exam benchmark — Chinese gaokao, SAT, LSAT, lawyer-qualification etc.

AI2 Reasoning Challenge (Challenge set) — grade-school science multiple-choice questions, hard subset.

AI2 Reasoning Challenge (Easy set) — grade-school science questions answerable by simple retrieval.

Chinese MMLU — 67-subject Chinese-language multiple-choice exam benchmark.

Multilingual knowledge benchmark — covers 65 languages and tests world-knowledge breadth.

Graduate-level cross-discipline benchmark spanning 285 disciplines — extension of GPQA.

Closed-book question answering benchmark of trivia questions sourced from quiz websites.

Long-context reasoning benchmark — multi-hop questions over Wikipedia evidence requiring information aggregation.

Repository-level long-context code understanding — find a specified function within a large code repo dump.

Physics-problem reasoning benchmark — high-school and undergraduate physics questions with numeric answers.