New: connect Claude & other AIs to GenAIList over MCP — research the catalog and contribute to the shared knowledge base. Learn how →
ARC-Challenge benchmark
AI model leaderboard for the ARC-Challenge benchmark. Compare how large language models score on ARC-Challenge, see the full ranking, and understand what this AI benchmark measures. Nemotron-H 56B currently leads with 94.97. AI2 Reasoning Challenge (Challenge set) — grade-school science multiple-choice questions, hard subset.
Leaderboard
| # | Model | Organization | Score | Variant | Source |
|---|---|---|---|---|---|
| #1 | Nemotron-H 56B | NVIDIA | 94.97 | base-25-shot | official ↗ |
| #2 | Nemotron-H 47B | NVIDIA | 94.6 | base-25-shot | official ↗ |
| #3 | GPT-4o mini | OpenAI | 93.5 | cited-phi35moe | official ↗ |
| #4 | Gemini 1.5 Flash (Sep 2024) | Google DeepMind | 92.8 | cited-phi35moe | official ↗ |
| #5 | Phi-3.5-MoE | Microsoft | 91 | instruct-10-shot | official ↗ |
| #6 | Qwen2.5-7B | Qwen | 90.1 | cited-phi4-mini | official ↗ |
| #7 | Gemma 2 9B | Google DeepMind | 89.8 | instruct-cited-phi35moe | official ↗ |
| #8 | Qwen2.5-7B | Qwen | 89.5 | base-cited-olmo2 | official ↗ |
| #9 | Gemma 2 9B | Google DeepMind | 89.5 | base-cited-olmo2 | official ↗ |
| #10 | Nemotron-H 8B | NVIDIA | 88.74 | base-25-shot | official ↗ |
| #11 | OLMo 2 32B | Allen Institute for AI | 87.6 | base-from-hf-readme | official ↗ |
| #12 | Mistral NeMo | Mistral AI | 84.8 | instruct-cited-phi35moe | official ↗ |
| #13 | phi-3.5-mini | Microsoft | 84.6 | cited-phi4-mini | official ↗ |
| #14 | Phi-4 Mini | Microsoft | 83.7 | instruct-10-shot | official ↗ |
| #15 | Qwen2.5-3B | Qwen | 82.6 | cited-phi4-mini | official ↗ |
| #16 | Ministral 8B | Mistral AI | 80.3 | cited-phi4-mini | official ↗ |
| #17 | Llama 3.1-8B | Meta AI | 79.5 | base-cited-olmo2 | official ↗ |
| #18 | Llama 3.2 3B | Meta AI | 76.1 | cited-phi4-mini | official ↗ |
| #19 | Llama 2-13B | Meta AI | 67.3 | base-cited-olmo2 | official ↗ |
| #20 | Mixtral 8x7B | Mistral AI | 66 | cited-qwen2-57b | official ↗ |
| #21 | Yi-1.5-34B | 01.AI | 65.6 | cited-qwen2-57b | official ↗ |
| #22 | Jamba | AI21 Labs | 64.4 | cited-qwen2-57b | official ↗ |
| #23 | Qwen2-57B-A14B | Qwen | 64.1 | base-from-hf-readme | official ↗ |
| #24 | Qwen1.5-32B | Qwen | 63.6 | cited-qwen2-57b | official ↗ |
| #25 | Marin 8B | Marin | 63.1 | base-from-hf-readme | official ↗ |
| #26 | Falcon3-7B | Technology Innovation Institute | 62.6 | instruct-25-shot | official ↗ |
| #27 | Falcon Mamba | Technology Innovation Institute | 62.03 | base-from-hf-readme | official ↗ |
| #28 | Phi-2 | Microsoft | 61.1 | cited-qwen2 | official ↗ |
| #29 | Gemma 7B | Google DeepMind | 61.09 | cited-falcon-mamba | official ↗ |
| #30 | Qwen2-7B | Qwen | 60.6 | base-from-hf-readme | official ↗ |
| #31 | Llama 3-8B | Meta AI | 60.24 | base-cited-falcon-mamba | official ↗ |
| #32 | Mistral 7B | Mistral AI | 59.98 | v0-1-cited-falcon-mamba | official ↗ |
| #33 | DCLM 7B | Apple | 59.9 | from-hf-readme | official ↗ |
| #34 | Llama 3.1-8B | Meta AI | 58.9 | base-cited-marin | official ↗ |
| #35 | Llama 3.1-8B | Meta AI | 58.6 | instruct-25-shot-cited-falcon3 | official ↗ |
| #36 | Qwen2.5-7B | Qwen | 57.8 | instruct-25-shot-cited-falcon3 | official ↗ |
| #37 | Gemma 2B | Google DeepMind | 48.5 | cited-qwen2 | official ↗ |
| #38 | Qwen2-1.5B | Qwen | 43.9 | base-from-hf-readme | official ↗ |
| #39 | Qwen2-0.5B | Qwen | 31.5 | base-from-hf-readme | official ↗ |
Frequently asked questions about ARC-Challenge
What is the ARC-Challenge benchmark?
AI2 Reasoning Challenge (Challenge set) — grade-school science multiple-choice questions, hard subset.
How is the ARC-Challenge benchmark scored?
ARC-Challenge is scored using the accuracy metric, where a higher score is better. GenAIList aggregates reported scores from model providers and papers into a single ranked leaderboard.
Which AI model scores highest on ARC-Challenge?
As of the latest reported scores on GenAIList, Nemotron-H 56B achieves the highest result on ARC-Challenge with a score of 94.97.
Is a higher ARC-Challenge score better?
Yes. On ARC-Challenge a higher score indicates better performance, so models near the top of the leaderboard are the strongest.