// COMPARISON

Llama 3.1-8B

A head-to-head benchmark comparison of Llama 3.1-8B across 24 evaluations.

The models

MA

Llama 3.1-8B

Meta AI
Language Open (Restricted)
Params
8B
Context
—
Released
Jul 2024

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd o.

Full Llama 3.1-8B specs →

Benchmark comparison

Benchmark Llama 3.1-8B
code
HumanEval 86.3%
HumanEval+ 82.9%
MBPP 69.4%
general
AlpacaEval 2 27.22%
ArenaHard 36.43%
IFEval 80.6%
LiveBench 28.3
MMLU 71.2%
MMLU-Pro 44%
MMMLU 56.2%
knowledge
AGIEval 51.3
ARC-Challenge 79.5
ARC-Easy 85.8
TriviaQA 80.3
math
GSM8K 83.4%
MATH 47.6%
MGSM 56.7
reasoning
BIG-Bench Hard 72.66%
DROP 61.5%
GPQA Diamond 33.5%
HellaSwag 81.9
PIQA 81.2
Winogrande 82.9
safety
TruthfulQA 69.2%

Best result per row highlighted in cyan. Each benchmark links to its definition and sources; each model links to its full scorecard.

Frequently asked questions

Which of these models is best for coding? ▶

Llama 3.1-8B has the strongest result on HumanEval among the models compared here. See the code-category rows in the table for the full picture.

How many benchmarks are compared? ▶

This comparison covers 24 benchmarks on which at least one of the selected models has a published score.

Where do the benchmark scores come from? ▶

Scores are aggregated from official model cards, technical reports and standard public evaluations, and link back to each benchmark's source. They are updated as new results are published.

Explore further

→ Llama 3.1-8B → AI benchmarks leaderboard → Build a different comparison
Build a different comparison

Models (0 selected)

Benchmarks (auto)

Select at least two models.