NVIDIA
NVIDIA
Language model · United States ·Jun 2024

Nemotron-4 340B

Language Open (Restricted)
340B
Parameters
5
Benchmarks

About Nemotron-4 340B

Nemotron-4 340B is an AI model developed by NVIDIA, in the language category, released in 2024, made available as an open-weights model with 340B params.

On this page you'll find Nemotron-4 340B's full specifications, including 5 benchmark results. Review provider pricing and benchmark scores below, or compare Nemotron-4 340B head-to-head with other language models.

→ More models by NVIDIA → All language models → AI benchmarks leaderboard

Links

Announcement ↗ Hugging Face ↗

Run Nemotron-4 340B locally — what it takes

Nemotron-4 340B (340B) is too large to run on a single consumer or Apple machine. At Q4 its weights alone are about 190 GB — it needs a multi-GPU / datacenter setup:

  • ≈ 1× AMD Instinct MI325X
  • ≈ 2× NVIDIA B200

Lower quantization (Q3/Q2) or CPU + system-RAM offload (e.g. ktransformers) reduces the requirement at the cost of speed.

Software support

·ExLlamaV2 ·KoboldCpp ·LM Studio ·MLX ·Ollama ·SGLang ·TensorRT-LLM ·llama.cpp ·vLLM ?KTransformers ?text-generation-webui

✓ measured · · compatible · — not supported. Informational only — speed is hardware-based.

Datacenter GPU
24.3 tok/s
1 option →

Community

Would you run Nemotron-4 340B again?
No votes yet — be the first.
Did it run? — community reports by hardware

No reports yet. Own this model on some hardware? Be the first to confirm it runs.

Discussion

No comments yet. Start the discussion.

Nemotron-4 340B benchmark scores▶
Compare on the benchmark table →
Benchmark Category Score Variant
HumanEval code 73.2% 0-shot-instruct-from-hf-readme
MBPP code 75.4% 0-shot-instruct-from-hf-readme
ArenaHard general 54.2% instruct-from-hf-readme
MMLU general 78.7% 0-shot-instruct-from-hf-readme
GSM8K math 92.3% 0-shot-instruct-from-hf-readme

More models like Nemotron-4 340B

AI
DMind-1
 AI
COLLECTIVE-1
 AI
Cogito 1 series
 AI
Euler (Euler大模型)
· 2.5B
 AI
SmolLM-1.7B
· 1.7B
 AI
Llama-3-70B-Synthia-v3.5
· 70B

Frequently asked questions

What is Nemotron-4 340B? ▶

Nemotron-4 340B is an AI model developed by NVIDIA, in the language category, released in 2024. It is tracked on GenAIList with its specifications, benchmark scores and provider pricing.

Who created Nemotron-4 340B? ▶

Nemotron-4 340B was developed by NVIDIA and released in 2024.

Is Nemotron-4 340B open source or proprietary? ▶

Nemotron-4 340B ships as an open-weights model: you can download and self-host the weights, though the licence may place some restrictions on use.

How much does Nemotron-4 340B cost? ▶

Pricing for Nemotron-4 340B depends on the provider. See the providers table on this page for the latest API rates.

How does Nemotron-4 340B perform on benchmarks? ▶

Nemotron-4 340B is benchmarked across 5 evaluations on GenAIList, including HumanEval (73.2%). See the full benchmark table below and compare it with other models.

No videos yet. Know a good one? Add it below.

Know a great video about Nemotron-4 340B?