IO
Institute of Science and Technology Austria (ISTA)
Language model · United States ·Oct 2023

QMoE: compressed SwitchTransformer

Language Open (Restricted)
1.6T
Parameters

About QMoE: compressed SwitchTransformer

QMoE: compressed SwitchTransformer is an AI model developed by Institute of Science and Technology Austria (ISTA), in the language category, released in 2023, made available as an open-weights model with 1571B params.

On this page you'll find QMoE: compressed SwitchTransformer's full specifications. Review provider pricing and benchmark scores below, or compare QMoE: compressed SwitchTransformer head-to-head with other language models.

→ More models by Institute of Science and Technology Austria (ISTA) → All language models → AI benchmarks leaderboard

Links

Announcement ↗

Run QMoE: compressed SwitchTransformer locally — what it takes

QMoE: compressed SwitchTransformer (1,571B) is too large to run on a single consumer or Apple machine. At Q4 its weights alone are about 880 GB — it needs a multi-GPU / datacenter setup:

  • ≈ 4× AMD Instinct MI325X
  • ≈ 5× NVIDIA B200

Lower quantization (Q3/Q2) or CPU + system-RAM offload (e.g. ktransformers) reduces the requirement at the cost of speed.

Software support

·ExLlamaV2 ·KoboldCpp ·LM Studio ·MLX ·Ollama ·SGLang ·TensorRT-LLM ·llama.cpp ·vLLM ?KTransformers ?text-generation-webui

✓ measured · · compatible · — not supported. Informational only — speed is hardware-based.

Community

Would you run QMoE: compressed SwitchTransformer again?
No votes yet — be the first.
Did it run? — community reports by hardware

No reports yet. Own this model on some hardware? Be the first to confirm it runs.

Discussion

No comments yet. Start the discussion.

More models like QMoE: compressed SwitchTransformer

AI
DMind-1
 AI
COLLECTIVE-1
 AI
Cogito 1 series
 AI
Euler (Euler大模型)
· 2.5B
 AI
SmolLM-1.7B
· 1.7B
 AI
Llama-3-70B-Synthia-v3.5
· 70B

Frequently asked questions

What is QMoE: compressed SwitchTransformer? ▶

QMoE: compressed SwitchTransformer is an AI model developed by Institute of Science and Technology Austria (ISTA), in the language category, released in 2023. It is tracked on GenAIList with its specifications, benchmark scores and provider pricing.

Who created QMoE: compressed SwitchTransformer? ▶

QMoE: compressed SwitchTransformer was developed by Institute of Science and Technology Austria (ISTA) and released in 2023.

Is QMoE: compressed SwitchTransformer open source or proprietary? ▶

QMoE: compressed SwitchTransformer ships as an open-weights model: you can download and self-host the weights, though the licence may place some restrictions on use.

How much does QMoE: compressed SwitchTransformer cost? ▶

Pricing for QMoE: compressed SwitchTransformer depends on the provider. See the providers table on this page for the latest API rates.

How does QMoE: compressed SwitchTransformer perform on benchmarks? ▶

Benchmark scores for QMoE: compressed SwitchTransformer are listed on this page as they are published. You can compare it head-to-head with other models on the GenAIList compare tool.

No videos yet. Know a good one? Add it below.

Know a great video about QMoE: compressed SwitchTransformer?