- Params
- 1.6T
- Context
- β
- Released
- Oct 2023
Mixture-of-Experts (MoE) architectures offer a general solution to the high inference costs of large language models (LLMs) via sparse routing, bringing faster .
Full QMoE: compressed SwitchTransformer specs β