- Params
- 107B
- Context
- β
- Released
- Jun 2025
The distributed training of foundation models, particularly large language models (LLMs), demands a high level of communication. Consequently, it is highly depe.
Full DiLoCoX (Qwen1.5-107B on WT-103) specs β