- Params
- 1.5B
- Context
- β
- Released
- Aug 2021
Recent works have demonstrated great success in pre-training large-scale autoregressive language models on massive GPUs. To reduce the wall-clock training time,.
Full GPT-2 (1.5B, Curriculum Learning 45K) specs β