Can I read Qwen2.5 Technical Report on EtoBox?

Qwen2.5 Technical Report by Qwen; :; Yang, An; Yang, Baosong; Zhang, Beichen; Hui, Binyuan; Zheng, Bo; Yu, Bowen; Li, Chengyuan; Liu, Dayiheng; Huang, Fei; Wei, Haoran; Lin, Huan; Yang, Jian; Tu, Jianhong; Zhang, Jianwei; Yang, Jianxin; Yang, Jiaxi; Zhou, Jingren; Lin, Junyang; Dang, Kai; Lu, Keming; Bao, Keqin; Yang, Kexin; Yu, Le; Li, Mei; Xue, Mingfeng; Zhang, Pei; Zhu, Qin; Men, Rui; Lin, Runji; Li, Tianhao; Tang, Tianyi; Xia, Tingyu; Ren, Xingzhang; Ren, Xuancheng; Fan, Yang; Su, Yang; Zhang, Yichang; Wan, Yu; Liu, Yuqiong; Cui, Zeyu; Zhang, Zhenru; Qiu, Zihan is a scholarly article available to read on EtoBox.

What is Qwen2.5 Technical Report about?

In this report, we introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs. Compared to previous iterations, Qwen 2.5 has been significantly improved during both the pre-training and post-training stages. In terms of pre-training, we have scaled the high-quality pre-training datasets from the previous 7 trillion tokens to 18 trillion tokens. This provides a strong foundation for common sense, expert knowledge, and reasoning capabilities. In terms of post-training, we implement intricate supervised finetuning with over 1 million samples, as well as multistage reinforcement learning. Post-training techniques enhance human preference, and notably improve long text generation, structural data analysis, and instruction following. To handle diverse and varied use cases effectively, we present Qwen2.5 LLM series in rich sizes. Open-weight offerings include base and instruction-tuned models, with quantized versions available. In addition, for hosted solutions, the proprietary models currently include two mixture-of-experts (MoE) variants: Qwen2.5-Turbo and Qwen2.5-Plus, both available from Alibaba Cloud Model Studio. Qwen2.5 has demonstrated

Author

Qwen; :; Yang, An; Yang, Baosong; Zhang, Beichen; Hui, Binyuan; Zheng, Bo; Yu, Bowen; Li, Chengyuan; Liu, Dayiheng; Huang, Fei; Wei, Haoran; Lin, Huan; Yang, Jian; Tu, Jianhong; Zhang, Jianwei; Yang, Jianxin; Yang, Jiaxi; Zhou, Jingren; Lin, Junyang; Dang, Kai; Lu, Keming; Bao, Keqin; Yang, Kexin; Yu, Le; Li, Mei; Xue, Mingfeng; Zhang, Pei; Zhu, Qin; Men, Rui; Lin, Runji; Li, Tianhao; Tang, Tianyi; Xia, Tingyu; Ren, Xingzhang; Ren, Xuancheng; Fan, Yang; Su, Yang; Zhang, Yichang; Wan, Yu; Liu, Yuqiong; Cui, Zeyu; Zhang, Zhenru; Qiu, Zihan

Published

2024

Language

EN