Technical Report · 2025

PCMind-2.1-Kaiyuan-2B Technical Report

Kairong Luo, Zhenbo Sun, Xinyu Shi, Shengqi Chen, Bowen Yu, Yunyi Chen, Chenyi Dang, Hengtao Tao, Hui Wang, Fangming Liu, Kaifeng Lyu, Wenguang Chen

Technical Report Related paper Model Dataset Kaiyuan-Spark kaiyuan-mindformers

An end-to-end open 2B pretraining stack releases the data, processing tools, training code, and model weights needed for reproducible research.

Kaiyuan-2B performance compared with fully open and open-weight language models. — Model performance · Fig. 1

Technical Report
Related conference paper: Open LLM Pretraining Guided by Quantile Benchmarking Heterogeneous Web Corpora, NeurIPS 2026 Spotlight.
Model
Dataset
Kaiyuan-Spark
kaiyuan-mindformers

PCMind-2.1-Kaiyuan-2B is a fully open 2B language model focused on efficient and effective pretraining under resource constraints. We release the full stack around it, including model weights, dataset, preprocessing framework, and training framework.

I led the project and am a co-first author, listed first.

Kairong Luo, Zhenbo Sun, Xinyu Shi, Shengqi Chen, Bowen Yu, Yunyi Chen, Chenyi Dang, Hengtao Tao, Hui Wang, Fangming Liu, Kaifeng Lyu, and Wenguang Chen. (2025). "PCMind-2.1-Kaiyuan-2B Technical Report." Technical Report.