Kairong Luo 罗开荣

Technical Report · 2026 Open pretraining

Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090

Kairong Luo^*, Jiarui Cui^*, Yaorui Yin^*, Shengqi Chen^*, Yiming Yang, Linxiang Gao, Yanmohan Wang, Chengxia Li, Mingzhe Zhang, Kaifeng Lyu^†, Wenguang Chen^†

^* Co-first authors ^† Co-corresponding authors

Training a 2B language model on consumer GPUs becomes practical when data, optimization, and FP8 systems are designed together.

NeurIPS 2026 Spotlight Data quality

Open LLM Pretraining Guided by Quantile Benchmarking Heterogeneous Web Corpora

(derived from Kaiyuan-2B)

Kairong Luo^*, Zhenbo Sun^*, Xinyu Shi, Shengqi Chen, Bowen Yu, Yunyi Chen, Chenyi Dang, Hengtao Tao, Hui Wang, Fangming Liu, Kaifeng Lyu, Wenguang Chen

^* Co-first authors

Comparing web corpora across quality quantiles helps choose better data mixtures for open LLM pretraining.

ICLR 2026 Oral Training dynamics

How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining

Kairong Luo, Zhenbo Sun, Haodong Wen, Xinyu Shi, Jiarui Cui, Chenyi Dang, Kaifeng Lyu^†, Wenguang Chen^†

^† Co-corresponding authors

Learning-rate decay weakens late-arriving high-quality data, while moderate decay and curriculum model averaging recover its training benefits.

ICLR 2025 Scaling laws

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules

Kairong Luo, Haodong Wen, Shengding Hu, Zhenbo Sun, Zhiyuan Liu, Maosong Sun, Kaifeng Lyu^†, Wenguang Chen^†

^† Co-corresponding authors

A multi-power law predicts loss under unseen learning-rate schedules, turning schedule selection from trial and error into a guided search.