ICLR 2026

Larger Datasets Can Be Repeated More: A Theoretical Analysis of Multi-Epoch Scaling in Linear Regression

Tingkai Yan^*, Haodong Wen^*, Binghui Li^*, Kairong Luo, Wenguang Chen, Kaifeng Lyu

^* Co-first authors

Theory and LLM experiments show why larger datasets can be repeated more: useful reuse depends on both dataset size and distribution.

LLM experiments show larger datasets sustain more useful epochs before saturation. — Data reuse in LLM experiments · Fig. 1

My contribution: experimental part
OpenReview
arXiv

This work analyzes how repeating a dataset for multiple epochs changes data scaling laws in linear regression and shows that larger datasets can be repeated more before the marginal benefit saturates.

Tingkai Yan, Haodong Wen, Binghui Li, Kairong Luo, Wenguang Chen, and Kaifeng Lyu. (2026). "Larger Datasets Can Be Repeated More: A Theoretical Analysis of Multi-Epoch Scaling in Linear Regression." ICLR 2026.