Chronos: A Unified Framework for Predicting Training Time and Convergence in Deep Learning

Abstract

Training deep neural networks requires extensive experimentation to estimate how long it will take, and how much it will cost, for a model to reach convergence. Chronos is a unified framework for pre-training prediction of both the convergence horizon and the resulting end-to-end training time and cost for a target configuration. Unlike approaches that estimate these quantities from observations collected during an executing training job, Chronos uses initialization-time probe signals together with computational, optimization, and hardware descriptors to make predictions before full training begins. Across a diverse spectrum of architectures, from lightweight models such as MobileNetV2 and DeiT-Tiny to deeper and more complex networks, including ViT, ResNet-50, and DenseNet-121, Chronos achieves an average prediction error of 13.7% MAPE for iteration-level execution time and 22.1% MAPE for convergence estimation.

Publication
Transactions on Machine Learning Research (TMLR)