From the source
- A preview paper from Dharma AI compares FLOP-based estimates of fine-tuning time against learned predictors, using 272 production supervised fine-tuning runs on shared
NVIDIA H200 GPUs.
- The authors say the standard compute-only estimate explains little of what they observed, with R² = 0.31 and a mean absolute error of 12.41 hours.
- A CatBoost regressor that sees only information available before dispatch reached R² = 0.98, a mean absolute error of 1.71 h and a median error of 0.51 h in 5-fold cross-validation.
- The gap comes from training-specific signals the closed-form estimate cannot use, such as evaluation cadence, checkpoint policy, per-GPU load and fine-tuning regime, and from nonlinear interactions between them.
- The runs cover several model families up to 15 billion parameters, LoRA and full fine-tuning, on 1, 2 or 4 GPUs drawn from a pool of 8 to 32 H200s, with durations from 1.6 to 172.4 hours.
- Sweeping MFU from 0.25 to 0.65 did not fix the FLOP baseline: the best setting, 30%, still leaves a mean error above 12 hours on jobs averaging 23 hours.
- The full version is to appear on arXiv.






