From the source
Cohere researchers say standard reasoning models cannot reliably report how likely their answers are to be correct, and propose a training method called CALIBER that teaches a model to give two separate confidence estimates.
- The first estimate is made before reasoning begins and answers how likely the model is to solve the question at all; the second is made after an answer is produced and answers how likely that particular answer is to be correct.
- CALIBER trains the model to report both in one response, alongside its reasoning and answer, using different targets for each: initial confidence is trained on how often the model solves that question across several attempts, while final confidence is trained on whether the individual answer is correct.
- The researchers trained 7B and 30B models on 30,000 math problems and evaluated them on 1,000 held-out problems plus three question-answering benchmarks.
- On BigMathDigits, the 7B model's Expected Calibration Error fell from 0.059 for the strongest single-confidence baseline to 0.028 for CALIBER, a 52.5% reduction, and CALIBER also posted the best AUROC (0.806) and Brier score (0.130).
- The team argues that a single confidence score is not enough, because a model that reports 70% every time can be perfectly calibrated yet give a user no way to decide which answers to check first.
