From the source
From the source
Three methods for assigning per-layer precision in NVFP4 quantization.
Baseten's post explains three approaches to assigning precision formats per layer in NVFP4 quantization: heuristic-based algorithms, isolated-layer sensitivity scoring, and SaturationQuant, which incorporates saturation so scores can be summed across a mixed-precision recipe.
The post also covers calibration steps and the role of block scales.
From the source
A high-quality NVFP4 quantization comes down to choosing which layers can run at 4 bits and which need more precision. This post covers three ways to assign a format to each layer: heuristic-based algorithms decide by architecture; isolated-layer sensitivity scoring measures one layer at a time, so its scores can't be added; and SaturationQuant builds saturation into each score, so they can be added across a mixed-precision recipe.
baseten.co
