From the source
ServiceNow-AI describes migrating their RL training pipeline from vLLM V0 to V1, identifying and fixing four issues (logprob semantics, runtime defaults, inflight weight updates, fp32 lm_head) to achieve training parity.
vLLM V0 to V1: Correctness Before Corrections in RL
From the source
vLLM V1 matched our vLLM V0 reference after we fixed four things: processed rollout logprobs, V1-specific runtime defaults, the inflight weight-update path, and the fp32 lm_head used for the final projection.
huggingface.co
