Single-Rollout Optimization

SAO's headline move: one rollout per prompt instead of GRPO's group, fed to training the instant it finishes — cutting off-policy drift and fitting online/agentic settings that only ever give one trajectory per prompt; the catch is REINFORCE-like variance, so it pays for the missing group-baseline by re-embracing a value model and spending its whole engineering budget on making the critic stable (faster value updates, frozen-attention critic, skip-observation GAE, scaled value pretraining)