Group Relative Policy Optimization (GRPO)

DeepSeek's critic-free RL objective that became the 2024–25 default for LLM post-training: sample a group of responses per prompt, use the group's mean reward as the baseline, and optimize the clipped PPO surrogate with no value network — cheaper and more stable than PPO in synchronous training, but the group is an implicit synchronization barrier (updates wait for the slowest member) that mismatches asynchronous and single-trajectory online agentic settings, which is the gap SAO exploits