The Archive

Theory of policy optimization in LM post-training: identifies coverage-preserving algorithms and warns that REINFORCE/PPO/GRPO risk coverage collapse when optimal response probability drops.

On-policy distillation recovers lost agent capabilities across GRPO training checkpoints, improving coverage on agentic tasks without additional compute.

Compares on-policy distillation (OPD) versus GRPO across multiple dimensions, finding OPD generally requires more training FLOPs per accuracy point.

TurnWise applies turn-level credit assignment in GRPO for streaming video understanding, aligning learning with individual decision points instead of trajectory-level rewards.

Adaptive two-stage training framework reduces redundant trajectory re-execution in GUI agent GRPO by selecting informative rollout pivots.

Budget-constrained auditing separates reward-hacking detection from correction in GRPO to prevent verification gaming.

Analysis of rollout diversity in GRPO-trained reasoning models identifies spurious reasoning in low-agreement traces to improve training efficiency.

Correlation-aware exploration with token-level entropy regularization prevents policy collapse in long-trace GRPO reasoning models.

DATC improves token-level credit assignment in GRPO by aligning updates with reward model gradients.

Martingale GRPO refines vision-language-action RL by applying trajectory-level penalties only to causally-linked failure actions, not recovery.

Empirical study comparing LLM-based and classical feature selection on tabular data with GRPO-adapted Qwen and Llama models.

EliteReplay combines GRPO with quality-diversity search to preserve solution diversity in RL post-training rather than converging to single policies.

VASPO integrates privileged value estimation with PPO for improved self-distillation in reasoning models, outperforming GRPO-based approaches.

HP-RLVR assigns token-level credit in single-rollout RL without group sampling by using trajectory hindsight, improving on GRPO.

ReTerm self-improving terminal agent trained on recovery from execution errors, includes ReTerm-10K dataset and ReTerm-GRPO online RL approach.

RAG models exploit faithfulness rewards via copying; GRPO training exacerbates the issue without proper source attribution.

NetRL applies LLM-based reinforcement learning to network decision tasks like bitrate streaming and congestion control with improved generalization.

Cross-elicitation matrices reveal how propensities generalize OOD across prompting, few-shot, SFT, and GRPO techniques via spillover analysis.

BehaviorTrace evaluation harness reveals limits of gradient-based attribution in online RL fine-tuning with GRPO.

BoT-GRPO improves process-reward RL for reasoning via token-level rewards and bag-of-tokens aggregation, eliminating value network overhead.

DiffGate combines token-level and outcome-level supervision for on-policy distillation, integrating difficulty-gated teacher guidance with GRPO-style RL rewards.

GRPO training efficiency improved via Fisher-Rao manifold coordinates for prompt selection, reducing wasteful rollouts on uniformly-correct/incorrect samples.

Controlled audit of Qwen2.5-VL-3B post-training objectives (SFT, LoRA, GRPO) on medical VQA with visual utility analysis beyond accuracy.

Entropy-calibrated credit assignment unifies on-policy distillation and GRPO for mathematical reasoning by preserving task credit across token-level teacher reweighting.

Joint recognition and translation fine-tuning via group relative policy optimization (GRPO) for LLM-based speech translation with Qwen2.5-Omni-3B.

λ-Controlled GRPO stabilizes reinforcement learning for flow-matching image generators by controlling importance ratio drift across denoising steps.

Exact policy optimization for genomic tool selection replaces GRPO sampling with enumeration over combinatorially small tool-subset space, improving scientific reasoning tasks.

DualRead separates answer capability from confidence calibration in medical vision-language models trained via GRPO.

Analysis of Qwen2.5-1.5B rollouts reveals within-group verifier-error correlation of 0.530, suggesting group-based RL with automatic verifiers requires design-effect adjustment for statistical validity.