Microsoft and Hugging Face release ThinkingBox-Training, using database-state checks as RL rewards; Qwen3.8-27B reaches 60.78% pass@1 after 50 GRPO steps

Microsoft and Hugging Face published ThinkingBox-Training on GitHub, a framework that turns checks of a terminal database's final state into reward signals for reinforcement learning on agents. The authors report that 50 GRPO training steps lifted Qwen3.8-27B's pass@1 on the 507-task ThinkingBox-Bench from 51.70% to 60.78%. Tasks solved on all 20 attempts rose from 38 to 107, while pass@20 slipped from 453 to 446 tasks. The training tasks were drawn from outside the benchmark.

The practical idea is that a grader you already wrote to test an agent can also be used to train it, which could save teams from building a separate reward model. The gain that matters most is consistency: far more tasks now succeed every time, which is what matters for agents running unattended on database work. The audience is mainly researchers and teams post-training open models. Others are unlikely to act on it yet, and the fall in pass@20 hints that training may narrow the range of things the model can do.

All figures are the authors' own, on their own benchmark, with no independent replication, and the evidence links list is empty in the extraction. The framing that a 27B model can "approach larger models" on consistency is generous, because the cited frontier models (241 and 231 on the same measure) are still more than double its 107. Nothing here shows the gains hold on other benchmarks, other base models or real enterprise databases, and a single run at 50 steps says little about variance.

1. 01Independent reruns of ThinkingBox-Training on other base models or other agent benchmarks, and whether the 20/20 gain and the pass@1 gain survive outside ThinkingBox-Bench.
2. 02Whether the small drop in pass@20 (453 to 446) grows with longer training, which would suggest reliability is being bought with reduced diversity.
3. 03Adoption signals over the next few months: GitHub stars, forks, issues and third-party papers or tools that reuse database-state graders as rewards.