What they found
The authors look at a known problem in GRPO, a reinforcement learning method that trains reasoning with checkable rewards. When a question is too hard, every answer fails, so there is no learning signal. Earlier fixes add a hint to the question during training, but no hint exists at test time, which creates a mismatch. The authors propose ASH, which treats a hint as a way to steer behavior. They train a steering vector, a small adjustment added to the model's internal activations, so hinted answers become more likely than plain ones. At inference they inject that vector with no hint in the text. On four models (Qwen3-1.7B/4B-Base, Llama-3.2-1B/3B-Instruct), the abstract says ASH consistently improves pass@k on math benchmarks (AMC, OlympiadBench, AIME24, AIME25) and on Minerva and GPQA-Diamond. Average pass@8 rose by 2.60% for Qwen3-1.7B-Base and 2.21% for Qwen3-4B-Base.
Why it matters
Reasoning training often stalls on questions the model cannot yet solve. This work offers a way to keep learning from those questions without needing hints later. The abstract gives no cost or size details beyond the four small models, so how well it scales is not stated.
How it changes the race
This is likely an incremental but useful step for training reasoning in SI models. If it holds, it could help teams get more from hard training questions using small models and little extra machinery.
The abstract, by the authors
GRPO has become the dominant reinforcement learning method for eliciting reasoning capability in LLMs using verifiable rewards. However, it suffers from a fundamental failure mode: when a training question exceeds the base model's current capability, all generated responses fail and receive zero reward, yielding no gradient signal and stalling further capability expansion. An emerging class of GRPO extensions, termed Hint-Augmented GRPO (HA-GRPO), adds a "hint" to the context to reduce the difficulty of the question, thereby producing a usable gradient signal. Nevertheless, existing HA-GRPO methods share a key issue: no hint is available at inference time to append to the question, creating a training-inference context mismatch that leads to inconsistent performance gains. In this work, we reinterpret the function of hinting - not as reducing question difficulty, but as steering model behavior - and propose ASH (Activation Steering for Hinting), a novel HA-GRPO method built on this view. During training, the model is presented with both hinted and plain questions, while a trainable steering vector is optimized to widen the margins between the model's generation probabilities for responses to the hinted version relative to the plain version, thereby capturing the question-agnostic component of the behavioral shift induced by the hint. During inference, the trained steering vector is directly injected into the model's activation, reproducing the hint-conditioned behavior without requiring any hint in the context. Experiments with four LLMs (Qwen3-1.7B/4B-Base, Llama-3.2-1B/3B-Instruct) show that ASH consistently improves pass@k on competition-level mathematics benchmarks (AMC, OlympiadBench, AIME24, AIME25) and out-of-distribution benchmarks (Minerva, GPQA-Diamond). Notably, it improves average pass@8 of Qwen3-1.7B-Base and Qwen3-4B-Base by 2.60% and 2.21%.
Quoted from arXiv:2611.23124. The research and its words belong to its authors.
