What they found
The authors built ARCHER, a two-stage way to train small language models (under 10B parameters) for math. In stage one, free-form chain-of-thought explanations are turned into typed action traces. These traces hold reasoning steps, calculator calls, results from a deterministic runtime, verification actions, and a final answer. In stage two, reinforcement learning refines the model using rewards for correct answers, tool grounding, process discipline, and trace alignment. Across five standard sub-10B backbones, ARCHER beats the strongest tool-augmented baseline by 5.51 percentage points on macro-average accuracy. Gains include 8.95 points on MATH-500, and 5.33 and 5.34 points on AIME-2024 and AIME-2025. The authors also report benefits on two more 7B backbones that were already trained for reasoning.
Why it matters
Small models are cheap to run, can work locally, and can keep data private. But they often fail at exact math. This work suggests that teaching them to use a calculator and check their own work, instead of imitating explanations, can make them more reliable. The traces are also easier to audit, since each step and tool result is recorded.
How it changes the race
If the results hold, smaller teams could get stronger math reasoning from compact models without needing very large ones. It looks like a solid step for tool-grounded training of small models, not a change in frontier capability.
Safety and control
The auditable traces, with recorded tool calls and verification steps, could make a model's reasoning easier for people to inspect. The abstract does not test safety or misuse directly.
The abstract, by the authors
Small language models are attractive for local, low-cost, and privacy-sensitive deployment, but they remain unreliable on mathematical reasoning tasks that require decomposition, exact computation, tool use, and verification. We introduce ARCHER: Action-trace Reinforcement for Calculator-grounded Hierarchical Executable Reasoning, a two-stage framework for training small language models (SLMs) to produce executable and auditable reasoning trajectories. In the first stage, ARCHER compiles free-form chain-of-thought rationales into typed action traces containing reasoning steps, calculator calls, deterministic runtime observations, verification actions, and final answers. This shifts the supervision target from imitating explanatory text to learning a tool-grounded reasoning policy. In the second stage, ARCHER refines this policy through asymmetric hierarchical reinforcement learning, using grouped and conditionally gated trajectory rewards for answer correctness, tool grounding, process discipline, and trace alignment. Across five standard sub-10B backbones, ARCHER improves over the strongest tool-augmented baseline by 5.51 percentage points on macro-average across benchmark accuracies, including gains of 8.95 points on MATH-500 and 5.33 and 5.34 points on AIME-2024 and AIME-2025. Evaluations on two additional reasoning-specialized 7B backbones further show that executable grounding remains beneficial after reasoning-focused reinforcement learning or distillation. These results demonstrate that executable supervision and reward-guided behavioral refinement provide a general and effective training paradigm for compact mathematical reasoners.
Quoted from arXiv:2611.18494. The research and its words belong to its authors.
