Think Less, Locally: Local Preference Learning for Reasoning Compression

LoCo learns state-conditioned continue-or-conclude preferences from self-generated math traces by freezing overthinking prefixes and training DPO on shorter verified continuations, reducing reasoning length while preserving or improving accuracy.

Large reasoning models often keep deliberating after sufficient information is available, and existing compression methods mainly operate on whole trajectories or modify reasoning content. They provide less direct supervision on the local decision of whether further reasoning is necessary from the current state. The paper argues compression should be framed as a state-aware choice to continue or conclude.

LoCo detects candidate overthinking states from the model's own traces using predictive confidence and a stronger judge model. For each detected state it freezes the reasoning prefix, samples k continuations, keeps the shortest continuation whose answer is symbolically equivalent to the gold answer, and discards points without a valid shorter continuation. It forms preference triples of frozen prefix, shorter verified continuation, and original continuation, retaining pairs with sufficient token savings and size limits and at most K pairs per question. DPO is applied with the shared prefix as context, so the preference is conditioned on the same reasoning state and teaches when to conclude instead of merely favoring shorter trajectories.

• On AIME24, LoCo raises accuracy by +3.34 while reducing tokens by 45.4%; on AIME25 it raises accuracy by +2.50 with 45.2% token reduction. §5.2
• On easier math benchmarks LoCo removes 41.7%–44.1% of generated tokens; on Olympiad, AIME24, and AIME25 it attains highest accuracy, highest AUC, and 44.6%–45.4% length reduction. §5.2
• On MBPP LoCo matches base accuracy while reducing tokens by 36.3% and improving AUC by 1.53 points. §5.3
• On LiveCodeBench and CodeContests LoCo produces shorter completions and improves both accuracy and AUC over all baselines; token reductions are 24.7% and 23.4%. §5.3
• In ablation on AIME24, LoCo reaches 54.17 accuracy with 7364 tokens, while RandCut reaches 48.33 with 7880 tokens and UniCut reaches 48.75 with 7895 tokens. §5.4
• The abstract reports 41.7%–45.4% completion length reduction on mathematics and 23.4%–36.3% on code while maintaining or improving accuracy, with best AUC across all code benchmarks. §Abstract

• No dedicated limitations section is provided; training data are constructed entirely from self-generated mathematical traces, so training on code or other domains is not tested.
• The detector uses predictive confidence and a stronger judge model, but the provided text does not report detector precision, recall, or sensitivity to the judge model.
• Evaluation covers five mathematical and three code benchmarks; broader reasoning domains and failure modes of the local preference pairs are not analyzed.

• Existing fine-grained compression methods still treat the trajectory as the object of compression rather than deciding whether further reasoning is necessary at the current state. §2.2
• LoCo converts self-generated overthinking traces into state-anchored preference triples: frozen prefix, shorter correct continuation, original continuation. §4.1
• Step-level overthink detection screens candidate states with model confidence and verifies them with a stronger judge model. §4.2
• Verified continuation generation samples k continuations from a frozen state and keeps the shortest one symbolically equivalent to the gold answer. §4.3
• Preference pairs are retained only when savings reach δ, prompt and continuation length limits are met, and at most K pairs per question are kept by largest savings. §4.4
• DPO uses the shared prefix as x so the likelihood comparison is conditioned on the identical reasoning state. §4.4
• After math-only training, LoCo transfers to code: on MBPP it matches base accuracy with 36.3% fewer tokens and AUC +1.53. §5.3
• Ablation compares global preference, GPT-5.4 continuations, RandCut, and UniCut; full LoCo gives the strongest overall compression-accuracy trade-off. §5.4