DecisionRank: On-Policy Distillation for Decision-Model Rerankers

DecisionRank trains a single-pass decision reranker by combining offline RankNet supervision with item-space on-policy distillation from a generative teacher that provides conditional feedback over candidate items.

Generative listwise rerankers autoregressively generate document IDs, which incurs substantial inference latency. Single-token decision rerankers avoid this by sorting first-token document-ID logits, but transferring a generative teacher’s ranking knowledge to these scores is difficult because the teacher expresses preferences as ranked ID sequences while the student ranks with a static score vector in one forward pass. Static offline rankings cannot adapt to the student’s evolving predictions, and naive token-level OPD mismatches sequential feedback with single-pass scoring.

DecisionRank assigns each candidate a distinct single-token document ID and sorts first-token logits to rank all candidates. For item-space OPD, it selects the student’s current top-K candidates and uses their order as a ranking prefix; the frozen teacher evaluates the remaining candidates at each step in one teacher-forced forward pass. The student’s first-token scores define Plackett–Luce choice distributions over the same remaining candidate set, and the teacher’s conditional scores are normalized over that set. The two distributions are aligned with reverse KL divergence at designated steps, directly supervising the first-token scores used at inference. Training jointly optimizes this on-policy objective with offline RankNet supervision from teacher-generated rankings. At inference the student ranks the full candidate set in a single forward pass without autoregressive generation.

• The 8B DecisionRank student outperforms listwise DirectRank-8B on both benchmarks while running 24.5× faster on TREC DL and 16.7× faster on BRIGHT. §3.2.2
• With Qwen3.8-27B as teacher, the 1.7B student reaches 63.89 TREC DL average nDCG@10×100 versus the teacher’s 62.97. §3.5
• Joint RankNet + Item-space OPD improves over RankNet for Qwen3-4B on TREC DL across seeds 42, 43, and 44, reaching 64.25, 64.32, and 64.06 versus 63.18, 63.13, and 63.17 with p<0.001. §C.1
• With the Qwen3.8-27B teacher, the 8B student improves BEIR from 46.99 zero-shot to 57.98, a gain of 10.99. §3.5
• Both DecisionRank students achieve lower ECE than their teachers on TREC DL and BRIGHT without a dedicated calibration objective. §3.2.3
• On BRIGHT, the 8B student averages 36.86 nDCG@10×100 versus DirectRank-8B at 34.84. §B

• The method assumes N≤100 candidates and assigns IDs from a pool of 100 single-token IDs; experiments leave open scaling to larger candidate sets.
• Latency comparisons are not fully hardware-controlled: local models are measured on an NVIDIA B200 while Jev uses median server-reported latency.
• Calibration is evaluated only on TREC DL and BRIGHT and without a dedicated calibration objective; calibration on other benchmarks is left open.
• Seed-variance and significance tests are reported only for Qwen3-4B and Qwen3-8B with the Qwen3.8-27B teacher; other sizes and teachers are left open.

• DecisionRank targets structured reranking by directly predicting scores rather than autoregressive decoding. §Abstract
• The student assigns candidates single-token IDs and sorts first-token logits; N≤100 and IDs come from a pool of 100 single-token IDs. §2.1
• Item-space alignment compares teacher and student choice probabilities over the same candidate items, so vocabularies or architectures need not match. §2.1
• On-policy feedback uses the student’s current top-K ranking as the prefix and the teacher evaluates remaining candidates in one teacher-forced pass. §2.2
• The student distribution is Plackett–Luce over remaining candidates from first-token scores; reverse KL aligns it with teacher conditional choices. §2.2
• Joint training combines RankNet offline supervision with λ-weighted item-space OPD on the same first-token scores. §2.3
• Item-space OPD is compared with Offline KL, AR SFT, AR OPD, and Listwise GRPO on top of a common RankNet loss. §3.3
• DecisionRank students form the effectiveness–latency Pareto frontier among locally evaluated models on TREC DL and BRIGHT. §3.2.2