A-RSI: Adaptive Task Refresh for Zero-Data Multimodal Self-Improvement

A-RSI uses training-log reward-variation and accuracy signals to decide when to replace a pool of programmatically generated multimodal tasks, and reports gains on an eleven-benchmark suite without external training examples or human answer labels.

Recursive self-improvement requires models to decide when their current training experience has stopped being useful, not only what experience to generate next. Existing methods can keep spending updates on a declined pool or refresh too often, and many use self-consistency or majority voting so supervision depends on the model's uncertain predictions. The paper separates refresh timing from answer correctness: training dynamics determine when to draw new experience, while new tasks should have independently checkable answers.

Before training, A-RSI uses a fixed library of model-written programs to instantiate multimodal tasks; each program samples parameters and produces an image, question, and program-computed answer, with admission, freshness, and contamination checks. The Solver trains with GRPO using correctness, format, and stopping rewards, and only the Solver policy is updated. The controller computes U_w, the number of logged prompts with nonzero within-group advantages, and accuracy a_w, then refreshes when n_w is at least n_min and either U_w falls below a retention threshold relative to a lagged value or accuracy rises by a threshold. A refresh replaces the whole active pool with unused tasks under fixed family quotas while carrying forward model weights; failed windows are retried once from the preceding checkpoint, and a second failure halts. The controller does not consult validation or test scores.

• On the primary eleven-benchmark suite, A-RSI raises the Qwen performance average from the post-initialization checkpoint to the later result with 800 fresh tasks. §Abstract
• On the same eleven-benchmark protocol, A-RSI scores 61.05 with 800 fresh tasks, while every-window refresh scores 60.64 with 3,200. §1
• At the matched sixth continuation window, A-RSI scores 60.40 against 58.35 for continued training on the old pool. §5.3
• At S32, A-RSI exceeds Periodic by +1.15 points and Continuous by +0.40 points while introducing fewer new tasks. §6.1
• On MiMo, the random-refresh endpoint is 60.71 versus 60.01 for continued reuse, a +0.70-point difference, and exceeds every-window and periodic refresh by +0.02 and +0.18. §10
• An independent audit of 1,000 generated tasks reports full answer agreement. §16

• Each arm was trained once; the main comparison spans eight continuation windows and has no 16k evaluation, small score differences lack significance testing, and No-Refresh halted at S31 leaving the pre-registered S32 endpoint unobserved (Section 31).
• The first refresh point was set using existing logs; without a random-refresh control on a fixed three-window schedule, the contribution of history-dependent timing remains entangled with the use of fresh tasks (Section 31).
• MiMo has a positive fresh-versus-retained endpoint difference in one run with small margins over Continuous and Periodic, and its pools were selected at Qwen's difficulty frontier, limiting conclusions about transfer across model families (Section 31).
• The fixed programmatic library covers geometry, charts, tables, and counting but omits function plots, numerical estimation, and spatial tasks; task-family shares and 19 thematic terms were informed by evaluation benchmarks, and the initial pool also contains model-rewritten questions (Section 31).

• Task templates are fixed before training and generate an image, question, and reference answer from the same sampled configuration; the learner receives only the image and question. §3.1
• The reward combines answer correctness, required boxed-answer format, and a stopping term that is 0 for normal completion with a final boxed answer and -1 otherwise. §3.2
• The refresh decision monitors U_w, the count of logged prompts with nonzero within-group advantages, and accuracy a_w, using a minimum observation period and fixed lag and thresholds. §3.3
• A refresh draws unused tasks without replacement under fixed family quotas and keeps the latest accepted model weights; the within-pool window count resets after replacement. §3.4
• In the Qwen run, tasks with nonzero group advantages fall from 239 to 195 over three windows, giving .816 and triggering replacement; the accuracy condition never fires alone. §5.3
• Failure-prioritized, learnable-band, and always-wrong replacement selection trail the every-window random control by -2.63, -1.90, and -2.40 points. §6.2
• On MiMo, advantage counts fall from 245 to 153 on the initial pool and from 199 to 143 on the first replacement, retaining 62.45% and 71.86%; the second pool retains 98.68% and is kept. §6.3