LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data

LongZero lets an LLM improve long-context reasoning from zero external data by first writing structured question-evidence-answer specifications, expanding them into long multi-document contexts, adding self-written retrieval conversations, and training with GRPO.

RL can strengthen long-context reasoning, but it needs long, diverse, challenging, and verifiable training data. Human annotation is expensive, naturally occurring long-context QA data are scarce, and existing synthesis pipelines depend on external corpora or stronger teacher models. Existing RL-from-zero-data methods have worked on short-context tasks but not on long-context tasks.

LongZero uses the policy model to synthesize two kinds of long-context tasks with verifiable answers. For reasoning data, the model first writes a compact fictional-world specification with multi-hop questions, supporting facts, answers, and distractors, then expands the plan into long documents of different genres. For retrieval data, the model writes responses to repeated requests in an MRCR-style format, and a program assembles long conversations with needles at random positions and constructs the answer as a prefix plus the target response. The model then answers each reasoning question multiple times, and only questions that receive both correct and incorrect answers are kept because they give GRPO a non-zero learning signal. Training uses GRPO with a token-level policy-gradient loss and a KL penalty, on a 2:1 mixture of reasoning questions and retrieval conversations.

• LongZero achieves the highest average score on two base models and outperforms RL on Enhanced HotpotQA and teacher-synthesized QA. §5
• For Qwen3-4B-Thinking-2507, 69.3% of filtered questions were always correct, 17.7% were unstable and kept, and 13.0% were always incorrect; for Qwen3-8B the shares were 68.3%, 13.1%, and 18.6%. §3.1.3
• Qwen3-8B keeps a smaller share of its own questions than Qwen3-4B-Thinking-2507, 13.1% versus 17.7%, and therefore needs more synthesis to obtain the same amount of data. §3.1.3
• On Qwen3-4B-Thinking-2507 LBv2, the 2:1 reasoning-plus-retrieval mixture gets 43.34 overall, versus 42.64 for reasoning-only and 41.65 for retrieval-only. §B
• On Qwen3-4B-Thinking-2507 LB1, the 2:1 reasoning-plus-retrieval mixture gets 67.22 average, versus 67.04 for reasoning-only and 64.51 for retrieval-only. §B

• Difficulty filtering cannot detect contradictory specifications whenever the model sometimes reproduces the reference answer; the paper's W81_Q1 case entered the training set, and the paper says a consistency check before expansion would remove such cases.
• The paper states that after structured synthesis the only remaining risk of error lies in expansion, and lists better ways to ensure data correctness as future work.
• Experiments cover two base models and five benchmarks; not every benchmark improves for Qwen3-8B, whose LBv2 score is 32.01 versus 32.11 for the base model, and scaling to larger or stronger models is left open.

• LongZero reverses the usual generation order: the model writes the question, evidence, and answer as a short structured specification before any long text exists, so the answer is fixed before expansion. §1
• Reasoning data are built top-down from fictional worlds: the specification fixes questions, supporting facts, answers, and distractor facts, and a later step expands the plan into multiple long documents. §3.1.1
• Retrieval data follow the MRCR format but use the policy's own responses; a program inserts N needles at random positions and asks for the k-th needle prefixed by a random string, making the answer correct by construction. §3.1.2
• Difficulty filtering keeps only the unstable band of questions that the model sometimes answers correctly and sometimes incorrectly, because always-correct or always-incorrect questions give no GRPO gradient. §3.1.3
• The training set uses 2,304 reasoning questions and 1,152 retrieval conversations in a 2:1 mixture; median reasoning-token counts are 19.4K and 14.6K, and median retrieval-token counts are 11.4K and 11.1K. §3.1.3
• LongZero trains with GRPO using group-normalized outcome rewards, a token-level policy-gradient loss following DAPO, and a per-token KL penalty against the initial policy. §3.2
• The paper reports that gains transfer from fictional to real-world documents and from short training contexts to much longer inputs, and that the two data types are complementary. §5