Mage-VL-Cowork: A Full Training Recipe to Learn Hybrid Computer Use From Evolved Harnesses

Mage-VL-Cowork is a 4B hybrid computer-use agent trained through a five-stage curriculum on 318,837 trajectories generated by evolved GUI and terminal harnesses and reusable workflows, reaching 66.0% on OSWorld-Verified with a 50-step budget.

Computer-use agents restricted to pixels must re-derive through the GUI what a command could accomplish, and they lose a reliable channel for checking their work. Hybrid access alone does not produce hybrid competence, because even frontier teachers show suboptimal channel routing and distilling their default rollouts passes those limits to the student. Prior pipelines post-train checkpoints already specialized for GUI grounding or coding, and sustaining a multi-stage curriculum requires massive, diverse environments and trajectories.

Mage-CUA-Forge builds a skill taxonomy of 6,915 atomic skills, reusable skill-conditioned generation workflows, and application-specific preparers to synthesize task packages containing an instruction, environment setup script, and rubric, with gates checking consistency. Before large-scale rollout, an authoring agent evolves the teacher harness scaffold, tool interfaces, prompts, and terminal environment using task success, token cost, execution feedback, and channel statistics, including pruning redundant tools. The training pipeline is a five-stage curriculum: GUI-oriented midtraining, GUI agent SFT, coding and terminal SFT, hybrid CUA SFT on 318,837 trajectories, and online hybrid RL on 1,988 ScaleCUA tasks with GRPO, frontier sampling, asynchronous rollouts, and truncated importance sampling capped at 2.0. Stage 5 uses 96 task groups with eight episode slots per group, a four-batch admission window, and policy lag bounded at three updates. Student evaluation feedback guides adaptive expansion and refinement of the skill taxonomy across three generation rounds. Training examples use an asymmetric context window retaining the three most recent visual turns and folding earlier steps into Action summaries, with step-level reasoning synthesis for think rationales, Action summaries, and tool calls.

• Mage-VL-Cowork-4B reaches 66.0% on OSWorld-Verified under a 50-step budget and outperforms larger open-source agents including Muse Glimmer-30B, Qwen3.6-27B, EvoCUA-32B, and OpenCUA-72B. §Main Results on OSWorld-Verified
• OSWorld-Verified performance rises from 45.56% after GUI SFT to 55.84% after Hybrid SFT and 66.0% after online RL, a 20.44 percentage-point gain across downstream stages. §Training Progression
• Workflow-first generation reduces recurring per-task expenditure by 82.6%, from $0.906 to $0.158, and token consumption by 93.3%, from 1,276K to 85K tokens. §Task-Generation Expenditure
• Across GUI SFT, Hybrid SFT, and RL, mean model calls decrease from 25.36 to 21.57 and 18.77, and RL reduces model calls by 13.0% and output and think tokens by approximately 25.4%, bringing mean output tokens to 5,499 per task. §Execution Efficiency and Token Usage
• With the teacher model held fixed, the beta1 harness beats the AX-augmented beta2 harness on teacher success and steps, and the beta1 student reaches 37.6% versus 25.1% for beta2 while using 21.5% fewer steps. §Effect of the Evolved Harness on Student Performance
• On WebVoyager-Static, Mage-VL-Cowork-SFT achieves 58.6% success, ranking second behind Fara1.5-4B and outperforming Qwen3-VL-8B-Thinking, Qwen3.5-9B, and OpenWebRL-4B. §Browser Task Performance

• The paper states that operational deployments require explicit authorization and sandbox isolation for terminal execution.
• The reported compute accounting covers language-model training stages and states that base pretraining, vision-encoder computation, and separate attention/recomputation overhead terms are outside its scope.
• OSWorld-Verified evaluation is on 361 held-out tasks with a 50-step budget, excluding Google Drive, and the reported result averages two evaluation runs.
• The paper states that live websites introduce anti-bot blocks, content drift, and non-resettable backend state, so it pairs desktop preparers with self-hosted deterministic web environments.

• The hybrid CUA acts through a GUI channel with visual observation and coordinate control and a terminal channel with a persistent shell for code, command-line utilities, and application libraries. §Introduction
• Harness evolution removes xdotool in favor of native screenshot-grounded GUI actions and adds domain libraries such as python-pptx in the terminal environment, while keeping teacher model weights fixed. §Evolution of the Hybrid Computer-Use Harness
• Mage-CUA-Forge spends authoring effort once per skill by building reusable generation workflows that combine LLM nodes, file preparers, and setup operations, with file-level and package-level gates. §Workflow-First Task and Environment Generation
• The synthesized inventory contains 293,806 tasks covering 6,915 atomic skills, with 61.76% single-application and 38.24% multi-application tasks, yielding 318,837 retained trajectories; GUI and terminal actions are 68.19% and 31.81% of teacher tool calls. §Dataset Statistics
• During Stage 1 GUI-oriented midtraining, screen grounding rises from 30.6% to 56.9% on OSWorld-G and from 78.8% to 87.2% on ScreenSpot-v2. §Training Progression
• Stage 4 trains on the 318,837 retained hybrid trajectories, and the final hybrid dataset contains 1,377,805 packed bins. §Stage 4: Hybrid CUA SFT
• The reference RL deployment provisions 1,120 GUI slots across 70 CPU hosts and 32 B200 GPUs, allocating 24 GPUs to training and eight to inference. §System Architecture and Asynchronous Execution
• Each turn keeps the three most recent visual turns with screenshots and full responses, while earlier steps are folded into imperative Action summaries, and a synthesis pass produces think rationale, Action summary, and verified tool call. §Context Window Design and Step-Level Reasoning Synthesis