PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse Personas

PersonaCUABench holds 74 tasks fixed while varying the requesting user across 44 personas, and across nine backbones persona requests lower mean task completion from 0.689 to 0.441 while user satisfaction and completion diverge.

Computer-use agents are commonly evaluated against fixed task specifications, but users may omit requirements, add constraints, or require approval before consequential actions. Completing the benchmark instruction therefore does not guarantee serving the person who asked. A fixed specification cannot reveal this gap, so the paper asks whether a CUA serves different people who ask for the same task in their own words.

PersonaCUABench fixes 74 tasks across 11 domains, drawn as subsets of Odysseys and REAL, with 38 information and 36 action tasks. It defines 44 personas from four groups instantiated in all 11 domains, each described by 37 attributes from the MatrAIx schema. A persona agent receives the profile and original instruction, writes the opening request in the user's own words, and answers the CUA's clarifying questions in character; the CUA sees only the dialogue, never the profile or original instruction. Scoring records task completion over requirements, whether each requirement was raised, engaged, and met, and the persona agent's 1-5 satisfaction at delivery and after verification. Nine backbones run in one Chrome harness over the same 370 cells for 3,330 runs, and the paper reruns GPT-6 Astra with the profile visible and with a forced clarification question.

• Persona requests lower mean task completion from 0.689 to 0.441, a 36% relative drop. §3.3
• Every backbone loses task completion under persona requests, with relative loss between 28% and 45%. §4.2
• Among 646 fully successful persona cells, only 62.7% receive the top satisfaction rating and 25.9% receive 3 or lower, versus 84.1% and 9.5% for fully successful original requests. §5.1
• In 59.9% of loss cells the agent engages with raised requirements and still misses at least one, while 11.2% are stops the user requested that the checker scores as failures. §5.2
• Showing the profile decreases information-task completion by 0.027 with 95% CI [-0.051,-0.001] and changes action-task completion by +0.028 with 95% CI [-0.007,+0.069]; forcing a question moves persona completion from 0.434 to 0.421 and original action completion from 0.583 to 0.472. §5.3
• Only 24% of the 609 questions asked by agents target a requirement the user omitted. §5.3

• The Limitations section states that tasks focus on finding information and taking actions on websites, leaving desktop applications, standalone chat, and coding workflows outside its scope.
• The Limitations section states that each episode begins with a single request and focuses on one task, whereas personal agents in real use can accumulate knowledge across interactions; repeated interactions with persistent memory are left to future extensions.
• The Ethical Statement states that constructed personas may encode stereotypes and cannot capture the full diversity within demographic groups, so performance differences should not be generalized to real populations.
• The Ethical Statement states that simulated satisfaction ratings cannot replace feedback from real users; Appendix A.4 also reports that on average 15.4% of profile dimensions are clearly expressed.

• The benchmark fixes 74 tasks across 11 domains and varies only who asks, using 44 personas in four groups. §3
• A persona agent uses a 37-attribute profile to rewrite the request and answer clarification questions in character, while the CUA sees only the dialogue. §3.3
• Scoring labels each requirement as raised, engaged, and met to locate lost requirements alongside task completion and satisfaction. §2.3
• Across nine backbones all models lose under persona requests, and relative loss lies between 28% and 45%. §4.2
• Most loss cells arise after the agent engages with a raised requirement: 59.9% engage and still miss, while 11.2% are requested stops scored as failures. §5.2
• Completion and satisfaction diverge: among 646 fully successful persona cells, 62.7% get top satisfaction versus 84.1% for original requests. §5.1
• On GPT-6 Astra, showing the profile or forcing a question does not recover persona task completion, and only 24% of 609 questions target an omitted requirement. §5.3
• Persona fidelity is partial: 15.4% of dimensions are clearly expressed on average, and removing contradictions changes the mean delta from -0.248 to -0.249. §A.4