Advances in Agentic Computer Graphics

This report proposes a four-dimension taxonomy for LLM-agent systems that perform computer graphics tasks and builds an agentic literature survey system that applies the taxonomy at scale to 350 papers, revealing design trends and open challenges.

Agentic graphics research has grown rapidly and is described with inconsistent terminology across CAD, animation, simulation, and 2D design, so shared design choices are obscured and cross-work comparison is difficult. The authors note that the literature is large and growing at a seemingly exponential rate, with a corpus of more than 300 papers published through September 2026, making manual review hard to keep pace with. The report addresses both the scale and the terminology problems through a unified taxonomy plus an automated literature survey system built on it.

The authors first define agentic graphics systems by three criteria: an LLM makes at least some action decisions, operations are executed in a graphics environment to create or modify content, and feedback on results or execution is used to select or revise subsequent actions. They then introduce a taxonomy with four dimensions: specification (task, input form, output), base model (model type and training), harness (plan and act, feedback mechanism, memory), and graphics interface and tooling (action type, graphics software, tools, resources, representation). The second contribution is an agentic literature survey system that proceeds through paper discovery, evidence-based annotation, and knowledge curation, using dedicated research tools following SWE-agent and recording taxonomy labels, design insights, limitations, and source evidence. Each run is instantiated with the taxonomy, candidate papers, previous records, a date cutoff, and a reading budget. They ran the system with GPT-6 Astra at medium reasoning effort to compile a corpus of 350 papers published between 2024 and September 2026, followed by manual verification of relevance. The taxonomy is then used to organize applications in 2D graphics, static 3D assets, and dynamic 3D assets, and to analyze trends and open problems.

• The survey system compiled and manually verified a corpus of 350 papers published between 2024 and September 2026, recording taxonomy labels, design insights, limitations, and source locations per paper. §4.2.1
• Visual feedback is the most widely used observation type, appearing in 77.0% of the works that use observation as feedback, and 58.2% combine multiple observation categories. §4.2.2
• Local patching as an update strategy becomes increasingly common over the analyzed period, rising from 50.0% to 83.0%. §4.2.2
• Documentation and rules become more prevalent as supporting resources, reaching 85.3% of records in the latest period. §4.2.2
• Task type correlates with update strategy: CAD generation more frequently relies on full regeneration while scene generation more frequently uses local patch updates, and scene generation accounts for the largest share of visual and structural observation assignments whereas CAD accounts for the largest share of runtime observations. §4.2.2
• Across tasks, most existing systems rely on off-the-shelf base models with relatively limited exploration of supervised or reinforcement post-training tailored to graphics tasks, while harness designs exhibit considerably greater diversity; code and structured specifications remain the dominant interaction forms. §10

• The corpus is bounded by a date cutoff and reading budget: the reported run covers papers published between 2024 and September 2026 and requires manual verification of relevance afterward (stated in §4.2.1).
• The trend analysis is reported as four selected patterns out of several notable patterns the authors say the analysis reveals, and some explanations are hedged as possible or may reflect (stated in §4.2.2).
• The abstract lists open challenges the report does not resolve—coordinating agents and tools across graphics tasks, designing interfaces and representations for human–agent collaboration, sustaining long-horizon planning and refinement, and reliably evaluating interactive graphics systems (stated in §Abstract).
• The surveyed systems' evidence is left open in places: the paper notes that CoDimRecon's material settings are not calibrated from real-world measurements and that even a close trajectory match leaves the underlying mechanism underconstrained (stated in §8).

• A system counts as agentic graphics only if an LLM makes at least some action decisions, operations execute in a graphics environment to create or modify content, and feedback on results or execution revises subsequent actions; single-pass LLM generation and feedback used only for training or final evaluation are excluded. §2
• The taxonomy has four dimensions: specification, base model, harness, and graphics interface and tooling, with icon-denoted options reused across the application tables for cross-task comparison. §3
• The harness coordinates planning and acting, feedback mechanisms (verifier plus feedback signal), and memory (retention time and access scope). §3.3
• Action types are code/script, structured specification, tool/function call, and GUI action, classified by the graphics instruction rather than the submission mechanism. §3.4.1
• Graphics environments are categorized as agent-native, agent-adapted, or general-purpose, with source availability, API coverage, and documentation access treated as separate properties. §3.4.2
• Six overlapping representations are distinguished: code, geometry, dynamic state, scene structure and layouts, pixels, and learned state; a workflow may retain several at once, so the representation used for editing and the one exposed through feedback should be identified separately. §3.4.5
• The survey system proceeds through paper discovery, evidence-based annotation, and knowledge curation, providing dedicated research tools following SWE-agent, and records are retained across runs so new papers can be added and annotations revisited. §4
• Open problems and opportunities discussed include coordinating agents and tools across graphics tasks, interfaces and representations for human–agent collaboration, long-horizon planning and refinement, and reliable evaluation of interactive graphics systems. §Abstract