Zehua Jiang

Research

From learned editors to tool-using agents

These projects explore reinforcement learning for controllable level generation and generalization across maps, open-ended generation from natural-language requests, and reliable execution with local language models.

1. 01

   Learning Controllable 3D Level Generators

   Zehua Jiang, Sam Earle, Michael Cerny Green, Julian Togelius

   Extends PCGRL to Minecraft tasks with jumping and gravity. Editing policies learn structures such as staircases from task feedback and adapt them to different fixed-door configurations, without authored level examples. Control is stronger for shorter path targets and remains limited for jump counts and large vertical separations.

   X / Twitter

   BibTeX

   Copy BibTeX

   @inproceedings{jiang20223DPCG,
   author = {Jiang, Zehua and Earle, Sam and Green, Michael and Togelius, Julian},
   title = {Learning Controllable 3D Level Generators},
   year = {2022},
   isbn = {9781450397957},
   publisher = {Association for Computing Machinery},
   address = {New York, NY, USA},
   url = {https://doi.org/10.1145/3555858.3563273},
   doi = {10.1145/3555858.3563273},
   abstract = {Procedural Content Generation via Reinforcement Learning (PCGRL) foregoes the need for large human-authored data-sets and allows agents to train explicitly on functional constraints, using computable, user-defined measures of quality instead of target output. We explore the application of PCGRL to 3D domains, in which content-generation tasks naturally have greater complexity and potential pertinence to real-world applications. Here, we introduce several PCGRL tasks for the 3D domain, Minecraft. These tasks will challenge RL-based generators using affordances often found in 3D environments, such as jumping, multiple dimensional movement, and gravity. We train agents to optimize each of these tasks to explore the capabilities of existing in PCGRL. The agents are able to generate relatively complex and diverse levels, and generalize to random initial states and control targets. Controllability tests in the presented tasks demonstrate their utility to analyze success and failure for 3D generators. We argue that these generators could serve both as co-creative tools for game designers, and as pre-trained environment generators in curriculum learning for player agents.},
   booktitle = {Proceedings of the 17th International Conference on the Foundations of Digital Games},
   articleno = {71},
   numpages = {9},
   location = {Athens, Greece},
   series = {FDG '22}
   }

   Learned 3D level generation in Minecraft.

2. 02

   Scaling, Control and Generalization in Reinforcement Learning Level Generators

   Sam Earle, Zehua Jiang, Julian Togelius

   We introduce a JAX implementation that parallelizes simulation and learning on the GPU, removing CPU–GPU transfer overhead to substantially accelerate training and enable experiments with one billion timesteps. We also introduce randomized map shapes and fixed key-tile positions for greater control, and show how local observations improve generalization to unseen maps.

   X / Twitter

   BibTeX

   Copy BibTeX

   @INPROCEEDINGS{earle2024pcgrl+,
     author={Earle, Sam and Jiang, Zehua and Togelius, Julian},
     booktitle={2024 IEEE Conference on Games (CoG)},
     title={Scaling, Control and Generalization in Reinforcement Learning Level Generators},
     year={2024},
     volume={},
     number={},
     pages={1-8},
     keywords={Training;Measurement;Procedural generation;Limiting;Shape;Layout;Graphics processing units;procedural content generation;reinforcement learning},
     doi={10.1109/CoG60054.2024.10645598}
   }

   

   PCGRL+ edits a dungeon step by step and generates levels across six map shapes. Select a small animation to enlarge it.

3. 03

   Video Game Level Design as a Multi-Agent Reinforcement Learning Problem

   Sam Earle, Zehua Jiang, Eugene Vinitsky, Julian Togelius

   We formulate level generation as a cooperative multi-agent reinforcement learning problem, with agents jointly editing a shared level. Multiple edits share each reward computation, reducing evaluation overhead. Experiments show improved generation efficiency and generalization to unseen map shapes.

   BibTeX

   Copy BibTeX

   @article{earle2025multiagentpcgrl,
       title={Video Game Level Design as a Multi-Agent Reinforcement Learning Problem},
       volume={21},
       url={https://ojs.aaai.org/index.php/AIIDE/article/view/36807},
       DOI={10.1609/aiide.v21i1.36807},
       abstractNote={Procedural Content Generation via Reinforcement Learning (PCGRL) offers a method for training controllable level designer agents without the need for human datasets, using metrics that serve as proxies for level quality as rewards. Existing PCGRL research focuses on single generator agents, but are bottlenecked by the need to frequently recalculate heuristics of level quality and the agent’s need to navigate around potentially large maps. By framing level generation as a multi-agent problem, we mitigate the efficiency bottleneck of single-agent PCGRL by reducing the number of reward calculations relative to the number of agent actions. We also find that multi-agent level generators are better able to generalize to out-of-distribution map shapes, which we argue is due to the generators’ learning more local, modular design policies. We conclude that treating content generation as a distributed, multi-agent task is beneficial for generating functional artifacts at scale.},
       number={1},
       journal={Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment},
       author={Earle, Sam and Jiang, Zehua and Vinitsky, Eugene and Togelius, Julian},
       year={2025},
       month={Nov.},
       pages={32-42}
   }

   

   Multiple agents cooperatively edit a shared Dungeon level, including a 20-agent generation episode.

4. 04

   Agentic PCG: Procedural Content Generation via Tool-using LLMs

   Zehua Jiang, Sam Earle, Ahmed Khalifa, Julian Togelius

   A language model builds and revises levels through tools, using symbolic observations and feedback after each action. Individual tile edits and higher-level generation algorithms share one interface, with evaluators deciding which revisions to keep. The framework supports computable targets and broader language requests across five game domains without task-specific fine-tuning.

   Project webpage X / Twitter

   BibTeX

   Copy BibTeX

   @article{jiang2026agentic,
     title   = {Agentic PCG: Procedural Content Generation via Tool-using LLMs},
     author  = {Jiang, Zehua and Earle, Sam and Khalifa, Ahmed and Togelius, Julian},
     journal = {Available at SSRN 6499021},
     year    = {2026},
     note    = {Available at SSRN: \url{https://ssrn.com/abstract=6499021}}
   }

   

   Tool-using agents generate and refine levels across game domains. Select an animation to enlarge it.

   For details, see the project webpage.

5. 05

   Enabling Agentic PCG with Harness-Controlled Local Models

   Zehua Jiang, Sam Earle, Ahmed Khalifa, Julian Togelius

   An execution harness enables local language models to recover incomplete tool calls, use feedback, and retain successful level edits without changing their weights. Controlled experiments evaluate the harness and compare high-level PCG tools with tile-by-tile editing on structural design tasks, measuring target matching, structural success, and playability.

   Project webpage

   BibTeX

   Copy BibTeX

   @article{jiang2026enabling,
     title   = {Enabling Agentic PCG with Harness-Controlled Local Models},
     author  = {Jiang, Zehua and Earle, Sam and Khalifa, Ahmed and Togelius, Julian},
     journal = {Available at SSRN 7502203},
     year    = {2026},
     month   = sep,
     doi     = {10.2139/ssrn.7502203},
     url     = {https://ssrn.com/abstract=7502203},
     note    = {Available at SSRN: \url{https://ssrn.com/abstract=7502203}}
   }

   

   Locally deployed models match Zelda path targets and use high-level PCG tools to build caves and rooms while preserving reachability. Select an animation to enlarge it.

   For details, see the project webpage.

6. 06

   Making generation cheaper while keeping control

   Adapting small models

   Exploring whether language models with fewer than four billion parameters can reliably generate and revise levels, bringing natural-language PCG to edge devices with limited memory and compute.

   Reusable level pieces

   Discovering reusable level pieces with diverse functional properties, such as jump patterns, enemy encounters, and item collection. We aim to turn these pieces into generation tools and explore how construction strategies from larger models can be captured in reusable components.

   Player agents for richer feedback

   Developing compact player agents that evaluate levels through gameplay, providing richer feedback on requested behaviors and helping generators produce more varied, playable content.

Full list on Google Scholar.