Enabling Agentic PCG with
Harness-Controlled Local Models

Motivation

Why Local Models Need a Harness

A designer may ask a PCG system to create connected rooms, lengthen a route, or build a platforming level with a particular number of jumps. Developers may want this to run offline, on a model version they control, with design requests and player data kept on their own machines. Inside a game, the model also has to share memory and compute with rendering and simulation, so it needs to be small enough to run locally. Compact local models meet these requirements, but a model that can describe a level design cannot necessarily execute it. In our experience, they often fail in four ways:

Fine-tuning or distillation could address this, but needs extra data and compute. We instead change how model actions are executed and how level updates are accepted—no model weights are changed. This work extends the tool-using Agentic PCG framework.

Method

The Execution Harness

The local model plans and emits tool calls. Everything after that is owned by the harness: it validates the calls, runs them on a copy of the saved level, measures the candidate, and decides whether to keep it. Measuring a candidate never automatically overwrites the saved level.

Two execution modes

Iterative editing: the model calls primitive tile edits or high-level PCG tools—a room generator, a depth-first cave digger, cellular automata and connectivity repair—through the same checked loop. Target extraction + construction: the model reads a natural-language request and fills structured target fields. The harness checks them for missing, ambiguous or unsupported requirements; a deterministic builder then joins short, ceiling-free level patches that were certified in simulation, so their event counts add up to the target, and an A* player verifies the result.

Reliable Level Editing

From 24/64 to 64/64 Perfect Matches

Each model is held fixed and evaluated on the same 64 Zelda targets: two requested path lengths (player→key and key→door, each 10–40 steps) on a 16×16 map, with one player, key, door, three enemies, and one connected region. An open-loop baseline saves every executable edit with no feedback or history; the full harness adds recovery, measured feedback, history, target-prioritized acceptance, path guidance and early stopping. A run is a perfect match if its final saved map is valid and hits both path lengths exactly.

Same target, three models

What makes the difference?

Action recovery makes more responses executable, but keeping successful edits gives the largest jump: adding acceptance and early stopping raises perfect matches from 20/64 to 56/64—before the model sees any evaluator feedback. Feedback, history and path guidance then fix the remaining errors and bring the result to 64/64. A separate 2×2 comparison shows that either selective acceptance (57/64) or early stopping (59/64) alone is enough to protect successful results, versus 20/64 with neither.

The harness is also cheaper: on the 64 targets it cuts Qwen3.8’s model calls from 256 to 120 and output tokens from 1.80M to 0.42M. On a larger reserved set of 721 Zelda targets, the fixed system reaches 99.03% (714/721) perfect matches with all 721 final maps valid, recovering 658 of 659 truncated responses. In a second domain, BinaryDoor, it reaches 20/32 perfect matches where the open-loop baseline solves none.

High-Level PCG Tools

Revising Levels Without Breaking Them

We ask the model to restructure existing 32×32 Zelda levels—add rooms, caves, mazes, repeating patterns, or reconnect islands—while keeping every entity and route intact. Six request types with three paraphrases each give 18 requests, scored by functional validity plus predefined geometric checks.

High-level tools let the agent express a large structural change in a single call, with 45% fewer tool calls. The execution loop is what keeps the level playable: removing history, numerical feedback and acceptance control leaves six of the 15 initially valid maps broken, including all three dense-maze requests. This protection has a cost—about 850k tokens for the full loop versus 384k for the ablation.

Same request, with and without the loop

Tile edits vs. high-level PCG tools

Language Control

From Requests to Verified Gameplay in Super Mario Bros.

In SMB, the model reads a request and extracts structured targets; the harness rejects incomplete, ambiguous or unsupported targets before anything runs. A deterministic builder then joins short level patches that were certified in simulation, and an A* player verifies what actually happens. We distinguish level-object targets (what the map contains: pipes, enemies, coins, blocks) from gameplay-event targets (what happens during play: coins collected, enemies stomped, jumps from the ground).

Simulation-certified patches

Each candidate patch—enemies behind pits, an obstacle or gap, a row of coins—is played in two to four fixed contexts and kept only if every playthrough reproduces its event counts exactly, with no ceilings and no uncounted takeoffs. Of 496 candidate placements, 67 pass. Stitching them, the builder satisfies 204 of the 210 supported structured targets (up to four stomps, five coins and six jumps), 196 with its first plan and all within nine simulation runs. The six remaining targets request neither stomps nor jumps, which a ceiling-free level cannot produce. An earlier library of 16 whole-level layouts covered only 96 targets, and primitive tile editing satisfies only 3 of 29 development targets even though all its levels are playable.

From language to a verified level

On 24 valid requests in English, Chinese and mixed language, the model extracts every target field correctly for 23/24, and the builder satisfies all 23. Given correct targets, the builder succeeds on all 24, so the remaining error is interpretation; a keyword parser attached to the same builder succeeds on only 12. In the one failure the model left out explicit zero-event requirements—and the harness rejected the incomplete target rather than building the wrong level.

Knowing when not to build

All eight ambiguous or unsupported requests are rejected before execution, for example:

• Needs clarification“Let Mario grab a few coins, beat some enemies, and jump whenever it feels natural.”
• Needs clarification“The final run must collect exactly two coins, but it must also finish with exactly four collected coins.”
• Unsupported“During one run, defeat eight enemies, collect one coin, and make two ground jumps.”
• Unsupported“Build a Zelda dungeon where the hero finds two keys before opening a locked door.”
• Needs clarification“Put three coin tiles in the file and have Mario collect two coins; use whichever count is more convenient.”

Limits: target extraction can only enforce what the target representation captures. On 12 Zelda scene descriptions involving shapes, guard rows, vaults and relative placement, the full system stays functionally valid in 12/12 cases but meets every scene detail in only 1/12. Broader language control needs richer target representations and evaluators.

Deployment

Running on a Laptop

We reran the main studies with Qwen3.8-27B quantized to Q4_K_M in LM Studio on an M1 Max MacBook Pro with 32 GiB of unified memory. The completed-output results closely match the BF16 server runs.

Successful Zelda episodes average 878 s on the laptop, so these results support portability, not real-time in-game generation. The laptop baseline was not rerun and the comparison does not isolate quantization effects. Laptop SMB levels were built with the earlier layout library; rebuilding the 22 correctly extracted targets with the patch-stitching builder succeeds in every case.

Code

Try It Yourself

Local-Deployable-AgenticPCG is self-contained: it bundles the six-domain PCG framework (Binary, BinaryDoor, Zelda, Sokoban, Lode Runner, SMB), the harness controls, fixed benchmark inputs, experiment runners and evaluators. The offline smoke test needs neither a GPU nor model weights.

# Python 3.12, macOS or Linux
python3.12 -m venv .venv && source .venv/bin/activate
python -m pip install -r requirements/client-lock.txt
python -m pip install -e ".[test]"

pcg-reproduce smoke --output runs/smoke          # offline example
pcg-reproduce z64 --model qwen38 --dry-run       # inspect the paired 64-target study

Citation

BibTeX

If you find this work useful in your project/research, please consider citing our preprint:

@article{jiang2026enabling,
  title   = {Enabling Agentic PCG with Harness-Controlled Local Models},
  author  = {Jiang, Zehua and Earle, Sam and Khalifa, Ahmed and Togelius, Julian},
  journal = {Available at SSRN 7502203},
  year    = {2026},
  month   = sep,
  doi     = {10.2139/ssrn.7502203},
  url     = {https://ssrn.com/abstract=7502203},
  note    = {Available at SSRN: \url{https://ssrn.com/abstract=7502203}}
}

@article{jiang2026agentic,
  title   = {Agentic PCG: Procedural Content Generation via Tool-using LLMs},
  author  = {Jiang, Zehua and Earle, Sam and Khalifa, Ahmed and Togelius, Julian},
  journal = {Available at SSRN 6499021},
  year    = {2026},
  note    = {Available at SSRN: \url{https://ssrn.com/abstract=6499021}}
}